2024-02-13T03:02:16.998 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-13T03:02:16.998 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7557169 --owner scheduled_lflores@teuthology --verbose --name lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi --description rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} -- /home/teuthworker/archive/lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7557169/orig.config.yaml 2024-02-13T03:02:17.028 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-13T03:02:17.102 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7557169 branch: wip-yuri10-testing-2024-02-08-0854-pacific description: rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} email: null first_in_suite: false job_id: '7557169' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: wip-yuri10-testing-2024-02-08-0854-pacific ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(MON_DOWN\) - missing hit_sets - do not have an application enabled - application not enabled on pool - mons down mon_bind_msgr2: false sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 extra_packages: - cephadm workunit: branch: wip-tracker-64343-pacific sha1: 84e714279aa684636ac645daf0a1a85c2094cfb3 owner: scheduled_lflores@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 964 sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-tracker-64343-pacific suite_path: /home/teuthworker/src/github.com_ljflores_ceph_84e714279aa684636ac645daf0a1a85c2094cfb3/qa suite_relpath: qa suite_repo: https://github.com/ljflores/ceph.git suite_sha1: 84e714279aa684636ac645daf0a1a85c2094cfb3 targets: smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSAlWjESMc0Gra5f/6dvvbjuYJc2qVK13Ztw1gt4S0hdg9pLl/eAcLsaHQz3OAGEzcbKS0DF4bknjuwe5HHhPE= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSAlWjESMc0Gra5f/6dvvbjuYJc2qVK13Ztw1gt4S0hdg9pLl/eAcLsaHQz3OAGEzcbKS0DF4bknjuwe5HHhPE= tasks: - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-12_21:56:27 tube: smithi user: lflores verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-13T03:02:17.102 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ljflores_ceph_84e714279aa684636ac645daf0a1a85c2094cfb3/qa; will attempt to use it 2024-02-13T03:02:17.103 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ljflores_ceph_84e714279aa684636ac645daf0a1a85c2094cfb3/qa/tasks 2024-02-13T03:02:17.103 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-13T03:02:17.104 INFO:teuthology.task.internal:Checking packages... 2024-02-13T03:02:17.127 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '0e714d9a4bd2a821113e6318adb87bd06cf81ec1' 2024-02-13T03:02:17.127 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-13T03:02:17.127 INFO:teuthology.packaging:ref: None 2024-02-13T03:02:17.127 INFO:teuthology.packaging:tag: None 2024-02-13T03:02:17.128 INFO:teuthology.packaging:branch: wip-yuri10-testing-2024-02-08-0854-pacific 2024-02-13T03:02:17.128 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-13T03:02:17.128 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=wip-yuri10-testing-2024-02-08-0854-pacific 2024-02-13T03:02:17.360 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-521-g0e714d9a-1bionic 2024-02-13T03:02:17.362 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-13T03:02:17.369 INFO:teuthology.task.internal:no buildpackages task found 2024-02-13T03:02:17.369 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-13T03:02:17.376 INFO:teuthology.task.internal:Saving configuration 2024-02-13T03:02:17.387 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-13T03:02:17.393 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-13T03:02:17.417 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7557169', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '18.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 02:56:43.344395', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII4U6ItCfOdoN1WdROC3KO40lcw/kzXeIM8M8q950kfL'} 2024-02-13T03:02:17.443 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7557169', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '18.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 02:56:43.342707', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII4U6ItCfOdoN1WdROC3KO40lcw/kzXeIM8M8q950kfL'} 2024-02-13T03:02:17.443 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-13T03:02:17.449 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-13T03:02:17.449 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-02-13T03:02:17.449 INFO:teuthology.run_tasks:Running task console_log... 2024-02-13T03:02:17.519 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f326321e0d0>, signals=[15]) 2024-02-13T03:02:17.519 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-13T03:02:17.525 INFO:teuthology.task.internal:Opening connections... 2024-02-13T03:02:17.525 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-02-13T03:02:17.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T03:02:17.605 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-02-13T03:02:17.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T03:02:17.684 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-13T03:02:17.693 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-02-13T03:02:17.728 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-02-13T03:02:17.728 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-02-13T03:02:17.780 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Ubuntu" 2024-02-13T03:02:17.780 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2024-02-13T03:02:17.780 INFO:teuthology.orchestra.run.smithi161.stdout:ID=ubuntu 2024-02-13T03:02:17.780 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE=debian 2024-02-13T03:02:17.780 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2024-02-13T03:02:17.780 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="18.04" 2024-02-13T03:02:17.780 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.ubuntu.com/" 2024-02-13T03:02:17.781 INFO:teuthology.orchestra.run.smithi161.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-02-13T03:02:17.781 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-02-13T03:02:17.781 INFO:teuthology.orchestra.run.smithi161.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-02-13T03:02:17.781 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_CODENAME=bionic 2024-02-13T03:02:17.781 INFO:teuthology.orchestra.run.smithi161.stdout:UBUNTU_CODENAME=bionic 2024-02-13T03:02:17.781 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-02-13T03:02:17.799 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-02-13T03:02:17.806 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-02-13T03:02:17.807 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-02-13T03:02:17.853 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="Ubuntu" 2024-02-13T03:02:17.853 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2024-02-13T03:02:17.854 INFO:teuthology.orchestra.run.smithi183.stdout:ID=ubuntu 2024-02-13T03:02:17.854 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE=debian 2024-02-13T03:02:17.854 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2024-02-13T03:02:17.854 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="18.04" 2024-02-13T03:02:17.854 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://www.ubuntu.com/" 2024-02-13T03:02:17.854 INFO:teuthology.orchestra.run.smithi183.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-02-13T03:02:17.854 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-02-13T03:02:17.854 INFO:teuthology.orchestra.run.smithi183.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-02-13T03:02:17.854 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_CODENAME=bionic 2024-02-13T03:02:17.854 INFO:teuthology.orchestra.run.smithi183.stdout:UBUNTU_CODENAME=bionic 2024-02-13T03:02:17.854 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-02-13T03:02:17.874 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-13T03:02:17.882 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-13T03:02:17.889 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-13T03:02:17.890 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-02-13T03:02:17.892 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-02-13T03:02:17.901 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-13T03:02:17.907 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-13T03:02:17.907 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-02-13T03:02:17.940 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-02-13T03:02:17.958 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-13T03:02:18.002 INFO:teuthology.run_tasks:Running task kernel... 2024-02-13T03:02:18.017 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-13T03:02:18.017 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-13T03:02:18.017 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-13T03:02:18.018 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-13T03:02:18.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T03:02:18.023 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-02-13T03:02:18.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T03:02:18.024 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-02-13T03:02:18.072 INFO:teuthology.orchestra.run.smithi161.stdout:4.15.0-200-generic 2024-02-13T03:02:18.073 INFO:teuthology.task.kernel:Running kernel on smithi161: 4.15.0-200-generic 2024-02-13T03:02:18.073 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get clean 2024-02-13T03:02:18.074 INFO:teuthology.orchestra.run.smithi183.stdout:4.15.0-200-generic 2024-02-13T03:02:18.074 INFO:teuthology.task.kernel:Running kernel on smithi183: 4.15.0-200-generic 2024-02-13T03:02:18.074 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get clean 2024-02-13T03:02:18.170 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get update 2024-02-13T03:02:18.176 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2024-02-13T03:02:18.320 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2024-02-13T03:02:18.326 INFO:teuthology.orchestra.run.smithi161.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2024-02-13T03:02:18.376 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2024-02-13T03:02:18.376 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2024-02-13T03:02:18.382 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2024-02-13T03:02:18.412 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [83.3 kB] 2024-02-13T03:02:18.450 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2024-02-13T03:02:18.463 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [83.3 kB] 2024-02-13T03:02:18.644 INFO:teuthology.orchestra.run.smithi161.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,665 kB] 2024-02-13T03:02:18.726 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [1,379 kB] 2024-02-13T03:02:18.790 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [3,045 kB] 2024-02-13T03:02:18.851 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [554 kB] 2024-02-13T03:02:18.862 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [39.7 kB] 2024-02-13T03:02:18.862 INFO:teuthology.orchestra.run.smithi161.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [1,347 kB] 2024-02-13T03:02:18.879 INFO:teuthology.orchestra.run.smithi161.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [187 kB] 2024-02-13T03:02:18.881 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,665 kB] 2024-02-13T03:02:18.882 INFO:teuthology.orchestra.run.smithi161.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,915 kB] 2024-02-13T03:02:18.912 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [2,717 kB] 2024-02-13T03:02:18.916 INFO:teuthology.orchestra.run.smithi161.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,663 kB] 2024-02-13T03:02:18.939 INFO:teuthology.orchestra.run.smithi161.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [421 kB] 2024-02-13T03:02:18.948 INFO:teuthology.orchestra.run.smithi161.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [11.2 kB] 2024-02-13T03:02:18.948 INFO:teuthology.orchestra.run.smithi161.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [25.6 kB] 2024-02-13T03:02:18.949 INFO:teuthology.orchestra.run.smithi161.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,088 B] 2024-02-13T03:02:18.949 INFO:teuthology.orchestra.run.smithi161.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [53.3 kB] 2024-02-13T03:02:18.950 INFO:teuthology.orchestra.run.smithi161.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-backports/main i386 Packages [53.2 kB] 2024-02-13T03:02:18.951 INFO:teuthology.orchestra.run.smithi161.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [18.2 kB] 2024-02-13T03:02:18.952 INFO:teuthology.orchestra.run.smithi161.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-backports/universe i386 Packages [18.1 kB] 2024-02-13T03:02:18.961 INFO:teuthology.orchestra.run.smithi161.stdout:Get:21 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [1,379 kB] 2024-02-13T03:02:18.994 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [467 kB] 2024-02-13T03:02:19.000 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [33.0 kB] 2024-02-13T03:02:19.000 INFO:teuthology.orchestra.run.smithi183.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [1,317 kB] 2024-02-13T03:02:19.023 INFO:teuthology.orchestra.run.smithi183.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [182 kB] 2024-02-13T03:02:19.026 INFO:teuthology.orchestra.run.smithi183.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,303 kB] 2024-02-13T03:02:19.047 INFO:teuthology.orchestra.run.smithi183.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [1,078 kB] 2024-02-13T03:02:19.066 INFO:teuthology.orchestra.run.smithi183.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [308 kB] 2024-02-13T03:02:19.071 INFO:teuthology.orchestra.run.smithi183.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [19.8 kB] 2024-02-13T03:02:19.072 INFO:teuthology.orchestra.run.smithi183.stdout:Get:16 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [6,008 B] 2024-02-13T03:02:19.072 INFO:teuthology.orchestra.run.smithi183.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [3,928 B] 2024-02-13T03:02:19.101 INFO:teuthology.orchestra.run.smithi183.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [3,045 kB] 2024-02-13T03:02:19.172 INFO:teuthology.orchestra.run.smithi183.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [554 kB] 2024-02-13T03:02:19.179 INFO:teuthology.orchestra.run.smithi183.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [1,347 kB] 2024-02-13T03:02:19.198 INFO:teuthology.orchestra.run.smithi183.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [39.7 kB] 2024-02-13T03:02:19.199 INFO:teuthology.orchestra.run.smithi183.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [187 kB] 2024-02-13T03:02:19.201 INFO:teuthology.orchestra.run.smithi183.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,915 kB] 2024-02-13T03:02:19.228 INFO:teuthology.orchestra.run.smithi183.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,663 kB] 2024-02-13T03:02:19.250 INFO:teuthology.orchestra.run.smithi183.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [421 kB] 2024-02-13T03:02:19.255 INFO:teuthology.orchestra.run.smithi183.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [25.6 kB] 2024-02-13T03:02:19.256 INFO:teuthology.orchestra.run.smithi183.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [11.2 kB] 2024-02-13T03:02:19.256 INFO:teuthology.orchestra.run.smithi183.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,088 B] 2024-02-13T03:02:19.257 INFO:teuthology.orchestra.run.smithi183.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic-backports/main i386 Packages [53.2 kB] 2024-02-13T03:02:19.257 INFO:teuthology.orchestra.run.smithi183.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [53.3 kB] 2024-02-13T03:02:19.259 INFO:teuthology.orchestra.run.smithi183.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic-backports/universe i386 Packages [18.1 kB] 2024-02-13T03:02:19.259 INFO:teuthology.orchestra.run.smithi183.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [18.2 kB] 2024-02-13T03:02:19.417 INFO:teuthology.orchestra.run.smithi161.stdout:Get:22 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [2,717 kB] 2024-02-13T03:02:19.554 INFO:teuthology.orchestra.run.smithi161.stdout:Get:23 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [467 kB] 2024-02-13T03:02:19.569 INFO:teuthology.orchestra.run.smithi161.stdout:Get:24 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [33.0 kB] 2024-02-13T03:02:19.571 INFO:teuthology.orchestra.run.smithi161.stdout:Get:25 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [1,317 kB] 2024-02-13T03:02:19.604 INFO:teuthology.orchestra.run.smithi161.stdout:Get:26 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [182 kB] 2024-02-13T03:02:19.607 INFO:teuthology.orchestra.run.smithi161.stdout:Get:27 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,303 kB] 2024-02-13T03:02:19.650 INFO:teuthology.orchestra.run.smithi161.stdout:Get:28 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [1,078 kB] 2024-02-13T03:02:19.683 INFO:teuthology.orchestra.run.smithi161.stdout:Get:29 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [308 kB] 2024-02-13T03:02:19.690 INFO:teuthology.orchestra.run.smithi161.stdout:Get:30 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [19.8 kB] 2024-02-13T03:02:19.854 INFO:teuthology.orchestra.run.smithi161.stdout:Get:31 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [6,008 B] 2024-02-13T03:02:19.936 INFO:teuthology.orchestra.run.smithi161.stdout:Get:32 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [3,928 B] 2024-02-13T03:02:21.609 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 20.1 MB in 3s (6,321 kB/s) 2024-02-13T03:02:21.625 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 20.1 MB in 3s (6,345 kB/s) 2024-02-13T03:02:23.277 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-02-13T03:02:23.318 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-02-13T03:02:23.327 DEBUG:teuthology.orchestra.run.smithi161:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-02-13T03:02:23.370 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-02-13T03:02:23.408 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-02-13T03:02:23.454 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-02-13T03:02:23.701 INFO:teuthology.orchestra.run.smithi161.stdout:Building dependency tree... 2024-02-13T03:02:23.702 INFO:teuthology.orchestra.run.smithi161.stdout:Reading state information... 2024-02-13T03:02:23.740 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2024-02-13T03:02:23.741 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2024-02-13T03:02:23.917 INFO:teuthology.orchestra.run.smithi161.stdout:The following additional packages will be installed: 2024-02-13T03:02:23.917 INFO:teuthology.orchestra.run.smithi161.stdout: linux-generic linux-headers-4.15.0-213 linux-headers-4.15.0-213-generic 2024-02-13T03:02:23.917 INFO:teuthology.orchestra.run.smithi161.stdout: linux-headers-generic linux-image-4.15.0-213-generic 2024-02-13T03:02:23.918 INFO:teuthology.orchestra.run.smithi161.stdout: linux-modules-4.15.0-213-generic linux-modules-extra-4.15.0-213-generic 2024-02-13T03:02:23.920 INFO:teuthology.orchestra.run.smithi161.stdout:Suggested packages: 2024-02-13T03:02:23.920 INFO:teuthology.orchestra.run.smithi161.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2024-02-13T03:02:23.920 INFO:teuthology.orchestra.run.smithi161.stdout:Recommended packages: 2024-02-13T03:02:23.920 INFO:teuthology.orchestra.run.smithi161.stdout: thermald 2024-02-13T03:02:23.961 INFO:teuthology.orchestra.run.smithi183.stdout:The following additional packages will be installed: 2024-02-13T03:02:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: linux-generic linux-headers-4.15.0-213 linux-headers-4.15.0-213-generic 2024-02-13T03:02:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: linux-headers-generic linux-image-4.15.0-213-generic 2024-02-13T03:02:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: linux-modules-4.15.0-213-generic linux-modules-extra-4.15.0-213-generic 2024-02-13T03:02:23.964 INFO:teuthology.orchestra.run.smithi183.stdout:Suggested packages: 2024-02-13T03:02:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2024-02-13T03:02:23.964 INFO:teuthology.orchestra.run.smithi183.stdout:Recommended packages: 2024-02-13T03:02:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: thermald 2024-02-13T03:02:23.985 INFO:teuthology.orchestra.run.smithi161.stdout:The following NEW packages will be installed: 2024-02-13T03:02:23.985 INFO:teuthology.orchestra.run.smithi161.stdout: linux-headers-4.15.0-213 linux-headers-4.15.0-213-generic 2024-02-13T03:02:23.985 INFO:teuthology.orchestra.run.smithi161.stdout: linux-image-4.15.0-213-generic linux-modules-4.15.0-213-generic 2024-02-13T03:02:23.985 INFO:teuthology.orchestra.run.smithi161.stdout: linux-modules-extra-4.15.0-213-generic 2024-02-13T03:02:23.988 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages will be upgraded: 2024-02-13T03:02:23.988 INFO:teuthology.orchestra.run.smithi161.stdout: linux-generic linux-headers-generic linux-image-generic 2024-02-13T03:02:24.029 INFO:teuthology.orchestra.run.smithi183.stdout:The following NEW packages will be installed: 2024-02-13T03:02:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: linux-headers-4.15.0-213 linux-headers-4.15.0-213-generic 2024-02-13T03:02:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-4.15.0-213-generic linux-modules-4.15.0-213-generic 2024-02-13T03:02:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: linux-modules-extra-4.15.0-213-generic 2024-02-13T03:02:24.031 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages will be upgraded: 2024-02-13T03:02:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: linux-generic linux-headers-generic linux-image-generic 2024-02-13T03:02:24.053 INFO:teuthology.orchestra.run.smithi161.stdout:3 upgraded, 5 newly installed, 0 to remove and 165 not upgraded. 2024-02-13T03:02:24.053 INFO:teuthology.orchestra.run.smithi161.stdout:Need to get 67.4 MB of archives. 2024-02-13T03:02:24.054 INFO:teuthology.orchestra.run.smithi161.stdout:After this operation, 351 MB of additional disk space will be used. 2024-02-13T03:02:24.054 INFO:teuthology.orchestra.run.smithi161.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-213-generic amd64 4.15.0-213.224 [13.4 MB] 2024-02-13T03:02:24.105 INFO:teuthology.orchestra.run.smithi183.stdout:3 upgraded, 5 newly installed, 0 to remove and 165 not upgraded. 2024-02-13T03:02:24.105 INFO:teuthology.orchestra.run.smithi183.stdout:Need to get 67.4 MB of archives. 2024-02-13T03:02:24.105 INFO:teuthology.orchestra.run.smithi183.stdout:After this operation, 351 MB of additional disk space will be used. 2024-02-13T03:02:24.105 INFO:teuthology.orchestra.run.smithi183.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-213-generic amd64 4.15.0-213.224 [13.4 MB] 2024-02-13T03:02:24.672 INFO:teuthology.orchestra.run.smithi161.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-213-generic amd64 4.15.0-213.224 [8,111 kB] 2024-02-13T03:02:24.818 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-213-generic amd64 4.15.0-213.224 [33.7 MB] 2024-02-13T03:02:25.308 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-213-generic amd64 4.15.0-213.224 [8,111 kB] 2024-02-13T03:02:25.565 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.213.196 [1,864 B] 2024-02-13T03:02:25.565 INFO:teuthology.orchestra.run.smithi161.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.213.196 [2,460 B] 2024-02-13T03:02:25.565 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-213 all 4.15.0-213.224 [11.0 MB] 2024-02-13T03:02:25.744 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-213-generic amd64 4.15.0-213.224 [1,250 kB] 2024-02-13T03:02:25.750 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-213-generic amd64 4.15.0-213.224 [33.7 MB] 2024-02-13T03:02:25.765 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.213.196 [2,360 B] 2024-02-13T03:02:26.480 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 67.4 MB in 2s (38.3 MB/s) 2024-02-13T03:02:26.660 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-modules-4.15.0-213-generic. 2024-02-13T03:02:26.760 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.213.196 [1,864 B] 2024-02-13T03:02:26.760 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.213.196 [2,460 B] 2024-02-13T03:02:26.760 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-213 all 4.15.0-213.224 [11.0 MB] 2024-02-13T03:02:26.937 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-213-generic amd64 4.15.0-213.224 [1,250 kB] 2024-02-13T03:02:26.956 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.213.196 [2,360 B] 2024-02-13T03:02:27.600 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 135963 files and directories currently installed.) 2024-02-13T03:02:27.605 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../0-linux-modules-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-13T03:02:27.651 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 67.4 MB in 3s (23.2 MB/s) 2024-02-13T03:02:27.681 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-modules-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:27.828 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-modules-4.15.0-213-generic. 2024-02-13T03:02:28.724 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 135963 files and directories currently installed.) 2024-02-13T03:02:28.728 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../0-linux-modules-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-13T03:02:28.799 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-modules-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:29.389 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-image-4.15.0-213-generic. 2024-02-13T03:02:29.404 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../1-linux-image-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-13T03:02:29.495 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-image-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:30.017 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-213-generic. 2024-02-13T03:02:30.031 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-13T03:02:30.067 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-modules-extra-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:30.473 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-image-4.15.0-213-generic. 2024-02-13T03:02:30.489 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../1-linux-image-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-13T03:02:30.586 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-image-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:31.101 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-213-generic. 2024-02-13T03:02:31.116 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-13T03:02:31.159 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-modules-extra-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:33.907 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../3-linux-generic_4.15.0.213.196_amd64.deb ... 2024-02-13T03:02:34.009 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-generic (4.15.0.213.196) over (4.15.0.200.183) ... 2024-02-13T03:02:34.352 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.213.196_amd64.deb ... 2024-02-13T03:02:34.479 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-image-generic (4.15.0.213.196) over (4.15.0.200.183) ... 2024-02-13T03:02:34.790 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-headers-4.15.0-213. 2024-02-13T03:02:34.804 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../5-linux-headers-4.15.0-213_4.15.0-213.224_all.deb ... 2024-02-13T03:02:34.848 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-headers-4.15.0-213 (4.15.0-213.224) ... 2024-02-13T03:02:34.943 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../3-linux-generic_4.15.0.213.196_amd64.deb ... 2024-02-13T03:02:35.068 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-generic (4.15.0.213.196) over (4.15.0.200.183) ... 2024-02-13T03:02:35.404 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.213.196_amd64.deb ... 2024-02-13T03:02:35.513 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-image-generic (4.15.0.213.196) over (4.15.0.200.183) ... 2024-02-13T03:02:35.824 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-headers-4.15.0-213. 2024-02-13T03:02:35.837 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../5-linux-headers-4.15.0-213_4.15.0-213.224_all.deb ... 2024-02-13T03:02:35.882 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-4.15.0-213 (4.15.0-213.224) ... 2024-02-13T03:02:39.568 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-headers-4.15.0-213-generic. 2024-02-13T03:02:39.592 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../6-linux-headers-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-13T03:02:39.634 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-headers-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:40.555 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-headers-4.15.0-213-generic. 2024-02-13T03:02:40.579 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../6-linux-headers-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-13T03:02:40.613 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:41.984 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.213.196_amd64.deb ... 2024-02-13T03:02:42.136 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-headers-generic (4.15.0.213.196) over (4.15.0.200.183) ... 2024-02-13T03:02:42.450 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-modules-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:42.947 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.213.196_amd64.deb ... 2024-02-13T03:02:43.058 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-image-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:43.076 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-generic (4.15.0.213.196) over (4.15.0.200.183) ... 2024-02-13T03:02:43.400 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-modules-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:43.564 INFO:teuthology.orchestra.run.smithi161.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-4.15.0-200-generic 2024-02-13T03:02:43.564 INFO:teuthology.orchestra.run.smithi161.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-4.15.0-200-generic 2024-02-13T03:02:43.564 INFO:teuthology.orchestra.run.smithi161.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-213-generic 2024-02-13T03:02:43.564 INFO:teuthology.orchestra.run.smithi161.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-213-generic 2024-02-13T03:02:43.693 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-modules-extra-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:44.018 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-image-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:44.245 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-headers-4.15.0-213 (4.15.0-213.224) ... 2024-02-13T03:02:44.371 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-image-generic (4.15.0.213.196) ... 2024-02-13T03:02:44.497 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-headers-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:44.522 INFO:teuthology.orchestra.run.smithi183.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-4.15.0-200-generic 2024-02-13T03:02:44.522 INFO:teuthology.orchestra.run.smithi183.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-4.15.0-200-generic 2024-02-13T03:02:44.522 INFO:teuthology.orchestra.run.smithi183.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-213-generic 2024-02-13T03:02:44.522 INFO:teuthology.orchestra.run.smithi183.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-213-generic 2024-02-13T03:02:44.641 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-headers-generic (4.15.0.213.196) ... 2024-02-13T03:02:44.645 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-modules-extra-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:45.172 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-4.15.0-213 (4.15.0-213.224) ... 2024-02-13T03:02:45.298 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-image-generic (4.15.0.213.196) ... 2024-02-13T03:02:45.424 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:45.525 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-generic (4.15.0.213.196) ... 2024-02-13T03:02:45.550 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-generic (4.15.0.213.196) ... 2024-02-13T03:02:45.667 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for linux-image-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:45.693 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-generic (4.15.0.213.196) ... 2024-02-13T03:02:45.819 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for linux-image-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-13T03:02:45.884 INFO:teuthology.orchestra.run.smithi161.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-02-13T03:02:45.884 INFO:teuthology.orchestra.run.smithi161.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-213-generic 2024-02-13T03:02:46.035 INFO:teuthology.orchestra.run.smithi183.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-02-13T03:02:46.035 INFO:teuthology.orchestra.run.smithi183.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-213-generic 2024-02-13T03:03:01.072 INFO:teuthology.orchestra.run.smithi183.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-02-13T03:03:01.072 INFO:teuthology.orchestra.run.smithi183.stdout:Sourcing file `/etc/default/grub' 2024-02-13T03:03:01.093 INFO:teuthology.orchestra.run.smithi183.stdout:Generating grub configuration file ... 2024-02-13T03:03:01.488 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-4.15.0-213-generic 2024-02-13T03:03:01.543 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-4.15.0-213-generic 2024-02-13T03:03:01.954 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-4.15.0-200-generic 2024-02-13T03:03:01.982 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-4.15.0-200-generic 2024-02-13T03:03:02.057 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-4.15.0-194-generic 2024-02-13T03:03:02.083 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-4.15.0-194-generic 2024-02-13T03:03:02.316 INFO:teuthology.orchestra.run.smithi161.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-02-13T03:03:02.317 INFO:teuthology.orchestra.run.smithi161.stdout:Sourcing file `/etc/default/grub' 2024-02-13T03:03:02.359 INFO:teuthology.orchestra.run.smithi161.stdout:Generating grub configuration file ... 2024-02-13T03:03:02.990 INFO:teuthology.orchestra.run.smithi161.stdout:Found linux image: /boot/vmlinuz-4.15.0-213-generic 2024-02-13T03:03:03.034 INFO:teuthology.orchestra.run.smithi161.stdout:Found initrd image: /boot/initrd.img-4.15.0-213-generic 2024-02-13T03:03:03.400 INFO:teuthology.orchestra.run.smithi183.stdout:done 2024-02-13T03:03:03.406 INFO:teuthology.orchestra.run.smithi161.stdout:Found linux image: /boot/vmlinuz-4.15.0-200-generic 2024-02-13T03:03:03.435 INFO:teuthology.orchestra.run.smithi161.stdout:Found initrd image: /boot/initrd.img-4.15.0-200-generic 2024-02-13T03:03:03.509 INFO:teuthology.orchestra.run.smithi161.stdout:Found linux image: /boot/vmlinuz-4.15.0-194-generic 2024-02-13T03:03:03.538 INFO:teuthology.orchestra.run.smithi161.stdout:Found initrd image: /boot/initrd.img-4.15.0-194-generic 2024-02-13T03:03:03.869 DEBUG:teuthology.orchestra.run.smithi183:> dpkg -s linux-image-generic 2024-02-13T03:03:03.933 INFO:teuthology.orchestra.run.smithi183.stdout:Package: linux-image-generic 2024-02-13T03:03:03.933 INFO:teuthology.orchestra.run.smithi183.stdout:Status: install ok installed 2024-02-13T03:03:03.933 INFO:teuthology.orchestra.run.smithi183.stdout:Priority: optional 2024-02-13T03:03:03.933 INFO:teuthology.orchestra.run.smithi183.stdout:Section: kernel 2024-02-13T03:03:03.934 INFO:teuthology.orchestra.run.smithi183.stdout:Installed-Size: 18 2024-02-13T03:03:03.934 INFO:teuthology.orchestra.run.smithi183.stdout:Maintainer: Ubuntu Kernel Team 2024-02-13T03:03:03.934 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: amd64 2024-02-13T03:03:03.934 INFO:teuthology.orchestra.run.smithi183.stdout:Source: linux-meta 2024-02-13T03:03:03.934 INFO:teuthology.orchestra.run.smithi183.stdout:Version: 4.15.0.213.196 2024-02-13T03:03:03.934 INFO:teuthology.orchestra.run.smithi183.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.4), zfs-modules (= 0.7.5-1ubuntu16.12) 2024-02-13T03:03:03.934 INFO:teuthology.orchestra.run.smithi183.stdout:Depends: linux-image-4.15.0-213-generic, linux-modules-extra-4.15.0-213-generic, linux-firmware, intel-microcode, amd64-microcode 2024-02-13T03:03:03.934 INFO:teuthology.orchestra.run.smithi183.stdout:Recommends: thermald 2024-02-13T03:03:03.934 INFO:teuthology.orchestra.run.smithi183.stdout:Description: Generic Linux kernel image 2024-02-13T03:03:03.934 INFO:teuthology.orchestra.run.smithi183.stdout: This package will always depend on the latest generic kernel image 2024-02-13T03:03:03.934 INFO:teuthology.orchestra.run.smithi183.stdout: available. 2024-02-13T03:03:03.934 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-200-generic Expected: 4.15.0-213-generic 2024-02-13T03:03:03.935 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-13T03:03:03.935 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-13T03:03:03.935 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-13T03:03:03.935 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi183.front.sepia.ceph.com, path=None, version=distro) 2024-02-13T03:03:03.935 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get clean 2024-02-13T03:03:04.020 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2024-02-13T03:03:04.265 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2024-02-13T03:03:04.292 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2024-02-13T03:03:04.305 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:3 http://security.ubuntu.com/ubuntu bionic-security InRelease 2024-02-13T03:03:04.343 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2024-02-13T03:03:04.871 INFO:teuthology.orchestra.run.smithi161.stdout:done 2024-02-13T03:03:05.352 DEBUG:teuthology.orchestra.run.smithi161:> dpkg -s linux-image-generic 2024-02-13T03:03:05.375 INFO:teuthology.orchestra.run.smithi161.stdout:Package: linux-image-generic 2024-02-13T03:03:05.376 INFO:teuthology.orchestra.run.smithi161.stdout:Status: install ok installed 2024-02-13T03:03:05.376 INFO:teuthology.orchestra.run.smithi161.stdout:Priority: optional 2024-02-13T03:03:05.376 INFO:teuthology.orchestra.run.smithi161.stdout:Section: kernel 2024-02-13T03:03:05.376 INFO:teuthology.orchestra.run.smithi161.stdout:Installed-Size: 18 2024-02-13T03:03:05.376 INFO:teuthology.orchestra.run.smithi161.stdout:Maintainer: Ubuntu Kernel Team 2024-02-13T03:03:05.376 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: amd64 2024-02-13T03:03:05.376 INFO:teuthology.orchestra.run.smithi161.stdout:Source: linux-meta 2024-02-13T03:03:05.376 INFO:teuthology.orchestra.run.smithi161.stdout:Version: 4.15.0.213.196 2024-02-13T03:03:05.376 INFO:teuthology.orchestra.run.smithi161.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.4), zfs-modules (= 0.7.5-1ubuntu16.12) 2024-02-13T03:03:05.376 INFO:teuthology.orchestra.run.smithi161.stdout:Depends: linux-image-4.15.0-213-generic, linux-modules-extra-4.15.0-213-generic, linux-firmware, intel-microcode, amd64-microcode 2024-02-13T03:03:05.376 INFO:teuthology.orchestra.run.smithi161.stdout:Recommends: thermald 2024-02-13T03:03:05.376 INFO:teuthology.orchestra.run.smithi161.stdout:Description: Generic Linux kernel image 2024-02-13T03:03:05.376 INFO:teuthology.orchestra.run.smithi161.stdout: This package will always depend on the latest generic kernel image 2024-02-13T03:03:05.377 INFO:teuthology.orchestra.run.smithi161.stdout: available. 2024-02-13T03:03:05.377 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-200-generic Expected: 4.15.0-213-generic 2024-02-13T03:03:05.377 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-13T03:03:05.377 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-13T03:03:05.377 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-13T03:03:05.377 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi161.front.sepia.ceph.com, path=None, version=distro) 2024-02-13T03:03:05.377 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get clean 2024-02-13T03:03:05.486 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get update 2024-02-13T03:03:05.626 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2024-02-13T03:03:05.729 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2024-02-13T03:03:05.810 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2024-02-13T03:03:05.900 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2024-02-13T03:03:06.359 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-02-13T03:03:06.410 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-02-13T03:03:06.491 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-02-13T03:03:06.789 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2024-02-13T03:03:06.790 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2024-02-13T03:03:07.000 INFO:teuthology.orchestra.run.smithi183.stdout:linux-image-generic is already the newest version (4.15.0.213.196). 2024-02-13T03:03:07.001 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2024-02-13T03:03:07.001 INFO:teuthology.orchestra.run.smithi183.stdout: linux-headers-4.15.0-194 linux-headers-4.15.0-194-generic 2024-02-13T03:03:07.001 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-4.15.0-194-generic linux-modules-4.15.0-194-generic 2024-02-13T03:03:07.001 INFO:teuthology.orchestra.run.smithi183.stdout: linux-modules-extra-4.15.0-194-generic 2024-02-13T03:03:07.001 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2024-02-13T03:03:07.057 INFO:teuthology.orchestra.run.smithi183.stdout:0 upgraded, 0 newly installed, 0 to remove and 165 not upgraded. 2024-02-13T03:03:07.059 DEBUG:teuthology.orchestra.run.smithi183:> dpkg -s linux-image-generic 2024-02-13T03:03:07.082 INFO:teuthology.orchestra.run.smithi183.stdout:Package: linux-image-generic 2024-02-13T03:03:07.082 INFO:teuthology.orchestra.run.smithi183.stdout:Status: install ok installed 2024-02-13T03:03:07.082 INFO:teuthology.orchestra.run.smithi183.stdout:Priority: optional 2024-02-13T03:03:07.082 INFO:teuthology.orchestra.run.smithi183.stdout:Section: kernel 2024-02-13T03:03:07.082 INFO:teuthology.orchestra.run.smithi183.stdout:Installed-Size: 18 2024-02-13T03:03:07.082 INFO:teuthology.orchestra.run.smithi183.stdout:Maintainer: Ubuntu Kernel Team 2024-02-13T03:03:07.082 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: amd64 2024-02-13T03:03:07.082 INFO:teuthology.orchestra.run.smithi183.stdout:Source: linux-meta 2024-02-13T03:03:07.082 INFO:teuthology.orchestra.run.smithi183.stdout:Version: 4.15.0.213.196 2024-02-13T03:03:07.083 INFO:teuthology.orchestra.run.smithi183.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.4), zfs-modules (= 0.7.5-1ubuntu16.12) 2024-02-13T03:03:07.083 INFO:teuthology.orchestra.run.smithi183.stdout:Depends: linux-image-4.15.0-213-generic, linux-modules-extra-4.15.0-213-generic, linux-firmware, intel-microcode, amd64-microcode 2024-02-13T03:03:07.083 INFO:teuthology.orchestra.run.smithi183.stdout:Recommends: thermald 2024-02-13T03:03:07.083 INFO:teuthology.orchestra.run.smithi183.stdout:Description: Generic Linux kernel image 2024-02-13T03:03:07.083 INFO:teuthology.orchestra.run.smithi183.stdout: This package will always depend on the latest generic kernel image 2024-02-13T03:03:07.083 INFO:teuthology.orchestra.run.smithi183.stdout: available. 2024-02-13T03:03:07.083 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2024-02-13T03:03:07.129 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.khSesyDMSl 2024-02-13T03:03:07.129 DEBUG:teuthology.orchestra.run.smithi183:> sudo cp /boot/grub/grub.cfg /tmp/tmp.khSesyDMSl 2024-02-13T03:03:07.192 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.khSesyDMSl 2024-02-13T03:03:07.333 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.khSesyDMSl is 10KB 2024-02-13T03:03:07.344 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.khSesyDMSl 2024-02-13T03:03:07.351 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-02-13T03:03:07.406 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-02-13T03:03:07.407 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-02-13T03:03:07.407 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-02-13T03:03:07.497 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-213-generic 2024-02-13T03:03:07.497 DEBUG:teuthology.orchestra.run.smithi183:> sudo update-grub 2024-02-13T03:03:07.708 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-02-13T03:03:07.760 DEBUG:teuthology.orchestra.run.smithi161:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-02-13T03:03:07.845 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-02-13T03:03:08.117 INFO:teuthology.orchestra.run.smithi161.stdout:Building dependency tree... 2024-02-13T03:03:08.117 INFO:teuthology.orchestra.run.smithi161.stdout:Reading state information... 2024-02-13T03:03:08.324 INFO:teuthology.orchestra.run.smithi161.stdout:linux-image-generic is already the newest version (4.15.0.213.196). 2024-02-13T03:03:08.324 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages were automatically installed and are no longer required: 2024-02-13T03:03:08.324 INFO:teuthology.orchestra.run.smithi161.stdout: linux-headers-4.15.0-194 linux-headers-4.15.0-194-generic 2024-02-13T03:03:08.324 INFO:teuthology.orchestra.run.smithi161.stdout: linux-image-4.15.0-194-generic linux-modules-4.15.0-194-generic 2024-02-13T03:03:08.325 INFO:teuthology.orchestra.run.smithi161.stdout: linux-modules-extra-4.15.0-194-generic 2024-02-13T03:03:08.325 INFO:teuthology.orchestra.run.smithi161.stdout:Use 'sudo apt autoremove' to remove them. 2024-02-13T03:03:08.378 INFO:teuthology.orchestra.run.smithi161.stdout:0 upgraded, 0 newly installed, 0 to remove and 165 not upgraded. 2024-02-13T03:03:08.380 DEBUG:teuthology.orchestra.run.smithi161:> dpkg -s linux-image-generic 2024-02-13T03:03:08.396 INFO:teuthology.orchestra.run.smithi161.stdout:Package: linux-image-generic 2024-02-13T03:03:08.396 INFO:teuthology.orchestra.run.smithi161.stdout:Status: install ok installed 2024-02-13T03:03:08.397 INFO:teuthology.orchestra.run.smithi161.stdout:Priority: optional 2024-02-13T03:03:08.397 INFO:teuthology.orchestra.run.smithi161.stdout:Section: kernel 2024-02-13T03:03:08.397 INFO:teuthology.orchestra.run.smithi161.stdout:Installed-Size: 18 2024-02-13T03:03:08.397 INFO:teuthology.orchestra.run.smithi161.stdout:Maintainer: Ubuntu Kernel Team 2024-02-13T03:03:08.397 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: amd64 2024-02-13T03:03:08.397 INFO:teuthology.orchestra.run.smithi161.stdout:Source: linux-meta 2024-02-13T03:03:08.397 INFO:teuthology.orchestra.run.smithi161.stdout:Version: 4.15.0.213.196 2024-02-13T03:03:08.397 INFO:teuthology.orchestra.run.smithi161.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.4), zfs-modules (= 0.7.5-1ubuntu16.12) 2024-02-13T03:03:08.397 INFO:teuthology.orchestra.run.smithi161.stdout:Depends: linux-image-4.15.0-213-generic, linux-modules-extra-4.15.0-213-generic, linux-firmware, intel-microcode, amd64-microcode 2024-02-13T03:03:08.397 INFO:teuthology.orchestra.run.smithi161.stdout:Recommends: thermald 2024-02-13T03:03:08.397 INFO:teuthology.orchestra.run.smithi161.stdout:Description: Generic Linux kernel image 2024-02-13T03:03:08.397 INFO:teuthology.orchestra.run.smithi161.stdout: This package will always depend on the latest generic kernel image 2024-02-13T03:03:08.398 INFO:teuthology.orchestra.run.smithi161.stdout: available. 2024-02-13T03:03:08.398 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-02-13T03:03:08.443 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.e9KSty88xb 2024-02-13T03:03:08.443 DEBUG:teuthology.orchestra.run.smithi161:> sudo cp /boot/grub/grub.cfg /tmp/tmp.e9KSty88xb 2024-02-13T03:03:08.503 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.e9KSty88xb 2024-02-13T03:03:08.645 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.e9KSty88xb is 10KB 2024-02-13T03:03:08.656 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.e9KSty88xb 2024-02-13T03:03:08.662 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-02-13T03:03:08.716 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:03:08.716 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-02-13T03:03:08.716 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-02-13T03:03:08.803 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-213-generic 2024-02-13T03:03:08.803 DEBUG:teuthology.orchestra.run.smithi161:> sudo update-grub 2024-02-13T03:03:08.933 INFO:teuthology.orchestra.run.smithi183.stderr:Sourcing file `/etc/default/grub' 2024-02-13T03:03:08.944 INFO:teuthology.orchestra.run.smithi183.stderr:Generating grub configuration file ... 2024-02-13T03:03:09.189 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-4.15.0-213-generic 2024-02-13T03:03:09.200 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-4.15.0-213-generic 2024-02-13T03:03:09.564 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-4.15.0-200-generic 2024-02-13T03:03:09.576 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-4.15.0-200-generic 2024-02-13T03:03:09.646 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-4.15.0-194-generic 2024-02-13T03:03:09.659 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-4.15.0-194-generic 2024-02-13T03:03:10.251 INFO:teuthology.orchestra.run.smithi161.stderr:Sourcing file `/etc/default/grub' 2024-02-13T03:03:10.267 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2024-02-13T03:03:10.521 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-4.15.0-213-generic 2024-02-13T03:03:10.533 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-4.15.0-213-generic 2024-02-13T03:03:10.626 INFO:teuthology.orchestra.run.smithi183.stderr:done 2024-02-13T03:03:10.627 DEBUG:teuthology.orchestra.run.smithi183:> sudo shutdown -r now 2024-02-13T03:03:10.843 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-4.15.0-200-generic 2024-02-13T03:03:10.855 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-4.15.0-200-generic 2024-02-13T03:03:10.933 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-4.15.0-194-generic 2024-02-13T03:03:10.946 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-4.15.0-194-generic 2024-02-13T03:03:11.851 INFO:teuthology.orchestra.run.smithi161.stderr:done 2024-02-13T03:03:11.852 DEBUG:teuthology.orchestra.run.smithi161:> sudo shutdown -r now 2024-02-13T03:03:40.654 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-13T03:03:40.655 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-02-13T03:03:40.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T03:03:41.856 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-13T03:03:41.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-02-13T03:03:41.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T03:04:16.444 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-02-13T03:04:25.453 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-02-13T03:04:25.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T03:04:40.665 DEBUG:teuthology.orchestra.remote:timed out 2024-02-13T03:04:43.836 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-02-13T03:04:49.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-02-13T03:04:49.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T03:04:49.771 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-02-13T03:04:50.093 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-02-13T03:04:50.093 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.15.0-213-generic"... 2024-02-13T03:04:50.094 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-02-13T03:04:50.147 INFO:teuthology.orchestra.run.smithi183.stdout:4.15.0-213-generic 2024-02-13T03:04:50.147 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-213-generic vs 4.15.0-213-generic 2024-02-13T03:04:50.148 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-13T03:04:50.148 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-02-13T03:04:51.149 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-13T03:04:51.149 DEBUG:teuthology.orchestra.run.smithi183:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-13T03:04:51.178 INFO:teuthology.orchestra.run.smithi183.stdout:ttyS1 2024-02-13T03:04:51.216 DEBUG:teuthology.parallel:result is None 2024-02-13T03:04:55.837 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-02-13T03:04:55.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T03:04:55.938 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-02-13T03:04:56.397 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-02-13T03:04:56.397 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.15.0-213-generic"... 2024-02-13T03:04:56.397 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-02-13T03:04:56.444 INFO:teuthology.orchestra.run.smithi161.stdout:4.15.0-213-generic 2024-02-13T03:04:56.444 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-213-generic vs 4.15.0-213-generic 2024-02-13T03:04:56.444 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-13T03:04:56.444 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-02-13T03:04:57.445 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-13T03:04:57.445 DEBUG:teuthology.orchestra.run.smithi161:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-13T03:04:57.493 INFO:teuthology.orchestra.run.smithi161.stdout:ttyS1 2024-02-13T03:04:57.528 DEBUG:teuthology.parallel:result is None 2024-02-13T03:04:57.528 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-13T03:04:57.537 INFO:teuthology.task.internal:Creating test directory... 2024-02-13T03:04:57.537 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-13T03:04:57.542 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-13T03:04:57.548 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-13T03:04:57.555 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-13T03:04:57.562 INFO:teuthology.task.internal:Creating archive directory... 2024-02-13T03:04:57.562 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-13T03:04:57.592 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-13T03:04:57.624 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-13T03:04:57.631 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-13T03:04:57.631 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-13T03:04:57.665 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-13T03:04:57.684 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T03:04:57.684 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T03:04:57.691 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T03:04:57.693 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T03:04:57.695 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-13T03:04:57.702 INFO:teuthology.task.internal:Configuring sudo... 2024-02-13T03:04:57.702 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-13T03:04:57.736 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-13T03:04:57.752 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-13T03:04:57.761 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-13T03:04:57.762 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-13T03:04:57.791 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-13T03:04:57.804 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-13T03:04:57.844 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-13T03:04:57.892 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:04:57.892 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-13T03:04:57.959 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-13T03:04:57.966 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-13T03:04:58.016 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-02-13T03:04:58.016 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-13T03:04:58.076 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-02-13T03:04:58.078 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-02-13T03:04:58.159 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-13T03:04:58.166 INFO:teuthology.task.internal:Starting timer... 2024-02-13T03:04:58.167 INFO:teuthology.run_tasks:Running task pcp... 2024-02-13T03:04:58.176 INFO:teuthology.run_tasks:Running task selinux... 2024-02-13T03:04:58.185 DEBUG:teuthology.task.selinux:Excluding smithi161: OS 'ubuntu' does not support SELinux 2024-02-13T03:04:58.185 DEBUG:teuthology.task.selinux:Excluding smithi183: OS 'ubuntu' does not support SELinux 2024-02-13T03:04:58.186 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-13T03:04:58.186 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-02-13T03:04:58.186 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-13T03:04:58.186 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-13T03:04:58.195 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-13T03:04:58.212 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-13T03:04:58.214 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi161.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-13T03:10:20.415 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi161.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2024-02-13T03:10:20.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-02-13T03:10:20.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T03:10:20.503 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-02-13T03:10:20.617 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-02-13T03:10:20.617 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-02-13T03:10:20.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T03:10:20.698 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-02-13T03:10:20.821 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-02-13T03:10:20.821 INFO:teuthology.run_tasks:Running task clock... 2024-02-13T03:10:20.835 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-13T03:10:20.835 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-13T03:10:20.836 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T03:10:20.839 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-13T03:10:20.839 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T03:10:20.882 INFO:teuthology.orchestra.run.smithi161.stdout:13 Feb 03:10:20 ntpd[14684]: ntpd 4.2.8p10@1.3728-o (1): Starting 2024-02-13T03:10:20.882 INFO:teuthology.orchestra.run.smithi161.stdout:13 Feb 03:10:20 ntpd[14684]: Command line: ntpd -gq 2024-02-13T03:10:20.882 INFO:teuthology.orchestra.run.smithi161.stdout:13 Feb 03:10:20 ntpd[14684]: proto: precision = 0.106 usec (-23) 2024-02-13T03:10:20.883 INFO:teuthology.orchestra.run.smithi161.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-02-13T03:10:20.883 INFO:teuthology.orchestra.run.smithi161.stdout:13 Feb 03:10:20 ntpd[14684]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-02-13T03:10:20.884 INFO:teuthology.orchestra.run.smithi161.stderr:restrict ::: KOD does nothing without LIMITED. 2024-02-13T03:10:20.885 INFO:teuthology.orchestra.run.smithi161.stdout:13 Feb 03:10:20 ntpd[14684]: restrict ::: KOD does nothing without LIMITED. 2024-02-13T03:10:20.885 INFO:teuthology.orchestra.run.smithi161.stdout:13 Feb 03:10:20 ntpd[14684]: Listen and drop on 0 v6wildcard [::]:123 2024-02-13T03:10:20.885 INFO:teuthology.orchestra.run.smithi161.stdout:13 Feb 03:10:20 ntpd[14684]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-02-13T03:10:20.885 INFO:teuthology.orchestra.run.smithi161.stdout:13 Feb 03:10:20 ntpd[14684]: Listen normally on 2 lo 127.0.0.1:123 2024-02-13T03:10:20.885 INFO:teuthology.orchestra.run.smithi161.stdout:13 Feb 03:10:20 ntpd[14684]: Listen normally on 3 enp3s0f1 172.21.15.161:123 2024-02-13T03:10:20.885 INFO:teuthology.orchestra.run.smithi161.stdout:13 Feb 03:10:20 ntpd[14684]: Listen normally on 4 lo [::1]:123 2024-02-13T03:10:20.886 INFO:teuthology.orchestra.run.smithi161.stdout:13 Feb 03:10:20 ntpd[14684]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7251%5]:123 2024-02-13T03:10:20.886 INFO:teuthology.orchestra.run.smithi161.stdout:13 Feb 03:10:20 ntpd[14684]: Listening on routing socket on fd #22 for interface updates 2024-02-13T03:10:20.925 INFO:teuthology.orchestra.run.smithi183.stdout:13 Feb 03:10:20 ntpd[14691]: ntpd 4.2.8p10@1.3728-o (1): Starting 2024-02-13T03:10:20.925 INFO:teuthology.orchestra.run.smithi183.stdout:13 Feb 03:10:20 ntpd[14691]: Command line: ntpd -gq 2024-02-13T03:10:20.925 INFO:teuthology.orchestra.run.smithi183.stdout:13 Feb 03:10:20 ntpd[14691]: proto: precision = 0.123 usec (-23) 2024-02-13T03:10:20.925 INFO:teuthology.orchestra.run.smithi183.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-02-13T03:10:20.926 INFO:teuthology.orchestra.run.smithi183.stdout:13 Feb 03:10:20 ntpd[14691]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-02-13T03:10:20.926 INFO:teuthology.orchestra.run.smithi183.stderr:restrict ::: KOD does nothing without LIMITED. 2024-02-13T03:10:20.927 INFO:teuthology.orchestra.run.smithi183.stdout:13 Feb 03:10:20 ntpd[14691]: restrict ::: KOD does nothing without LIMITED. 2024-02-13T03:10:20.927 INFO:teuthology.orchestra.run.smithi183.stdout:13 Feb 03:10:20 ntpd[14691]: Listen and drop on 0 v6wildcard [::]:123 2024-02-13T03:10:20.927 INFO:teuthology.orchestra.run.smithi183.stdout:13 Feb 03:10:20 ntpd[14691]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-02-13T03:10:20.927 INFO:teuthology.orchestra.run.smithi183.stdout:13 Feb 03:10:20 ntpd[14691]: Listen normally on 2 lo 127.0.0.1:123 2024-02-13T03:10:20.927 INFO:teuthology.orchestra.run.smithi183.stdout:13 Feb 03:10:20 ntpd[14691]: Listen normally on 3 enp3s0f1 172.21.15.183:123 2024-02-13T03:10:20.928 INFO:teuthology.orchestra.run.smithi183.stdout:13 Feb 03:10:20 ntpd[14691]: Listen normally on 4 lo [::1]:123 2024-02-13T03:10:20.928 INFO:teuthology.orchestra.run.smithi183.stdout:13 Feb 03:10:20 ntpd[14691]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8085%5]:123 2024-02-13T03:10:20.928 INFO:teuthology.orchestra.run.smithi183.stdout:13 Feb 03:10:20 ntpd[14691]: Listening on routing socket on fd #22 for interface updates 2024-02-13T03:10:21.883 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:21 ntpd[14684]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:21.883 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:21 ntpd[14684]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:21.883 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:21 ntpd[14684]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:21.883 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:21 ntpd[14684]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:21.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:21 ntpd[14684]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:21.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:21 ntpd[14684]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:21.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:21 ntpd[14684]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:21.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:21 ntpd[14684]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:21.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:21 ntpd[14691]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:21.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:21 ntpd[14691]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:21.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:21 ntpd[14691]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:21.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:21 ntpd[14691]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:22.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:22 ntpd[14691]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:22.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:22 ntpd[14691]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:22.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:22 ntpd[14691]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:22.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:22 ntpd[14691]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:23.883 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:23 ntpd[14684]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:23.883 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:23 ntpd[14684]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:23.883 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:23 ntpd[14684]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:23.883 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:23 ntpd[14684]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:23.883 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:23 ntpd[14684]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:23.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:23 ntpd[14684]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:23.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:23 ntpd[14684]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:23.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:23 ntpd[14684]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:23.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:23 ntpd[14691]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:23.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:23 ntpd[14691]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:23.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:23 ntpd[14691]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:23.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:23 ntpd[14691]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:24.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:24 ntpd[14691]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:24.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:24 ntpd[14691]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:24.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:24 ntpd[14691]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:24.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:24 ntpd[14691]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:24.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:24 ntpd[14691]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:24.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:24 ntpd[14691]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:24.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:24 ntpd[14691]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:24.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:24 ntpd[14691]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:25.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:25 ntpd[14684]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:25.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:25 ntpd[14684]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:25.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:25 ntpd[14684]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:25.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:25 ntpd[14684]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:25.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:25 ntpd[14684]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:25.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:25 ntpd[14684]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:25.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:25 ntpd[14684]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:25.885 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:25 ntpd[14684]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:25.885 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:25 ntpd[14684]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:25.885 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:25 ntpd[14684]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:25.885 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:25 ntpd[14684]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:25.885 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:25 ntpd[14684]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:25.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:25 ntpd[14691]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:25.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:25 ntpd[14691]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:25.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:25 ntpd[14691]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:25.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:25 ntpd[14691]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:26.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:26 ntpd[14691]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:26.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:26 ntpd[14691]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:26.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:26 ntpd[14691]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:26.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:26 ntpd[14691]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:26.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:26 ntpd[14691]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:26.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:26 ntpd[14691]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:26.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:26 ntpd[14691]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:26.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:26 ntpd[14691]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:27.883 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:27 ntpd[14684]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:27.883 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:27 ntpd[14684]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:27.883 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:27 ntpd[14684]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:27.883 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:27 ntpd[14684]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:27.883 INFO:teuthology.orchestra.run.smithi161.stdout:13 Feb 03:10:27 ntpd[14684]: ntpd: time slew +0.001364 s 2024-02-13T03:10:27.883 INFO:teuthology.orchestra.run.smithi161.stdout:ntpd: time slew +0.001364s 2024-02-13T03:10:27.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:27 ntpd[14684]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-02-13T03:10:27.884 INFO:teuthology.orchestra.run.smithi161.stderr:13 Feb 03:10:27 ntpd[14684]: can't open /var/log/ntpstats/loopstats.20240213: Permission denied 2024-02-13T03:10:27.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:27 ntpd[14691]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-13T03:10:27.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:27 ntpd[14691]: can't open /var/log/ntpstats/rawstats.20240213: Permission denied 2024-02-13T03:10:27.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:27 ntpd[14691]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-13T03:10:27.925 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:27 ntpd[14691]: can't open /var/log/ntpstats/peerstats.20240213: Permission denied 2024-02-13T03:10:27.925 INFO:teuthology.orchestra.run.smithi183.stdout:13 Feb 03:10:27 ntpd[14691]: ntpd: time slew -0.000450 s 2024-02-13T03:10:27.926 INFO:teuthology.orchestra.run.smithi183.stdout:ntpd: time slew -0.000450s 2024-02-13T03:10:27.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:27 ntpd[14691]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-02-13T03:10:27.926 INFO:teuthology.orchestra.run.smithi183.stderr:13 Feb 03:10:27 ntpd[14691]: can't open /var/log/ntpstats/loopstats.20240213: Permission denied 2024-02-13T03:10:27.935 INFO:teuthology.orchestra.run.smithi161.stdout: remote refid st t when poll reach delay offset jitter 2024-02-13T03:10:27.935 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================== 2024-02-13T03:10:27.935 INFO:teuthology.orchestra.run.smithi161.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-13T03:10:27.935 INFO:teuthology.orchestra.run.smithi161.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-13T03:10:27.935 INFO:teuthology.orchestra.run.smithi161.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-13T03:10:27.935 INFO:teuthology.orchestra.run.smithi161.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-13T03:10:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: remote refid st t when poll reach delay offset jitter 2024-02-13T03:10:27.984 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================== 2024-02-13T03:10:27.984 INFO:teuthology.orchestra.run.smithi183.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-13T03:10:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-13T03:10:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-13T03:10:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-13T03:10:27.988 INFO:teuthology.run_tasks:Running task install... 2024-02-13T03:10:28.007 DEBUG:teuthology.task.install:project ceph 2024-02-13T03:10:28.007 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1'}, 'extra_packages': ['cephadm']} 2024-02-13T03:10:28.007 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1'} 2024-02-13T03:10:28.007 INFO:teuthology.task.install:Using flavor: default 2024-02-13T03:10:28.023 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-13T03:10:28.023 INFO:teuthology.task.install:extra packages: [] 2024-02-13T03:10:28.023 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-key list | grep Ceph 2024-02-13T03:10:28.024 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-key list | grep Ceph 2024-02-13T03:10:28.119 INFO:teuthology.orchestra.run.smithi161.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-02-13T03:10:28.124 INFO:teuthology.orchestra.run.smithi183.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-02-13T03:10:28.176 INFO:teuthology.orchestra.run.smithi161.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-02-13T03:10:28.176 INFO:teuthology.orchestra.run.smithi161.stdout:uid [ unknown] Ceph.com (release key) 2024-02-13T03:10:28.177 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-02-13T03:10:28.177 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-13T03:10:28.183 INFO:teuthology.orchestra.run.smithi183.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-02-13T03:10:28.184 INFO:teuthology.orchestra.run.smithi183.stdout:uid [ unknown] Ceph.com (release key) 2024-02-13T03:10:28.184 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-02-13T03:10:28.184 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-13T03:10:28.331 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default/ 2024-02-13T03:10:28.332 INFO:teuthology.task.install.deb:Package version is 16.2.14-521-g0e714d9a-1bionic 2024-02-13T03:10:28.404 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default/ 2024-02-13T03:10:28.405 INFO:teuthology.task.install.deb:Package version is 16.2.14-521-g0e714d9a-1bionic 2024-02-13T03:10:28.458 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:10:28.458 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-02-13T03:10:28.473 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get update 2024-02-13T03:10:28.572 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-02-13T03:10:28.572 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-02-13T03:10:28.588 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2024-02-13T03:10:28.681 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2024-02-13T03:10:28.773 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2024-02-13T03:10:28.774 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2024-02-13T03:10:28.774 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2024-02-13T03:10:28.783 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2024-02-13T03:10:28.790 INFO:teuthology.orchestra.run.smithi161.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic InRelease 2024-02-13T03:10:28.803 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2024-02-13T03:10:28.826 INFO:teuthology.orchestra.run.smithi183.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic InRelease 2024-02-13T03:10:28.836 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic Release [7,690 B] 2024-02-13T03:10:28.847 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic Release [7,690 B] 2024-02-13T03:10:28.854 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:5 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2024-02-13T03:10:28.871 INFO:teuthology.orchestra.run.smithi183.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic Release.gpg 2024-02-13T03:10:28.875 INFO:teuthology.orchestra.run.smithi161.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic Release.gpg 2024-02-13T03:10:28.896 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 Packages [17.2 kB] 2024-02-13T03:10:28.911 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 Packages [17.2 kB] 2024-02-13T03:10:28.940 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,113 B] 2024-02-13T03:10:28.940 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2024-02-13T03:10:28.959 INFO:teuthology.orchestra.run.smithi161.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,113 B] 2024-02-13T03:10:29.272 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 28.0 kB in 1s (47.2 kB/s) 2024-02-13T03:10:29.383 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 28.0 kB in 1s (47.1 kB/s) 2024-02-13T03:10:30.757 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-02-13T03:10:30.807 DEBUG:teuthology.orchestra.run.smithi161:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.2.14-521-g0e714d9a-1bionic cephadm=16.2.14-521-g0e714d9a-1bionic ceph-mds=16.2.14-521-g0e714d9a-1bionic ceph-mgr=16.2.14-521-g0e714d9a-1bionic ceph-common=16.2.14-521-g0e714d9a-1bionic ceph-fuse=16.2.14-521-g0e714d9a-1bionic ceph-test=16.2.14-521-g0e714d9a-1bionic radosgw=16.2.14-521-g0e714d9a-1bionic python3-rados=16.2.14-521-g0e714d9a-1bionic python3-rgw=16.2.14-521-g0e714d9a-1bionic python3-cephfs=16.2.14-521-g0e714d9a-1bionic python3-rbd=16.2.14-521-g0e714d9a-1bionic libcephfs2=16.2.14-521-g0e714d9a-1bionic libcephfs-dev=16.2.14-521-g0e714d9a-1bionic librados2=16.2.14-521-g0e714d9a-1bionic librbd1=16.2.14-521-g0e714d9a-1bionic rbd-fuse=16.2.14-521-g0e714d9a-1bionic 2024-02-13T03:10:30.868 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-02-13T03:10:30.891 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-02-13T03:10:30.923 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.2.14-521-g0e714d9a-1bionic cephadm=16.2.14-521-g0e714d9a-1bionic ceph-mds=16.2.14-521-g0e714d9a-1bionic ceph-mgr=16.2.14-521-g0e714d9a-1bionic ceph-common=16.2.14-521-g0e714d9a-1bionic ceph-fuse=16.2.14-521-g0e714d9a-1bionic ceph-test=16.2.14-521-g0e714d9a-1bionic radosgw=16.2.14-521-g0e714d9a-1bionic python3-rados=16.2.14-521-g0e714d9a-1bionic python3-rgw=16.2.14-521-g0e714d9a-1bionic python3-cephfs=16.2.14-521-g0e714d9a-1bionic python3-rbd=16.2.14-521-g0e714d9a-1bionic libcephfs2=16.2.14-521-g0e714d9a-1bionic libcephfs-dev=16.2.14-521-g0e714d9a-1bionic librados2=16.2.14-521-g0e714d9a-1bionic librbd1=16.2.14-521-g0e714d9a-1bionic rbd-fuse=16.2.14-521-g0e714d9a-1bionic 2024-02-13T03:10:31.003 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-02-13T03:10:31.170 INFO:teuthology.orchestra.run.smithi161.stdout:Building dependency tree... 2024-02-13T03:10:31.170 INFO:teuthology.orchestra.run.smithi161.stdout:Reading state information... 2024-02-13T03:10:31.293 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2024-02-13T03:10:31.293 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2024-02-13T03:10:31.376 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages were automatically installed and are no longer required: 2024-02-13T03:10:31.376 INFO:teuthology.orchestra.run.smithi161.stdout: linux-headers-4.15.0-194 linux-headers-4.15.0-194-generic 2024-02-13T03:10:31.376 INFO:teuthology.orchestra.run.smithi161.stdout: linux-image-4.15.0-194-generic linux-modules-4.15.0-194-generic 2024-02-13T03:10:31.376 INFO:teuthology.orchestra.run.smithi161.stdout: linux-modules-extra-4.15.0-194-generic 2024-02-13T03:10:31.376 INFO:teuthology.orchestra.run.smithi161.stdout:Use 'sudo apt autoremove' to remove them. 2024-02-13T03:10:31.377 INFO:teuthology.orchestra.run.smithi161.stdout:The following additional packages will be installed: 2024-02-13T03:10:31.377 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-02-13T03:10:31.377 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq libjaeger 2024-02-13T03:10:31.377 INFO:teuthology.orchestra.run.smithi161.stdout: libjq1 libleveldb1v5 liblttng-ust0 liblua5.3-0 liboath0 libonig4 2024-02-13T03:10:31.377 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2024-02-13T03:10:31.378 INFO:teuthology.orchestra.run.smithi161.stdout: libsqlite3-mod-ceph libxslt1.1 nvme-cli python-attr python-funcsigs 2024-02-13T03:10:31.378 INFO:teuthology.orchestra.run.smithi161.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2024-02-13T03:10:31.378 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pastescript python3-pecan python3-pil python3-pkg-resources 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2024-02-13T03:10:31.379 INFO:teuthology.orchestra.run.smithi161.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2024-02-13T03:10:31.380 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2024-02-13T03:10:31.380 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2024-02-13T03:10:31.381 INFO:teuthology.orchestra.run.smithi161.stdout:Suggested packages: 2024-02-13T03:10:31.381 INFO:teuthology.orchestra.run.smithi161.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2024-02-13T03:10:31.381 INFO:teuthology.orchestra.run.smithi161.stdout: python-pytest-xdist python-mock python-cryptography-doc 2024-02-13T03:10:31.381 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2024-02-13T03:10:31.381 INFO:teuthology.orchestra.run.smithi161.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2024-02-13T03:10:31.381 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2024-02-13T03:10:31.381 INFO:teuthology.orchestra.run.smithi161.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2024-02-13T03:10:31.381 INFO:teuthology.orchestra.run.smithi161.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2024-02-13T03:10:31.382 INFO:teuthology.orchestra.run.smithi161.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2024-02-13T03:10:31.382 INFO:teuthology.orchestra.run.smithi161.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2024-02-13T03:10:31.382 INFO:teuthology.orchestra.run.smithi161.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-02-13T03:10:31.382 INFO:teuthology.orchestra.run.smithi161.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2024-02-13T03:10:31.382 INFO:teuthology.orchestra.run.smithi161.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2024-02-13T03:10:31.382 INFO:teuthology.orchestra.run.smithi161.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2024-02-13T03:10:31.382 INFO:teuthology.orchestra.run.smithi161.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2024-02-13T03:10:31.382 INFO:teuthology.orchestra.run.smithi161.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2024-02-13T03:10:31.497 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2024-02-13T03:10:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: linux-headers-4.15.0-194 linux-headers-4.15.0-194-generic 2024-02-13T03:10:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-4.15.0-194-generic linux-modules-4.15.0-194-generic 2024-02-13T03:10:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: linux-modules-extra-4.15.0-194-generic 2024-02-13T03:10:31.498 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2024-02-13T03:10:31.498 INFO:teuthology.orchestra.run.smithi183.stdout:The following additional packages will be installed: 2024-02-13T03:10:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-02-13T03:10:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq libjaeger 2024-02-13T03:10:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: libjq1 libleveldb1v5 liblttng-ust0 liblua5.3-0 liboath0 libonig4 2024-02-13T03:10:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2024-02-13T03:10:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: libsqlite3-mod-ceph libxslt1.1 nvme-cli python-attr python-funcsigs 2024-02-13T03:10:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2024-02-13T03:10:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2024-02-13T03:10:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2024-02-13T03:10:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2024-02-13T03:10:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2024-02-13T03:10:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-02-13T03:10:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-02-13T03:10:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2024-02-13T03:10:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2024-02-13T03:10:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pastescript python3-pecan python3-pil python3-pkg-resources 2024-02-13T03:10:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2024-02-13T03:10:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2024-02-13T03:10:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2024-02-13T03:10:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2024-02-13T03:10:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2024-02-13T03:10:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2024-02-13T03:10:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2024-02-13T03:10:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2024-02-13T03:10:31.502 INFO:teuthology.orchestra.run.smithi161.stdout:The following NEW packages will be installed: 2024-02-13T03:10:31.502 INFO:teuthology.orchestra.run.smithi161.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-02-13T03:10:31.502 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-02-13T03:10:31.503 INFO:teuthology.orchestra.run.smithi183.stdout:Suggested packages: 2024-02-13T03:10:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2024-02-13T03:10:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: python-pytest-xdist python-mock python-cryptography-doc 2024-02-13T03:10:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2024-02-13T03:10:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2024-02-13T03:10:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2024-02-13T03:10:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2024-02-13T03:10:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2024-02-13T03:10:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2024-02-13T03:10:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2024-02-13T03:10:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-02-13T03:10:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2024-02-13T03:10:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2024-02-13T03:10:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2024-02-13T03:10:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2024-02-13T03:10:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2024-02-13T03:10:31.505 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test cephadm jq libcephfs-dev 2024-02-13T03:10:31.505 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 libjaeger libjq1 libleveldb1v5 liblttng-ust0 liblua5.3-0 liboath0 2024-02-13T03:10:31.505 INFO:teuthology.orchestra.run.smithi161.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2024-02-13T03:10:31.505 INFO:teuthology.orchestra.run.smithi161.stdout: libsqlite3-mod-ceph libxslt1.1 nvme-cli python-attr python-funcsigs 2024-02-13T03:10:31.505 INFO:teuthology.orchestra.run.smithi161.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2024-02-13T03:10:31.505 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2024-02-13T03:10:31.505 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2024-02-13T03:10:31.505 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2024-02-13T03:10:31.505 INFO:teuthology.orchestra.run.smithi161.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-jwt 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados python3-rbd python3-repoze.lru python3-rgw python3-routes 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa python3-scipy python3-setuptools python3-simplegeneric 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: python3-sklearn-lib python3-sqlalchemy python3-tempita python3-tk python3-tz 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: python3-uritemplate python3-waitress python3-webencodings python3-webob 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket python3-webtest python3-werkzeug radosgw rbd-fuse socat 2024-02-13T03:10:31.506 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet 2024-02-13T03:10:31.507 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages will be upgraded: 2024-02-13T03:10:31.507 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 librbd1 python3-pkg-resources 2024-02-13T03:10:31.629 INFO:teuthology.orchestra.run.smithi183.stdout:The following NEW packages will be installed: 2024-02-13T03:10:31.629 INFO:teuthology.orchestra.run.smithi183.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-02-13T03:10:31.629 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-02-13T03:10:31.629 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test cephadm jq libcephfs-dev 2024-02-13T03:10:31.630 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 libjaeger libjq1 libleveldb1v5 liblttng-ust0 liblua5.3-0 liboath0 2024-02-13T03:10:31.630 INFO:teuthology.orchestra.run.smithi183.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2024-02-13T03:10:31.630 INFO:teuthology.orchestra.run.smithi183.stdout: libsqlite3-mod-ceph libxslt1.1 nvme-cli python-attr python-funcsigs 2024-02-13T03:10:31.631 INFO:teuthology.orchestra.run.smithi183.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2024-02-13T03:10:31.631 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2024-02-13T03:10:31.631 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2024-02-13T03:10:31.631 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2024-02-13T03:10:31.631 INFO:teuthology.orchestra.run.smithi183.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2024-02-13T03:10:31.631 INFO:teuthology.orchestra.run.smithi183.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-jwt 2024-02-13T03:10:31.631 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-02-13T03:10:31.631 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2024-02-13T03:10:31.631 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2024-02-13T03:10:31.631 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2024-02-13T03:10:31.631 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2024-02-13T03:10:31.631 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2024-02-13T03:10:31.632 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados python3-rbd python3-repoze.lru python3-rgw python3-routes 2024-02-13T03:10:31.632 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa python3-scipy python3-setuptools python3-simplegeneric 2024-02-13T03:10:31.632 INFO:teuthology.orchestra.run.smithi183.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-02-13T03:10:31.632 INFO:teuthology.orchestra.run.smithi183.stdout: python3-sklearn-lib python3-sqlalchemy python3-tempita python3-tk python3-tz 2024-02-13T03:10:31.632 INFO:teuthology.orchestra.run.smithi183.stdout: python3-uritemplate python3-waitress python3-webencodings python3-webob 2024-02-13T03:10:31.632 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket python3-webtest python3-werkzeug radosgw rbd-fuse socat 2024-02-13T03:10:31.632 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet 2024-02-13T03:10:31.632 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages will be upgraded: 2024-02-13T03:10:31.633 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 librbd1 python3-pkg-resources 2024-02-13T03:10:31.702 INFO:teuthology.orchestra.run.smithi161.stdout:3 upgraded, 110 newly installed, 0 to remove and 145 not upgraded. 2024-02-13T03:10:31.702 INFO:teuthology.orchestra.run.smithi161.stdout:Need to get 156 MB of archives. 2024-02-13T03:10:31.702 INFO:teuthology.orchestra.run.smithi161.stdout:After this operation, 690 MB of additional disk space will be used. 2024-02-13T03:10:31.702 INFO:teuthology.orchestra.run.smithi161.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2024-02-13T03:10:31.706 INFO:teuthology.orchestra.run.smithi183.stdout:3 upgraded, 110 newly installed, 0 to remove and 145 not upgraded. 2024-02-13T03:10:31.706 INFO:teuthology.orchestra.run.smithi183.stdout:Need to get 156 MB of archives. 2024-02-13T03:10:31.706 INFO:teuthology.orchestra.run.smithi183.stdout:After this operation, 690 MB of additional disk space will be used. 2024-02-13T03:10:31.706 INFO:teuthology.orchestra.run.smithi183.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2024-02-13T03:10:31.727 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2024-02-13T03:10:31.785 INFO:teuthology.orchestra.run.smithi161.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2024-02-13T03:10:31.796 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2024-02-13T03:10:31.799 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 liblua5.3-0 amd64 5.3.3-1ubuntu0.18.04.1 [115 kB] 2024-02-13T03:10:31.815 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2024-02-13T03:10:31.819 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2024-02-13T03:10:31.838 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2024-02-13T03:10:31.839 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2024-02-13T03:10:31.845 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2024-02-13T03:10:31.846 INFO:teuthology.orchestra.run.smithi183.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2024-02-13T03:10:31.849 INFO:teuthology.orchestra.run.smithi183.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2024-02-13T03:10:31.851 INFO:teuthology.orchestra.run.smithi183.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2024-02-13T03:10:31.853 INFO:teuthology.orchestra.run.smithi183.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2024-02-13T03:10:31.862 INFO:teuthology.orchestra.run.smithi183.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2024-02-13T03:10:31.862 INFO:teuthology.orchestra.run.smithi183.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2024-02-13T03:10:31.863 INFO:teuthology.orchestra.run.smithi183.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2024-02-13T03:10:31.866 INFO:teuthology.orchestra.run.smithi183.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jwt all 1.5.3+ds1-1ubuntu0.1 [16.6 kB] 2024-02-13T03:10:31.883 INFO:teuthology.orchestra.run.smithi183.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2024-02-13T03:10:31.883 INFO:teuthology.orchestra.run.smithi183.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2024-02-13T03:10:31.883 INFO:teuthology.orchestra.run.smithi183.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-mako all 1.0.7+ds1-1ubuntu0.2 [59.4 kB] 2024-02-13T03:10:31.885 INFO:teuthology.orchestra.run.smithi183.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2024-02-13T03:10:31.886 INFO:teuthology.orchestra.run.smithi183.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2024-02-13T03:10:31.904 INFO:teuthology.orchestra.run.smithi183.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2024-02-13T03:10:31.906 INFO:teuthology.orchestra.run.smithi183.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2024-02-13T03:10:31.908 INFO:teuthology.orchestra.run.smithi183.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2024-02-13T03:10:31.910 INFO:teuthology.orchestra.run.smithi183.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pkg-resources all 39.0.1-2ubuntu0.1 [99.0 kB] 2024-02-13T03:10:31.914 INFO:teuthology.orchestra.run.smithi183.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2024-02-13T03:10:31.924 INFO:teuthology.orchestra.run.smithi183.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2024-02-13T03:10:31.933 INFO:teuthology.orchestra.run.smithi183.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2024-02-13T03:10:31.933 INFO:teuthology.orchestra.run.smithi183.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2024-02-13T03:10:31.933 INFO:teuthology.orchestra.run.smithi183.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2024-02-13T03:10:31.934 INFO:teuthology.orchestra.run.smithi183.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2024-02-13T03:10:31.945 INFO:teuthology.orchestra.run.smithi183.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.2 [175 kB] 2024-02-13T03:10:31.948 INFO:teuthology.orchestra.run.smithi183.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2024-02-13T03:10:31.950 INFO:teuthology.orchestra.run.smithi183.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2024-02-13T03:10:31.950 INFO:teuthology.orchestra.run.smithi183.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2024-02-13T03:10:31.950 INFO:teuthology.orchestra.run.smithi183.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2024-02-13T03:10:31.972 INFO:teuthology.orchestra.run.smithi183.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2024-02-13T03:10:31.972 INFO:teuthology.orchestra.run.smithi183.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2024-02-13T03:10:32.062 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2024-02-13T03:10:32.075 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 liblua5.3-0 amd64 5.3.3-1ubuntu0.18.04.1 [115 kB] 2024-02-13T03:10:32.086 INFO:teuthology.orchestra.run.smithi183.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2024-02-13T03:10:32.103 INFO:teuthology.orchestra.run.smithi183.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2024-02-13T03:10:32.105 INFO:teuthology.orchestra.run.smithi183.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2024-02-13T03:10:32.121 INFO:teuthology.orchestra.run.smithi183.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2024-02-13T03:10:32.121 INFO:teuthology.orchestra.run.smithi183.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2024-02-13T03:10:32.121 INFO:teuthology.orchestra.run.smithi183.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.3 [28.8 kB] 2024-02-13T03:10:32.121 INFO:teuthology.orchestra.run.smithi183.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2024-02-13T03:10:32.122 INFO:teuthology.orchestra.run.smithi183.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2024-02-13T03:10:32.123 INFO:teuthology.orchestra.run.smithi183.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2024-02-13T03:10:32.123 INFO:teuthology.orchestra.run.smithi183.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2024-02-13T03:10:32.139 INFO:teuthology.orchestra.run.smithi161.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2024-02-13T03:10:32.142 INFO:teuthology.orchestra.run.smithi183.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2024-02-13T03:10:32.143 INFO:teuthology.orchestra.run.smithi183.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-setuptools all 39.0.1-2ubuntu0.1 [248 kB] 2024-02-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi183.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2024-02-13T03:10:32.149 INFO:teuthology.orchestra.run.smithi183.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2024-02-13T03:10:32.153 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2024-02-13T03:10:32.163 INFO:teuthology.orchestra.run.smithi183.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2024-02-13T03:10:32.167 INFO:teuthology.orchestra.run.smithi183.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2024-02-13T03:10:32.169 INFO:teuthology.orchestra.run.smithi183.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2024-02-13T03:10:32.170 INFO:teuthology.orchestra.run.smithi183.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2024-02-13T03:10:32.176 INFO:teuthology.orchestra.run.smithi183.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.3 [150 kB] 2024-02-13T03:10:32.177 INFO:teuthology.orchestra.run.smithi183.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2024-02-13T03:10:32.183 INFO:teuthology.orchestra.run.smithi183.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.2 [184 kB] 2024-02-13T03:10:32.190 INFO:teuthology.orchestra.run.smithi183.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2024-02-13T03:10:32.191 INFO:teuthology.orchestra.run.smithi183.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2024-02-13T03:10:32.192 INFO:teuthology.orchestra.run.smithi183.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2024-02-13T03:10:32.204 INFO:teuthology.orchestra.run.smithi183.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python-py all 1.5.2-1ubuntu0.1 [65.3 kB] 2024-02-13T03:10:32.205 INFO:teuthology.orchestra.run.smithi183.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2024-02-13T03:10:32.207 INFO:teuthology.orchestra.run.smithi183.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2024-02-13T03:10:32.207 INFO:teuthology.orchestra.run.smithi183.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2024-02-13T03:10:32.207 INFO:teuthology.orchestra.run.smithi183.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2024-02-13T03:10:32.208 INFO:teuthology.orchestra.run.smithi183.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2024-02-13T03:10:32.224 INFO:teuthology.orchestra.run.smithi183.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.6 [1,098 kB] 2024-02-13T03:10:32.227 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2024-02-13T03:10:32.229 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2024-02-13T03:10:32.242 INFO:teuthology.orchestra.run.smithi183.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2024-02-13T03:10:32.243 INFO:teuthology.orchestra.run.smithi183.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2024-02-13T03:10:32.243 INFO:teuthology.orchestra.run.smithi183.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2024-02-13T03:10:32.253 INFO:teuthology.orchestra.run.smithi161.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2024-02-13T03:10:32.260 INFO:teuthology.orchestra.run.smithi161.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2024-02-13T03:10:32.268 INFO:teuthology.orchestra.run.smithi161.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2024-02-13T03:10:32.284 INFO:teuthology.orchestra.run.smithi183.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2024-02-13T03:10:32.284 INFO:teuthology.orchestra.run.smithi183.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2024-02-13T03:10:32.285 INFO:teuthology.orchestra.run.smithi183.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.8 [331 kB] 2024-02-13T03:10:32.289 INFO:teuthology.orchestra.run.smithi183.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2024-02-13T03:10:32.289 INFO:teuthology.orchestra.run.smithi183.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-py all 1.5.2-1ubuntu0.1 [65.3 kB] 2024-02-13T03:10:32.290 INFO:teuthology.orchestra.run.smithi183.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2024-02-13T03:10:32.290 INFO:teuthology.orchestra.run.smithi183.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2024-02-13T03:10:32.292 INFO:teuthology.orchestra.run.smithi183.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2024-02-13T03:10:32.292 INFO:teuthology.orchestra.run.smithi183.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2024-02-13T03:10:32.442 INFO:teuthology.orchestra.run.smithi161.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2024-02-13T03:10:32.734 INFO:teuthology.orchestra.run.smithi161.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2024-02-13T03:10:32.913 INFO:teuthology.orchestra.run.smithi161.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2024-02-13T03:10:32.927 INFO:teuthology.orchestra.run.smithi161.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2024-02-13T03:10:32.938 INFO:teuthology.orchestra.run.smithi161.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2024-02-13T03:10:32.991 INFO:teuthology.orchestra.run.smithi161.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jwt all 1.5.3+ds1-1ubuntu0.1 [16.6 kB] 2024-02-13T03:10:32.995 INFO:teuthology.orchestra.run.smithi161.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2024-02-13T03:10:32.999 INFO:teuthology.orchestra.run.smithi161.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2024-02-13T03:10:33.002 INFO:teuthology.orchestra.run.smithi161.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-mako all 1.0.7+ds1-1ubuntu0.2 [59.4 kB] 2024-02-13T03:10:33.014 INFO:teuthology.orchestra.run.smithi161.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2024-02-13T03:10:33.016 INFO:teuthology.orchestra.run.smithi161.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2024-02-13T03:10:33.181 INFO:teuthology.orchestra.run.smithi161.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2024-02-13T03:10:33.460 INFO:teuthology.orchestra.run.smithi161.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2024-02-13T03:10:33.519 INFO:teuthology.orchestra.run.smithi161.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2024-02-13T03:10:33.577 INFO:teuthology.orchestra.run.smithi161.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pkg-resources all 39.0.1-2ubuntu0.1 [99.0 kB] 2024-02-13T03:10:33.639 INFO:teuthology.orchestra.run.smithi161.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2024-02-13T03:10:33.644 INFO:teuthology.orchestra.run.smithi161.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2024-02-13T03:10:33.763 INFO:teuthology.orchestra.run.smithi161.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2024-02-13T03:10:33.764 INFO:teuthology.orchestra.run.smithi161.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2024-02-13T03:10:33.765 INFO:teuthology.orchestra.run.smithi161.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2024-02-13T03:10:33.770 INFO:teuthology.orchestra.run.smithi161.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2024-02-13T03:10:33.786 INFO:teuthology.orchestra.run.smithi161.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.2 [175 kB] 2024-02-13T03:10:33.978 INFO:teuthology.orchestra.run.smithi161.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2024-02-13T03:10:34.324 INFO:teuthology.orchestra.run.smithi161.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2024-02-13T03:10:34.330 INFO:teuthology.orchestra.run.smithi161.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2024-02-13T03:10:34.348 INFO:teuthology.orchestra.run.smithi161.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2024-02-13T03:10:34.718 INFO:teuthology.orchestra.run.smithi161.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2024-02-13T03:10:34.719 INFO:teuthology.orchestra.run.smithi161.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2024-02-13T03:10:35.021 INFO:teuthology.orchestra.run.smithi161.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2024-02-13T03:10:35.040 INFO:teuthology.orchestra.run.smithi161.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2024-02-13T03:10:35.067 INFO:teuthology.orchestra.run.smithi161.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2024-02-13T03:10:35.098 INFO:teuthology.orchestra.run.smithi161.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2024-02-13T03:10:35.098 INFO:teuthology.orchestra.run.smithi161.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2024-02-13T03:10:35.261 INFO:teuthology.orchestra.run.smithi161.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.3 [28.8 kB] 2024-02-13T03:10:35.464 INFO:teuthology.orchestra.run.smithi161.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2024-02-13T03:10:35.554 INFO:teuthology.orchestra.run.smithi161.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2024-02-13T03:10:35.613 INFO:teuthology.orchestra.run.smithi161.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2024-02-13T03:10:35.645 INFO:teuthology.orchestra.run.smithi161.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2024-02-13T03:10:35.871 INFO:teuthology.orchestra.run.smithi161.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2024-02-13T03:10:35.879 INFO:teuthology.orchestra.run.smithi161.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-setuptools all 39.0.1-2ubuntu0.1 [248 kB] 2024-02-13T03:10:35.908 INFO:teuthology.orchestra.run.smithi161.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2024-02-13T03:10:35.910 INFO:teuthology.orchestra.run.smithi161.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2024-02-13T03:10:35.928 INFO:teuthology.orchestra.run.smithi161.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2024-02-13T03:10:35.938 INFO:teuthology.orchestra.run.smithi161.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2024-02-13T03:10:36.110 INFO:teuthology.orchestra.run.smithi161.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2024-02-13T03:10:36.362 INFO:teuthology.orchestra.run.smithi161.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2024-02-13T03:10:36.615 INFO:teuthology.orchestra.run.smithi161.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.3 [150 kB] 2024-02-13T03:10:36.654 INFO:teuthology.orchestra.run.smithi161.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2024-02-13T03:10:36.726 INFO:teuthology.orchestra.run.smithi161.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.2 [184 kB] 2024-02-13T03:10:36.747 INFO:teuthology.orchestra.run.smithi161.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2024-02-13T03:10:36.749 INFO:teuthology.orchestra.run.smithi161.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2024-02-13T03:10:36.751 INFO:teuthology.orchestra.run.smithi161.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2024-02-13T03:10:36.752 INFO:teuthology.orchestra.run.smithi161.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python-py all 1.5.2-1ubuntu0.1 [65.3 kB] 2024-02-13T03:10:36.758 INFO:teuthology.orchestra.run.smithi161.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2024-02-13T03:10:36.771 INFO:teuthology.orchestra.run.smithi161.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2024-02-13T03:10:36.938 INFO:teuthology.orchestra.run.smithi161.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2024-02-13T03:10:37.021 INFO:teuthology.orchestra.run.smithi161.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2024-02-13T03:10:37.046 INFO:teuthology.orchestra.run.smithi161.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2024-02-13T03:10:37.243 INFO:teuthology.orchestra.run.smithi161.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.6 [1,098 kB] 2024-02-13T03:10:37.548 INFO:teuthology.orchestra.run.smithi161.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2024-02-13T03:10:37.553 INFO:teuthology.orchestra.run.smithi161.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2024-02-13T03:10:37.555 INFO:teuthology.orchestra.run.smithi161.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2024-02-13T03:10:37.730 INFO:teuthology.orchestra.run.smithi161.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2024-02-13T03:10:37.731 INFO:teuthology.orchestra.run.smithi161.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2024-02-13T03:10:37.733 INFO:teuthology.orchestra.run.smithi161.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.8 [331 kB] 2024-02-13T03:10:37.737 INFO:teuthology.orchestra.run.smithi161.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2024-02-13T03:10:37.901 INFO:teuthology.orchestra.run.smithi161.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-py all 1.5.2-1ubuntu0.1 [65.3 kB] 2024-02-13T03:10:38.186 INFO:teuthology.orchestra.run.smithi161.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2024-02-13T03:10:38.207 INFO:teuthology.orchestra.run.smithi161.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2024-02-13T03:10:38.339 INFO:teuthology.orchestra.run.smithi161.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2024-02-13T03:10:38.363 INFO:teuthology.orchestra.run.smithi161.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2024-02-13T03:10:38.611 INFO:teuthology.orchestra.run.smithi183.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 16.2.14-521-g0e714d9a-1bionic [3,968 B] 2024-02-13T03:10:38.611 INFO:teuthology.orchestra.run.smithi161.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 16.2.14-521-g0e714d9a-1bionic [3,968 B] 2024-02-13T03:10:38.708 INFO:teuthology.orchestra.run.smithi183.stdout:Get:84 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.2.14-521-g0e714d9a-1bionic [3,178 kB] 2024-02-13T03:10:38.709 INFO:teuthology.orchestra.run.smithi161.stdout:Get:84 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.2.14-521-g0e714d9a-1bionic [3,178 kB] 2024-02-13T03:10:39.038 INFO:teuthology.orchestra.run.smithi183.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.2.14-521-g0e714d9a-1bionic [3,210 kB] 2024-02-13T03:10:39.055 INFO:teuthology.orchestra.run.smithi161.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.2.14-521-g0e714d9a-1bionic [3,210 kB] 2024-02-13T03:10:39.248 INFO:teuthology.orchestra.run.smithi183.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.2.14-521-g0e714d9a-1bionic [684 kB] 2024-02-13T03:10:39.273 INFO:teuthology.orchestra.run.smithi161.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.2.14-521-g0e714d9a-1bionic [684 kB] 2024-02-13T03:10:39.292 INFO:teuthology.orchestra.run.smithi183.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.2.14-521-g0e714d9a-1bionic [343 kB] 2024-02-13T03:10:39.310 INFO:teuthology.orchestra.run.smithi183.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.2.14-521-g0e714d9a-1bionic [22.2 kB] 2024-02-13T03:10:39.311 INFO:teuthology.orchestra.run.smithi183.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.2.14-521-g0e714d9a-1bionic [178 kB] 2024-02-13T03:10:39.321 INFO:teuthology.orchestra.run.smithi161.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.2.14-521-g0e714d9a-1bionic [343 kB] 2024-02-13T03:10:39.332 INFO:teuthology.orchestra.run.smithi183.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.2.14-521-g0e714d9a-1bionic [38.4 kB] 2024-02-13T03:10:39.333 INFO:teuthology.orchestra.run.smithi183.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.2.14-521-g0e714d9a-1bionic [337 kB] 2024-02-13T03:10:39.336 INFO:teuthology.orchestra.run.smithi161.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.2.14-521-g0e714d9a-1bionic [22.2 kB] 2024-02-13T03:10:39.338 INFO:teuthology.orchestra.run.smithi161.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.2.14-521-g0e714d9a-1bionic [178 kB] 2024-02-13T03:10:39.350 INFO:teuthology.orchestra.run.smithi183.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.2.14-521-g0e714d9a-1bionic [3,433 kB] 2024-02-13T03:10:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.2.14-521-g0e714d9a-1bionic [38.4 kB] 2024-02-13T03:10:39.361 INFO:teuthology.orchestra.run.smithi161.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.2.14-521-g0e714d9a-1bionic [337 kB] 2024-02-13T03:10:39.379 INFO:teuthology.orchestra.run.smithi161.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.2.14-521-g0e714d9a-1bionic [3,433 kB] 2024-02-13T03:10:39.579 INFO:teuthology.orchestra.run.smithi183.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.2.14-521-g0e714d9a-1bionic [99.6 kB] 2024-02-13T03:10:39.582 INFO:teuthology.orchestra.run.smithi183.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.2.14-521-g0e714d9a-1bionic [416 kB] 2024-02-13T03:10:39.607 INFO:teuthology.orchestra.run.smithi161.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.2.14-521-g0e714d9a-1bionic [99.6 kB] 2024-02-13T03:10:39.610 INFO:teuthology.orchestra.run.smithi161.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.2.14-521-g0e714d9a-1bionic [416 kB] 2024-02-13T03:10:39.675 INFO:teuthology.orchestra.run.smithi183.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.2.14-521-g0e714d9a-1bionic [20.4 MB] 2024-02-13T03:10:39.675 INFO:teuthology.orchestra.run.smithi161.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.2.14-521-g0e714d9a-1bionic [20.4 MB] 2024-02-13T03:10:47.758 INFO:teuthology.orchestra.run.smithi161.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.2.14-521-g0e714d9a-1bionic [4,565 kB] 2024-02-13T03:10:47.777 INFO:teuthology.orchestra.run.smithi183.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.2.14-521-g0e714d9a-1bionic [4,565 kB] 2024-02-13T03:10:48.044 INFO:teuthology.orchestra.run.smithi183.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.2.14-521-g0e714d9a-1bionic [204 kB] 2024-02-13T03:10:48.045 INFO:teuthology.orchestra.run.smithi161.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.2.14-521-g0e714d9a-1bionic [204 kB] 2024-02-13T03:10:48.065 INFO:teuthology.orchestra.run.smithi183.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 libsqlite3-mod-ceph amd64 16.2.14-521-g0e714d9a-1bionic [122 kB] 2024-02-13T03:10:48.066 INFO:teuthology.orchestra.run.smithi161.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 libsqlite3-mod-ceph amd64 16.2.14-521-g0e714d9a-1bionic [122 kB] 2024-02-13T03:10:48.109 INFO:teuthology.orchestra.run.smithi183.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.2.14-521-g0e714d9a-1bionic [1,442 kB] 2024-02-13T03:10:48.109 INFO:teuthology.orchestra.run.smithi161.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.2.14-521-g0e714d9a-1bionic [1,442 kB] 2024-02-13T03:10:48.206 INFO:teuthology.orchestra.run.smithi161.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.2.14-521-g0e714d9a-1bionic [3,784 kB] 2024-02-13T03:10:48.212 INFO:teuthology.orchestra.run.smithi183.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.2.14-521-g0e714d9a-1bionic [3,784 kB] 2024-02-13T03:10:48.443 INFO:teuthology.orchestra.run.smithi161.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.2.14-521-g0e714d9a-1bionic [20.1 MB] 2024-02-13T03:10:48.468 INFO:teuthology.orchestra.run.smithi183.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.2.14-521-g0e714d9a-1bionic [20.1 MB] 2024-02-13T03:10:49.703 INFO:teuthology.orchestra.run.smithi161.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.2.14-521-g0e714d9a-1bionic [4,060 B] 2024-02-13T03:10:49.704 INFO:teuthology.orchestra.run.smithi161.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.2.14-521-g0e714d9a-1bionic [792 kB] 2024-02-13T03:10:49.749 INFO:teuthology.orchestra.run.smithi161.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.2.14-521-g0e714d9a-1bionic [2,201 kB] 2024-02-13T03:10:49.860 INFO:teuthology.orchestra.run.smithi183.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.2.14-521-g0e714d9a-1bionic [4,060 B] 2024-02-13T03:10:49.860 INFO:teuthology.orchestra.run.smithi183.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.2.14-521-g0e714d9a-1bionic [792 kB] 2024-02-13T03:10:49.890 INFO:teuthology.orchestra.run.smithi161.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.2.14-521-g0e714d9a-1bionic [77.4 kB] 2024-02-13T03:10:49.895 INFO:teuthology.orchestra.run.smithi161.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.2.14-521-g0e714d9a-1bionic [100 kB] 2024-02-13T03:10:49.900 INFO:teuthology.orchestra.run.smithi161.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.2.14-521-g0e714d9a-1bionic [2,824 kB] 2024-02-13T03:10:49.909 INFO:teuthology.orchestra.run.smithi183.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.2.14-521-g0e714d9a-1bionic [2,201 kB] 2024-02-13T03:10:50.061 INFO:teuthology.orchestra.run.smithi183.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.2.14-521-g0e714d9a-1bionic [77.4 kB] 2024-02-13T03:10:50.065 INFO:teuthology.orchestra.run.smithi183.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.2.14-521-g0e714d9a-1bionic [100 kB] 2024-02-13T03:10:50.073 INFO:teuthology.orchestra.run.smithi183.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.2.14-521-g0e714d9a-1bionic [2,824 kB] 2024-02-13T03:10:50.077 INFO:teuthology.orchestra.run.smithi161.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.2.14-521-g0e714d9a-1bionic [7,874 kB] 2024-02-13T03:10:50.272 INFO:teuthology.orchestra.run.smithi183.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.2.14-521-g0e714d9a-1bionic [7,874 kB] 2024-02-13T03:10:50.572 INFO:teuthology.orchestra.run.smithi161.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.2.14-521-g0e714d9a-1bionic [4,176 B] 2024-02-13T03:10:50.572 INFO:teuthology.orchestra.run.smithi161.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.2.14-521-g0e714d9a-1bionic [42.8 MB] 2024-02-13T03:10:50.856 INFO:teuthology.orchestra.run.smithi183.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.2.14-521-g0e714d9a-1bionic [4,176 B] 2024-02-13T03:10:50.856 INFO:teuthology.orchestra.run.smithi183.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.2.14-521-g0e714d9a-1bionic [42.8 MB] 2024-02-13T03:10:52.906 INFO:teuthology.orchestra.run.smithi161.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.2.14-521-g0e714d9a-1bionic [21.5 kB] 2024-02-13T03:10:52.906 INFO:teuthology.orchestra.run.smithi161.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.2.14-521-g0e714d9a-1bionic [10.5 MB] 2024-02-13T03:10:53.478 INFO:teuthology.orchestra.run.smithi161.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.2.14-521-g0e714d9a-1bionic [78.5 kB] 2024-02-13T03:10:53.859 INFO:teuthology.orchestra.run.smithi183.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.2.14-521-g0e714d9a-1bionic [21.5 kB] 2024-02-13T03:10:53.859 INFO:teuthology.orchestra.run.smithi183.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.2.14-521-g0e714d9a-1bionic [10.5 MB] 2024-02-13T03:10:54.238 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 156 MB in 22s (7,109 kB/s) 2024-02-13T03:10:54.275 INFO:teuthology.orchestra.run.smithi183.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.2.14-521-g0e714d9a-1bionic [78.5 kB] 2024-02-13T03:10:54.360 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-uritemplate. 2024-02-13T03:10:54.424 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172445 files and directories currently installed.) 2024-02-13T03:10:54.429 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2024-02-13T03:10:54.492 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2024-02-13T03:10:54.702 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libjaeger. 2024-02-13T03:10:54.721 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../001-libjaeger_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:54.752 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libjaeger (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:10:55.025 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 156 MB in 23s (6,899 kB/s) 2024-02-13T03:10:55.080 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-02-13T03:10:55.096 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../002-liblttng-ust0_2.10.1-1_amd64.deb ... 2024-02-13T03:10:55.151 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-uritemplate. 2024-02-13T03:10:55.180 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2024-02-13T03:10:55.209 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172445 files and directories currently installed.) 2024-02-13T03:10:55.214 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2024-02-13T03:10:55.249 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2024-02-13T03:10:55.485 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libjaeger. 2024-02-13T03:10:55.503 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../001-libjaeger_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:55.532 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../003-librbd1_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:55.543 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libjaeger (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:10:55.659 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librbd1 (16.2.14-521-g0e714d9a-1bionic) over (12.2.13-0ubuntu0.18.04.11) ... 2024-02-13T03:10:55.862 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-02-13T03:10:55.880 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../002-liblttng-ust0_2.10.1-1_amd64.deb ... 2024-02-13T03:10:55.962 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2024-02-13T03:10:56.289 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../004-librados2_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:56.300 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../003-librbd1_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:56.400 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librbd1 (16.2.14-521-g0e714d9a-1bionic) over (12.2.13-0ubuntu0.18.04.11) ... 2024-02-13T03:10:56.412 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librados2 (16.2.14-521-g0e714d9a-1bionic) over (12.2.13-0ubuntu0.18.04.11) ... 2024-02-13T03:10:57.050 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libcephfs2. 2024-02-13T03:10:57.069 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../005-libcephfs2_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:57.069 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../004-librados2_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:57.108 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libcephfs2 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:10:57.195 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librados2 (16.2.14-521-g0e714d9a-1bionic) over (12.2.13-0ubuntu0.18.04.11) ... 2024-02-13T03:10:57.435 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rados. 2024-02-13T03:10:57.454 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../006-python3-rados_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:57.493 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rados (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:10:57.788 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-02-13T03:10:57.804 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../007-python3-ceph-argparse_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:10:57.807 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libcephfs2. 2024-02-13T03:10:57.825 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../005-libcephfs2_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:57.837 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-ceph-argparse (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:10:57.857 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libcephfs2 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:10:58.132 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cephfs. 2024-02-13T03:10:58.148 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../008-python3-cephfs_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:58.176 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rados. 2024-02-13T03:10:58.190 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cephfs (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:10:58.193 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../006-python3-rados_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:58.235 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rados (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:10:58.543 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-ceph-common. 2024-02-13T03:10:58.559 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../009-python3-ceph-common_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:10:58.587 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-02-13T03:10:58.601 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-ceph-common (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:10:58.604 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../007-python3-ceph-argparse_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:10:58.646 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-ceph-argparse (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:10:58.904 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-prettytable. 2024-02-13T03:10:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../010-python3-prettytable_0.7.2-3_all.deb ... 2024-02-13T03:10:58.923 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cephfs. 2024-02-13T03:10:58.940 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../008-python3-cephfs_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2024-02-13T03:10:58.982 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cephfs (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:10:59.268 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-ceph-common. 2024-02-13T03:10:59.283 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../009-python3-ceph-common_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:10:59.318 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-ceph-common (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:10:59.373 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rbd. 2024-02-13T03:10:59.391 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../011-python3-rbd_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:59.423 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rbd (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:10:59.587 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-prettytable. 2024-02-13T03:10:59.606 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../010-python3-prettytable_0.7.2-3_all.deb ... 2024-02-13T03:10:59.637 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2024-02-13T03:10:59.742 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-02-13T03:10:59.761 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../012-liblua5.3-0_5.3.3-1ubuntu0.18.04.1_amd64.deb ... 2024-02-13T03:10:59.800 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1ubuntu0.18.04.1) ... 2024-02-13T03:10:59.890 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rbd. 2024-02-13T03:10:59.907 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../011-python3-rbd_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:10:59.948 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rbd (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:00.153 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-02-13T03:11:00.171 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../013-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2024-02-13T03:11:00.228 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2024-02-13T03:11:00.317 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-02-13T03:11:00.333 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../012-liblua5.3-0_5.3.3-1ubuntu0.18.04.1_amd64.deb ... 2024-02-13T03:11:00.375 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1ubuntu0.18.04.1) ... 2024-02-13T03:11:00.514 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-02-13T03:11:00.530 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../014-librdkafka1_0.11.3-1build1_amd64.deb ... 2024-02-13T03:11:00.564 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2024-02-13T03:11:00.686 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-02-13T03:11:00.704 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../013-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2024-02-13T03:11:00.736 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2024-02-13T03:11:00.850 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librgw2. 2024-02-13T03:11:00.865 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../015-librgw2_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:00.900 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librgw2 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:01.056 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-02-13T03:11:01.073 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../014-librdkafka1_0.11.3-1build1_amd64.deb ... 2024-02-13T03:11:01.114 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2024-02-13T03:11:01.450 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librgw2. 2024-02-13T03:11:01.466 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../015-librgw2_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:01.477 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rgw. 2024-02-13T03:11:01.495 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../016-python3-rgw_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:01.509 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librgw2 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:01.535 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rgw (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:01.846 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2024-02-13T03:11:01.863 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../017-libsnappy1v5_1.1.7-1_amd64.deb ... 2024-02-13T03:11:01.905 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2024-02-13T03:11:02.120 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rgw. 2024-02-13T03:11:02.138 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../016-python3-rgw_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:02.178 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rgw (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:02.182 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2024-02-13T03:11:02.202 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../018-libleveldb1v5_1.20-2_amd64.deb ... 2024-02-13T03:11:02.232 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2024-02-13T03:11:02.447 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2024-02-13T03:11:02.464 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../017-libsnappy1v5_1.1.7-1_amd64.deb ... 2024-02-13T03:11:02.514 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2024-02-13T03:11:02.576 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liboath0. 2024-02-13T03:11:02.595 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../019-liboath0_2.6.1-1_amd64.deb ... 2024-02-13T03:11:02.637 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liboath0 (2.6.1-1) ... 2024-02-13T03:11:02.956 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libradosstriper1. 2024-02-13T03:11:02.972 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../020-libradosstriper1_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:03.016 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libradosstriper1 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:03.438 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-common. 2024-02-13T03:11:03.455 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../021-ceph-common_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:03.496 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-common (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:03.583 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2024-02-13T03:11:03.601 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../018-libleveldb1v5_1.20-2_amd64.deb ... 2024-02-13T03:11:03.641 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2024-02-13T03:11:04.003 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liboath0. 2024-02-13T03:11:04.020 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../019-liboath0_2.6.1-1_amd64.deb ... 2024-02-13T03:11:04.053 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liboath0 (2.6.1-1) ... 2024-02-13T03:11:04.339 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libradosstriper1. 2024-02-13T03:11:04.358 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../020-libradosstriper1_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:04.405 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libradosstriper1 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:04.850 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-common. 2024-02-13T03:11:04.866 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../021-ceph-common_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:04.910 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-common (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:05.885 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-base. 2024-02-13T03:11:05.900 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../022-ceph-base_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:05.954 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-base (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:06.790 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-dateutil. 2024-02-13T03:11:06.806 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../023-python3-dateutil_2.6.1-1_all.deb ... 2024-02-13T03:11:06.849 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2024-02-13T03:11:06.868 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-base. 2024-02-13T03:11:06.885 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../022-ceph-base_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:06.937 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-base (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:07.161 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-asn1crypto. 2024-02-13T03:11:07.179 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../024-python3-asn1crypto_0.24.0-1_all.deb ... 2024-02-13T03:11:07.220 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2024-02-13T03:11:07.456 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cffi-backend. 2024-02-13T03:11:07.472 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../025-python3-cffi-backend_1.11.5-1_amd64.deb ... 2024-02-13T03:11:07.514 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2024-02-13T03:11:07.762 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-dateutil. 2024-02-13T03:11:07.780 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../023-python3-dateutil_2.6.1-1_all.deb ... 2024-02-13T03:11:07.820 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2024-02-13T03:11:07.833 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cryptography. 2024-02-13T03:11:07.849 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../026-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2024-02-13T03:11:07.891 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2024-02-13T03:11:08.125 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-asn1crypto. 2024-02-13T03:11:08.141 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../024-python3-asn1crypto_0.24.0-1_all.deb ... 2024-02-13T03:11:08.193 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2024-02-13T03:11:08.261 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-openssl. 2024-02-13T03:11:08.281 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../027-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2024-02-13T03:11:08.319 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2024-02-13T03:11:08.479 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cffi-backend. 2024-02-13T03:11:08.496 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../025-python3-cffi-backend_1.11.5-1_amd64.deb ... 2024-02-13T03:11:08.538 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2024-02-13T03:11:08.563 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-02-13T03:11:08.580 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:11:08.622 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-modules-core (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:08.827 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cryptography. 2024-02-13T03:11:08.846 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../026-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2024-02-13T03:11:08.886 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2024-02-13T03:11:09.009 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-bcrypt. 2024-02-13T03:11:09.029 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.4-2_amd64.deb ... 2024-02-13T03:11:09.068 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2024-02-13T03:11:09.251 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-openssl. 2024-02-13T03:11:09.266 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../027-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2024-02-13T03:11:09.309 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2024-02-13T03:11:09.353 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cherrypy3. 2024-02-13T03:11:09.372 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-2_all.deb ... 2024-02-13T03:11:09.412 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2024-02-13T03:11:09.578 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-02-13T03:11:09.595 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:11:09.636 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-modules-core (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:09.748 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jwt. 2024-02-13T03:11:09.764 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../031-python3-jwt_1.5.3+ds1-1ubuntu0.1_all.deb ... 2024-02-13T03:11:09.807 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jwt (1.5.3+ds1-1ubuntu0.1) ... 2024-02-13T03:11:10.008 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-bcrypt. 2024-02-13T03:11:10.026 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.4-2_amd64.deb ... 2024-02-13T03:11:10.066 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2024-02-13T03:11:10.114 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-logutils. 2024-02-13T03:11:10.131 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../032-python3-logutils_0.3.3-5_all.deb ... 2024-02-13T03:11:10.172 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-logutils (0.3.3-5) ... 2024-02-13T03:11:10.368 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cherrypy3. 2024-02-13T03:11:10.385 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-2_all.deb ... 2024-02-13T03:11:10.427 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2024-02-13T03:11:10.493 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-markupsafe. 2024-02-13T03:11:10.509 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../033-python3-markupsafe_1.0-1build1_amd64.deb ... 2024-02-13T03:11:10.552 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2024-02-13T03:11:10.771 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jwt. 2024-02-13T03:11:10.790 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../031-python3-jwt_1.5.3+ds1-1ubuntu0.1_all.deb ... 2024-02-13T03:11:10.804 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-mako. 2024-02-13T03:11:10.821 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../034-python3-mako_1.0.7+ds1-1ubuntu0.2_all.deb ... 2024-02-13T03:11:10.829 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jwt (1.5.3+ds1-1ubuntu0.1) ... 2024-02-13T03:11:10.854 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-mako (1.0.7+ds1-1ubuntu0.2) ... 2024-02-13T03:11:11.132 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-logutils. 2024-02-13T03:11:11.148 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-simplegeneric. 2024-02-13T03:11:11.149 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../032-python3-logutils_0.3.3-5_all.deb ... 2024-02-13T03:11:11.165 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../035-python3-simplegeneric_0.8.1-1_all.deb ... 2024-02-13T03:11:11.182 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-logutils (0.3.3-5) ... 2024-02-13T03:11:11.207 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2024-02-13T03:11:11.435 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-markupsafe. 2024-02-13T03:11:11.454 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../033-python3-markupsafe_1.0-1build1_amd64.deb ... 2024-02-13T03:11:11.493 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2024-02-13T03:11:11.509 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-singledispatch. 2024-02-13T03:11:11.527 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../036-python3-singledispatch_3.4.0.3-2_all.deb ... 2024-02-13T03:11:11.567 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2024-02-13T03:11:11.779 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-mako. 2024-02-13T03:11:11.798 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../034-python3-mako_1.0.7+ds1-1ubuntu0.2_all.deb ... 2024-02-13T03:11:11.837 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-mako (1.0.7+ds1-1ubuntu0.2) ... 2024-02-13T03:11:11.843 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-webob. 2024-02-13T03:11:11.860 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../037-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2024-02-13T03:11:11.902 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2024-02-13T03:11:12.157 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-simplegeneric. 2024-02-13T03:11:12.175 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../035-python3-simplegeneric_0.8.1-1_all.deb ... 2024-02-13T03:11:12.209 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-bs4. 2024-02-13T03:11:12.215 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2024-02-13T03:11:12.226 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../038-python3-bs4_4.6.0-1_all.deb ... 2024-02-13T03:11:12.267 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2024-02-13T03:11:12.491 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-singledispatch. 2024-02-13T03:11:12.508 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../036-python3-singledispatch_3.4.0.3-2_all.deb ... 2024-02-13T03:11:12.549 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2024-02-13T03:11:12.575 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-waitress. 2024-02-13T03:11:12.594 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../039-python3-waitress_1.0.1-1_all.deb ... 2024-02-13T03:11:12.633 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-waitress (1.0.1-1) ... 2024-02-13T03:11:12.852 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-webob. 2024-02-13T03:11:12.868 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../037-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2024-02-13T03:11:12.910 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2024-02-13T03:11:12.985 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../040-python3-pkg-resources_39.0.1-2ubuntu0.1_all.deb ... 2024-02-13T03:11:13.205 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-bs4. 2024-02-13T03:11:13.220 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../038-python3-bs4_4.6.0-1_all.deb ... 2024-02-13T03:11:13.254 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pkg-resources (39.0.1-2ubuntu0.1) over (39.0.1-2) ... 2024-02-13T03:11:13.263 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2024-02-13T03:11:13.541 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-waitress. 2024-02-13T03:11:13.559 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../039-python3-waitress_1.0.1-1_all.deb ... 2024-02-13T03:11:13.599 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-waitress (1.0.1-1) ... 2024-02-13T03:11:13.650 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-tempita. 2024-02-13T03:11:13.665 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2024-02-13T03:11:13.708 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-tempita (0.5.2-2) ... 2024-02-13T03:11:13.933 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../040-python3-pkg-resources_39.0.1-2ubuntu0.1_all.deb ... 2024-02-13T03:11:14.036 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-paste. 2024-02-13T03:11:14.052 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2024-02-13T03:11:14.094 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2024-02-13T03:11:14.190 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pkg-resources (39.0.1-2ubuntu0.1) over (39.0.1-2) ... 2024-02-13T03:11:14.430 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-02-13T03:11:14.448 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2024-02-13T03:11:14.488 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2024-02-13T03:11:14.583 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-tempita. 2024-02-13T03:11:14.601 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2024-02-13T03:11:14.641 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-tempita (0.5.2-2) ... 2024-02-13T03:11:14.799 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pastedeploy. 2024-02-13T03:11:14.816 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2024-02-13T03:11:14.858 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2024-02-13T03:11:14.902 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-paste. 2024-02-13T03:11:14.918 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2024-02-13T03:11:14.961 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2024-02-13T03:11:15.160 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-webtest. 2024-02-13T03:11:15.176 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2024-02-13T03:11:15.210 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2024-02-13T03:11:15.256 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-02-13T03:11:15.266 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2024-02-13T03:11:15.297 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2024-02-13T03:11:15.530 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pecan. 2024-02-13T03:11:15.545 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2024-02-13T03:11:15.575 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pastedeploy. 2024-02-13T03:11:15.588 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pecan (1.2.1-2) ... 2024-02-13T03:11:15.592 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2024-02-13T03:11:15.633 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2024-02-13T03:11:15.943 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-werkzeug. 2024-02-13T03:11:15.960 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.2_all.deb ... 2024-02-13T03:11:15.961 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-webtest. 2024-02-13T03:11:15.977 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2024-02-13T03:11:16.001 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.2) ... 2024-02-13T03:11:16.019 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2024-02-13T03:11:16.321 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-02-13T03:11:16.337 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../048-libsqlite3-mod-ceph_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:16.364 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pecan. 2024-02-13T03:11:16.379 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libsqlite3-mod-ceph (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:16.381 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2024-02-13T03:11:16.422 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pecan (1.2.1-2) ... 2024-02-13T03:11:16.707 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr. 2024-02-13T03:11:16.725 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../049-ceph-mgr_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:16.764 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:16.776 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-werkzeug. 2024-02-13T03:11:16.795 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.2_all.deb ... 2024-02-13T03:11:16.835 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.2) ... 2024-02-13T03:11:17.171 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-02-13T03:11:17.188 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../048-libsqlite3-mod-ceph_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:17.226 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mon. 2024-02-13T03:11:17.229 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libsqlite3-mod-ceph (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:17.243 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../050-ceph-mon_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:17.284 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mon (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:17.532 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr. 2024-02-13T03:11:17.548 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../049-ceph-mgr_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:17.577 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:17.963 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-osd. 2024-02-13T03:11:17.979 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../051-ceph-osd_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:18.021 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-osd (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:18.041 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mon. 2024-02-13T03:11:18.057 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../050-ceph-mon_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:18.092 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mon (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:18.837 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-osd. 2024-02-13T03:11:18.853 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../051-ceph-osd_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:18.895 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-osd (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:19.701 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph. 2024-02-13T03:11:19.718 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../052-ceph_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:19.759 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:20.062 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-fuse. 2024-02-13T03:11:20.079 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../053-ceph-fuse_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:20.111 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-fuse (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:20.515 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mds. 2024-02-13T03:11:20.531 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../054-ceph-mds_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:20.559 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph. 2024-02-13T03:11:20.573 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mds (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:20.578 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../052-ceph_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:20.617 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:20.936 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-fuse. 2024-02-13T03:11:20.953 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../053-ceph-fuse_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:20.994 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-fuse (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:21.084 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package cephadm. 2024-02-13T03:11:21.100 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../055-cephadm_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:21.142 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking cephadm (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:21.390 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mds. 2024-02-13T03:11:21.408 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../054-ceph-mds_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:21.436 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jinja2. 2024-02-13T03:11:21.448 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mds (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:21.454 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../056-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2024-02-13T03:11:21.495 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2024-02-13T03:11:21.789 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-02-13T03:11:21.807 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:11:21.847 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-cephadm (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:21.942 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package cephadm. 2024-02-13T03:11:21.958 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../055-cephadm_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:21.992 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking cephadm (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:22.184 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-repoze.lru. 2024-02-13T03:11:22.200 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-1_all.deb ... 2024-02-13T03:11:22.242 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2024-02-13T03:11:22.311 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jinja2. 2024-02-13T03:11:22.331 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../056-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2024-02-13T03:11:22.369 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2024-02-13T03:11:22.545 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-routes. 2024-02-13T03:11:22.563 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../059-python3-routes_2.4.1-1_all.deb ... 2024-02-13T03:11:22.594 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-routes (2.4.1-1) ... 2024-02-13T03:11:22.672 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-02-13T03:11:22.689 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:11:22.731 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-cephadm (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:22.880 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-02-13T03:11:22.895 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../060-ceph-mgr-dashboard_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:11:22.938 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-dashboard (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:23.058 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-repoze.lru. 2024-02-13T03:11:23.076 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-1_all.deb ... 2024-02-13T03:11:23.117 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2024-02-13T03:11:23.369 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-routes. 2024-02-13T03:11:23.386 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../059-python3-routes_2.4.1-1_all.deb ... 2024-02-13T03:11:23.436 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-routes (2.4.1-1) ... 2024-02-13T03:11:23.747 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-02-13T03:11:23.764 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../060-ceph-mgr-dashboard_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:11:23.830 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-dashboard (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:23.899 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-numpy. 2024-02-13T03:11:23.919 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2024-02-13T03:11:23.948 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2024-02-13T03:11:24.591 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-decorator. 2024-02-13T03:11:24.608 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2024-02-13T03:11:24.640 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-decorator (4.1.2-1) ... 2024-02-13T03:11:24.763 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-numpy. 2024-02-13T03:11:24.780 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2024-02-13T03:11:24.821 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2024-02-13T03:11:24.963 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-scipy. 2024-02-13T03:11:24.982 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2024-02-13T03:11:25.021 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2024-02-13T03:11:25.418 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-decorator. 2024-02-13T03:11:25.434 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2024-02-13T03:11:25.476 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-decorator (4.1.2-1) ... 2024-02-13T03:11:25.812 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-scipy. 2024-02-13T03:11:25.831 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2024-02-13T03:11:25.874 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2024-02-13T03:11:26.147 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-02-13T03:11:26.164 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2024-02-13T03:11:26.205 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2024-02-13T03:11:26.630 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-joblib. 2024-02-13T03:11:26.646 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2024-02-13T03:11:26.688 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-joblib (0.11-1) ... 2024-02-13T03:11:27.000 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-sklearn. 2024-02-13T03:11:27.016 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2024-02-13T03:11:27.025 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-02-13T03:11:27.042 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2024-02-13T03:11:27.049 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2024-02-13T03:11:27.078 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2024-02-13T03:11:27.548 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-joblib. 2024-02-13T03:11:27.565 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2024-02-13T03:11:27.581 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-02-13T03:11:27.597 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:11:27.615 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-joblib (0.11-1) ... 2024-02-13T03:11:27.639 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-diskprediction-local (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:27.968 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-sklearn. 2024-02-13T03:11:27.985 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2024-02-13T03:11:28.027 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2024-02-13T03:11:28.725 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-monotonic. 2024-02-13T03:11:28.741 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2024-02-13T03:11:28.783 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-monotonic (1.1-2) ... 2024-02-13T03:11:28.843 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-02-13T03:11:28.861 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:11:28.902 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-diskprediction-local (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:29.078 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-fasteners. 2024-02-13T03:11:29.097 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2024-02-13T03:11:29.127 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2024-02-13T03:11:29.438 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-httplib2. 2024-02-13T03:11:29.456 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb ... 2024-02-13T03:11:29.496 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2024-02-13T03:11:29.791 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyasn1. 2024-02-13T03:11:29.808 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2024-02-13T03:11:29.907 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2024-02-13T03:11:30.029 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-monotonic. 2024-02-13T03:11:30.049 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2024-02-13T03:11:30.087 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-monotonic (1.1-2) ... 2024-02-13T03:11:30.348 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-fasteners. 2024-02-13T03:11:30.365 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2024-02-13T03:11:30.394 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-02-13T03:11:30.406 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2024-02-13T03:11:30.414 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2024-02-13T03:11:30.452 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2024-02-13T03:11:30.718 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-httplib2. 2024-02-13T03:11:30.734 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb ... 2024-02-13T03:11:30.771 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rsa. 2024-02-13T03:11:30.776 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2024-02-13T03:11:30.788 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2024-02-13T03:11:30.829 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rsa (3.4.2-1) ... 2024-02-13T03:11:31.070 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyasn1. 2024-02-13T03:11:31.086 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2024-02-13T03:11:31.128 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2024-02-13T03:11:31.141 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-sqlalchemy. 2024-02-13T03:11:31.159 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2024-02-13T03:11:31.191 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2024-02-13T03:11:31.473 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-02-13T03:11:31.492 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2024-02-13T03:11:31.531 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2024-02-13T03:11:31.636 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-oauth2client. 2024-02-13T03:11:31.653 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2024-02-13T03:11:31.694 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2024-02-13T03:11:31.867 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rsa. 2024-02-13T03:11:31.887 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2024-02-13T03:11:31.925 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rsa (3.4.2-1) ... 2024-02-13T03:11:32.014 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-setuptools. 2024-02-13T03:11:32.032 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2ubuntu0.1_all.deb ... 2024-02-13T03:11:32.072 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-setuptools (39.0.1-2ubuntu0.1) ... 2024-02-13T03:11:32.245 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-sqlalchemy. 2024-02-13T03:11:32.262 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2024-02-13T03:11:32.303 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2024-02-13T03:11:32.450 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-websocket. 2024-02-13T03:11:32.468 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2024-02-13T03:11:32.508 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2024-02-13T03:11:32.723 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-oauth2client. 2024-02-13T03:11:32.741 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2024-02-13T03:11:32.781 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2024-02-13T03:11:32.862 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-kubernetes. 2024-02-13T03:11:32.880 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2024-02-13T03:11:32.920 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2024-02-13T03:11:33.093 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-setuptools. 2024-02-13T03:11:33.112 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2ubuntu0.1_all.deb ... 2024-02-13T03:11:33.151 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-setuptools (39.0.1-2ubuntu0.1) ... 2024-02-13T03:11:33.340 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-02-13T03:11:33.358 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:11:33.476 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-k8sevents (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:33.496 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-websocket. 2024-02-13T03:11:33.513 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2024-02-13T03:11:33.545 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2024-02-13T03:11:33.812 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libonig4:amd64. 2024-02-13T03:11:33.831 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2024-02-13T03:11:33.857 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-kubernetes. 2024-02-13T03:11:33.870 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2024-02-13T03:11:33.873 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2024-02-13T03:11:33.915 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2024-02-13T03:11:34.207 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libjq1:amd64. 2024-02-13T03:11:34.226 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2024-02-13T03:11:34.265 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2024-02-13T03:11:34.327 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-02-13T03:11:34.345 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_16.2.14-521-g0e714d9a-1bionic_all.deb ... 2024-02-13T03:11:34.471 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-k8sevents (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:34.559 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package jq. 2024-02-13T03:11:34.579 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2024-02-13T03:11:34.609 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking jq (1.5+dfsg-2) ... 2024-02-13T03:11:34.790 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libonig4:amd64. 2024-02-13T03:11:34.807 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2024-02-13T03:11:34.849 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2024-02-13T03:11:34.853 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package socat. 2024-02-13T03:11:34.869 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2024-02-13T03:11:34.911 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2024-02-13T03:11:35.202 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libjq1:amd64. 2024-02-13T03:11:35.218 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2024-02-13T03:11:35.260 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2024-02-13T03:11:35.299 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libxslt1.1:amd64. 2024-02-13T03:11:35.318 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.3_amd64.deb ... 2024-02-13T03:11:35.357 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.3) ... 2024-02-13T03:11:35.504 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package jq. 2024-02-13T03:11:35.521 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2024-02-13T03:11:35.562 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking jq (1.5+dfsg-2) ... 2024-02-13T03:11:35.626 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package xmlstarlet. 2024-02-13T03:11:35.646 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2024-02-13T03:11:35.685 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2024-02-13T03:11:35.840 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package socat. 2024-02-13T03:11:35.857 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2024-02-13T03:11:35.898 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2024-02-13T03:11:36.015 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-test. 2024-02-13T03:11:36.032 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../086-ceph-test_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:36.073 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-test (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:36.285 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libxslt1.1:amd64. 2024-02-13T03:11:36.302 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.3_amd64.deb ... 2024-02-13T03:11:36.343 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.3) ... 2024-02-13T03:11:36.613 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package xmlstarlet. 2024-02-13T03:11:36.630 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2024-02-13T03:11:36.671 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2024-02-13T03:11:37.002 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-test. 2024-02-13T03:11:37.018 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../086-ceph-test_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:37.060 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-test (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:39.343 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libcephfs-dev. 2024-02-13T03:11:39.360 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../087-libcephfs-dev_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:39.401 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libcephfs-dev (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:39.720 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package nvme-cli. 2024-02-13T03:11:39.740 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.2_amd64.deb ... 2024-02-13T03:11:39.778 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking nvme-cli (1.5-1ubuntu1.2) ... 2024-02-13T03:11:40.132 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-attr. 2024-02-13T03:11:40.149 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2024-02-13T03:11:40.190 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-attr (17.4.0-2) ... 2024-02-13T03:11:40.246 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libcephfs-dev. 2024-02-13T03:11:40.266 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../087-libcephfs-dev_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:40.304 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libcephfs-dev (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:40.493 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-funcsigs. 2024-02-13T03:11:40.513 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2024-02-13T03:11:40.551 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2024-02-13T03:11:40.556 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package nvme-cli. 2024-02-13T03:11:40.576 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.2_amd64.deb ... 2024-02-13T03:11:40.606 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking nvme-cli (1.5-1ubuntu1.2) ... 2024-02-13T03:11:40.837 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-pluggy. 2024-02-13T03:11:40.856 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2024-02-13T03:11:40.895 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-pluggy (0.6.0-1) ... 2024-02-13T03:11:40.935 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-attr. 2024-02-13T03:11:40.952 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2024-02-13T03:11:40.993 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-attr (17.4.0-2) ... 2024-02-13T03:11:41.181 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-py. 2024-02-13T03:11:41.201 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../092-python-py_1.5.2-1ubuntu0.1_all.deb ... 2024-02-13T03:11:41.239 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-py (1.5.2-1ubuntu0.1) ... 2024-02-13T03:11:41.279 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-funcsigs. 2024-02-13T03:11:41.299 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2024-02-13T03:11:41.337 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2024-02-13T03:11:41.601 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-pytest. 2024-02-13T03:11:41.617 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2024-02-13T03:11:41.640 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-pluggy. 2024-02-13T03:11:41.657 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2024-02-13T03:11:41.659 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-pytest (3.3.2-2) ... 2024-02-13T03:11:41.714 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-pluggy (0.6.0-1) ... 2024-02-13T03:11:41.979 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-attr. 2024-02-13T03:11:41.992 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-py. 2024-02-13T03:11:41.996 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2024-02-13T03:11:42.011 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../092-python-py_1.5.2-1ubuntu0.1_all.deb ... 2024-02-13T03:11:42.037 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-attr (17.4.0-2) ... 2024-02-13T03:11:42.051 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-py (1.5.2-1ubuntu0.1) ... 2024-02-13T03:11:42.354 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-pytest. 2024-02-13T03:11:42.356 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cycler. 2024-02-13T03:11:42.372 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2024-02-13T03:11:42.373 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2024-02-13T03:11:42.403 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-pytest (3.3.2-2) ... 2024-02-13T03:11:42.414 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cycler (0.10.0-1) ... 2024-02-13T03:11:42.698 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-attr. 2024-02-13T03:11:42.709 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-webencodings. 2024-02-13T03:11:42.714 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2024-02-13T03:11:42.727 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2024-02-13T03:11:42.756 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-attr (17.4.0-2) ... 2024-02-13T03:11:42.767 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-webencodings (0.5-2) ... 2024-02-13T03:11:43.017 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cycler. 2024-02-13T03:11:43.034 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2024-02-13T03:11:43.036 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-html5lib. 2024-02-13T03:11:43.054 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2024-02-13T03:11:43.067 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cycler (0.10.0-1) ... 2024-02-13T03:11:43.103 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2024-02-13T03:11:43.369 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-webencodings. 2024-02-13T03:11:43.386 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2024-02-13T03:11:43.428 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-webencodings (0.5-2) ... 2024-02-13T03:11:43.439 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-02-13T03:11:43.456 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.6_amd64.deb ... 2024-02-13T03:11:43.497 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.6) ... 2024-02-13T03:11:43.755 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-html5lib. 2024-02-13T03:11:43.773 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2024-02-13T03:11:43.813 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2024-02-13T03:11:43.909 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyparsing. 2024-02-13T03:11:43.927 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2024-02-13T03:11:43.967 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2024-02-13T03:11:44.149 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-02-13T03:11:44.168 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.6_amd64.deb ... 2024-02-13T03:11:44.207 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.6) ... 2024-02-13T03:11:44.286 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-tz. 2024-02-13T03:11:44.304 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2024-02-13T03:11:44.344 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-tz (2018.3-2) ... 2024-02-13T03:11:44.622 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-matplotlib. 2024-02-13T03:11:44.640 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2024-02-13T03:11:44.644 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyparsing. 2024-02-13T03:11:44.661 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2024-02-13T03:11:44.672 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2024-02-13T03:11:44.703 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2024-02-13T03:11:45.005 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-tz. 2024-02-13T03:11:45.022 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2024-02-13T03:11:45.063 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-tz (2018.3-2) ... 2024-02-13T03:11:45.332 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-matplotlib. 2024-02-13T03:11:45.352 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2024-02-13T03:11:45.352 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-olefile. 2024-02-13T03:11:45.369 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2024-02-13T03:11:45.382 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2024-02-13T03:11:45.419 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-olefile (0.45.1-1) ... 2024-02-13T03:11:45.730 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pastescript. 2024-02-13T03:11:45.748 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2024-02-13T03:11:45.788 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2024-02-13T03:11:46.037 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-olefile. 2024-02-13T03:11:46.054 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2024-02-13T03:11:46.083 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pil:amd64. 2024-02-13T03:11:46.095 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-olefile (0.45.1-1) ... 2024-02-13T03:11:46.102 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.8_amd64.deb ... 2024-02-13T03:11:46.132 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.8) ... 2024-02-13T03:11:46.406 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pastescript. 2024-02-13T03:11:46.422 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2024-02-13T03:11:46.460 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pluggy. 2024-02-13T03:11:46.464 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2024-02-13T03:11:46.481 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2024-02-13T03:11:46.546 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2024-02-13T03:11:46.793 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pil:amd64. 2024-02-13T03:11:46.812 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.8_amd64.deb ... 2024-02-13T03:11:46.848 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-py. 2024-02-13T03:11:46.851 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.8) ... 2024-02-13T03:11:46.865 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../106-python3-py_1.5.2-1ubuntu0.1_all.deb ... 2024-02-13T03:11:46.906 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-py (1.5.2-1ubuntu0.1) ... 2024-02-13T03:11:47.218 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyinotify. 2024-02-13T03:11:47.220 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pluggy. 2024-02-13T03:11:47.238 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2024-02-13T03:11:47.238 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2024-02-13T03:11:47.276 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2024-02-13T03:11:47.279 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2024-02-13T03:11:47.562 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pytest. 2024-02-13T03:11:47.581 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-py. 2024-02-13T03:11:47.582 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2024-02-13T03:11:47.600 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../106-python3-py_1.5.2-1ubuntu0.1_all.deb ... 2024-02-13T03:11:47.620 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pytest (3.3.2-2) ... 2024-02-13T03:11:47.656 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-py (1.5.2-1ubuntu0.1) ... 2024-02-13T03:11:47.951 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyinotify. 2024-02-13T03:11:47.967 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-simplejson. 2024-02-13T03:11:47.971 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2024-02-13T03:11:47.981 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2024-02-13T03:11:48.009 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2024-02-13T03:11:48.023 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2024-02-13T03:11:48.312 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pytest. 2024-02-13T03:11:48.328 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2024-02-13T03:11:48.370 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pytest (3.3.2-2) ... 2024-02-13T03:11:48.376 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-tk:amd64. 2024-02-13T03:11:48.393 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2024-02-13T03:11:48.434 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2024-02-13T03:11:48.714 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-simplejson. 2024-02-13T03:11:48.733 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2024-02-13T03:11:48.772 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2024-02-13T03:11:48.796 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package radosgw. 2024-02-13T03:11:48.813 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../111-radosgw_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:48.845 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking radosgw (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:49.100 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-tk:amd64. 2024-02-13T03:11:49.118 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2024-02-13T03:11:49.158 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2024-02-13T03:11:49.629 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package radosgw. 2024-02-13T03:11:49.646 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../111-radosgw_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:49.687 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking radosgw (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:49.882 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package rbd-fuse. 2024-02-13T03:11:49.900 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../112-rbd-fuse_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:49.940 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking rbd-fuse (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:50.249 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-logutils (0.3.3-5) ... 2024-02-13T03:11:50.653 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.2) ... 2024-02-13T03:11:50.740 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package rbd-fuse. 2024-02-13T03:11:50.757 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../112-rbd-fuse_16.2.14-521-g0e714d9a-1bionic_amd64.deb ... 2024-02-13T03:11:50.798 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking rbd-fuse (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:51.013 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2024-02-13T03:11:51.103 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-logutils (0.3.3-5) ... 2024-02-13T03:11:51.139 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2024-02-13T03:11:51.437 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.2) ... 2024-02-13T03:11:51.759 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.8) ... 2024-02-13T03:11:51.796 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2024-02-13T03:11:51.897 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2024-02-13T03:11:52.110 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2024-02-13T03:11:52.378 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-webencodings (0.5-2) ... 2024-02-13T03:11:52.508 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.8) ... 2024-02-13T03:11:52.679 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-attr (17.4.0-2) ... 2024-02-13T03:11:52.888 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2024-02-13T03:11:52.893 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2024-02-13T03:11:53.014 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-waitress (1.0.1-1) ... 2024-02-13T03:11:53.177 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-webencodings (0.5-2) ... 2024-02-13T03:11:53.347 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2024-02-13T03:11:53.480 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-attr (17.4.0-2) ... 2024-02-13T03:11:53.659 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-olefile (0.45.1-1) ... 2024-02-13T03:11:53.705 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2024-02-13T03:11:53.805 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-waitress (1.0.1-1) ... 2024-02-13T03:11:53.935 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-ceph-argparse (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:54.139 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2024-02-13T03:11:54.202 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-tempita (0.5.2-2) ... 2024-02-13T03:11:54.503 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2024-02-13T03:11:54.516 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-olefile (0.45.1-1) ... 2024-02-13T03:11:54.638 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2024-02-13T03:11:54.818 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-ceph-argparse (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:55.110 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-tempita (0.5.2-2) ... 2024-02-13T03:11:55.428 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2024-02-13T03:11:55.566 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2024-02-13T03:11:55.570 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2024-02-13T03:11:55.867 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2024-02-13T03:11:56.135 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-py (1.5.2-1ubuntu0.1) ... 2024-02-13T03:11:56.344 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-simplejson (3.13.2-1) ... 2024-02-13T03:11:56.489 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2024-02-13T03:11:56.679 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2024-02-13T03:11:56.815 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2024-02-13T03:11:56.827 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-pluggy (0.6.0-1) ... 2024-02-13T03:11:57.022 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2024-02-13T03:11:57.108 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-py (1.5.2-1ubuntu0.1) ... 2024-02-13T03:11:57.315 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2024-02-13T03:11:57.360 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-simplejson (3.13.2-1) ... 2024-02-13T03:11:57.641 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cycler (0.10.0-1) ... 2024-02-13T03:11:57.702 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2024-02-13T03:11:57.828 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-pluggy (0.6.0-1) ... 2024-02-13T03:11:57.925 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2024-02-13T03:11:58.029 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2024-02-13T03:11:58.285 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pkg-resources (39.0.1-2ubuntu0.1) ... 2024-02-13T03:11:58.322 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2024-02-13T03:11:58.603 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-bs4 (4.6.0-1) ... 2024-02-13T03:11:58.623 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cycler (0.10.0-1) ... 2024-02-13T03:11:58.907 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2024-02-13T03:11:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pluggy (0.6.0-1) ... 2024-02-13T03:11:59.216 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up cephadm (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:11:59.422 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pkg-resources (39.0.1-2ubuntu0.1) ... 2024-02-13T03:11:59.798 INFO:teuthology.orchestra.run.smithi161.stdout:Adding system user cephadm....done 2024-02-13T03:11:59.918 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-bs4 (4.6.0-1) ... 2024-02-13T03:11:59.928 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2024-02-13T03:12:00.213 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.3) ... 2024-02-13T03:12:00.252 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pluggy (0.6.0-1) ... 2024-02-13T03:12:00.355 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up nvme-cli (1.5-1ubuntu1.2) ... 2024-02-13T03:12:00.498 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2024-02-13T03:12:00.562 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up cephadm (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:00.949 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-prettytable (0.7.2-3) ... 2024-02-13T03:12:01.163 INFO:teuthology.orchestra.run.smithi183.stdout:Adding system user cephadm....done 2024-02-13T03:12:01.242 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2024-02-13T03:12:01.326 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2024-02-13T03:12:01.351 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2024-02-13T03:12:01.610 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.3) ... 2024-02-13T03:12:01.669 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-ceph-common (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:01.736 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up nvme-cli (1.5-1ubuntu1.2) ... 2024-02-13T03:12:01.871 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2024-02-13T03:12:01.987 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-monotonic (1.1-2) ... 2024-02-13T03:12:02.222 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-prettytable (0.7.2-3) ... 2024-02-13T03:12:02.288 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2024-02-13T03:12:02.414 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2024-02-13T03:12:02.522 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2024-02-13T03:12:02.632 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2024-02-13T03:12:02.749 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-html5lib (0.999999999-1) ... 2024-02-13T03:12:02.966 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-ceph-common (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:03.100 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-repoze.lru (0.7-1) ... 2024-02-13T03:12:03.293 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-monotonic (1.1-2) ... 2024-02-13T03:12:03.393 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2024-02-13T03:12:03.602 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2024-02-13T03:12:03.669 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2024-02-13T03:12:03.728 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2024-02-13T03:12:04.011 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-attr (17.4.0-2) ... 2024-02-13T03:12:04.079 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-html5lib (0.999999999-1) ... 2024-02-13T03:12:04.313 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-funcsigs (1.0.2-4) ... 2024-02-13T03:12:04.439 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-repoze.lru (0.7-1) ... 2024-02-13T03:12:04.530 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libjaeger (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:04.656 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2024-02-13T03:12:04.731 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2024-02-13T03:12:05.024 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2024-02-13T03:12:05.066 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2024-02-13T03:12:05.156 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-02-13T03:12:05.366 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-attr (17.4.0-2) ... 2024-02-13T03:12:05.443 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liboath0 (2.6.1-1) ... 2024-02-13T03:12:05.586 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2024-02-13T03:12:05.676 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-funcsigs (1.0.2-4) ... 2024-02-13T03:12:05.687 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2024-02-13T03:12:05.869 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libjaeger (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:05.946 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2024-02-13T03:12:05.994 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2024-02-13T03:12:06.072 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-setuptools (39.0.1-2ubuntu0.1) ... 2024-02-13T03:12:06.404 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2024-02-13T03:12:06.465 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2024-02-13T03:12:06.485 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-02-13T03:12:06.591 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1ubuntu0.18.04.1) ... 2024-02-13T03:12:06.717 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-decorator (4.1.2-1) ... 2024-02-13T03:12:06.747 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liboath0 (2.6.1-1) ... 2024-02-13T03:12:06.907 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2024-02-13T03:12:06.985 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-tz (2018.3-2) ... 2024-02-13T03:12:07.033 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2024-02-13T03:12:07.244 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2024-02-13T03:12:07.276 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2024-02-13T03:12:07.402 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-setuptools (39.0.1-2ubuntu0.1) ... 2024-02-13T03:12:07.537 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jwt (1.5.3+ds1-1ubuntu0.1) ... 2024-02-13T03:12:07.803 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2024-02-13T03:12:07.839 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-joblib (0.11-1) ... 2024-02-13T03:12:07.929 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1ubuntu0.18.04.1) ... 2024-02-13T03:12:08.055 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-decorator (4.1.2-1) ... 2024-02-13T03:12:08.215 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-routes (2.4.1-1) ... 2024-02-13T03:12:08.348 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-tz (2018.3-2) ... 2024-02-13T03:12:08.524 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2024-02-13T03:12:08.624 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2024-02-13T03:12:08.825 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-dateutil (2.6.1-1) ... 2024-02-13T03:12:08.908 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jwt (1.5.3+ds1-1ubuntu0.1) ... 2024-02-13T03:12:09.168 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2024-02-13T03:12:09.209 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-joblib (0.11-1) ... 2024-02-13T03:12:09.503 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-fasteners (0.12.0-3) ... 2024-02-13T03:12:09.602 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-routes (2.4.1-1) ... 2024-02-13T03:12:09.796 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up jq (1.5+dfsg-2) ... 2024-02-13T03:12:09.911 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2024-02-13T03:12:09.922 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2024-02-13T03:12:10.212 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-dateutil (2.6.1-1) ... 2024-02-13T03:12:10.547 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2024-02-13T03:12:10.898 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-fasteners (0.12.0-3) ... 2024-02-13T03:12:11.085 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-py (1.5.2-1ubuntu0.1) ... 2024-02-13T03:12:11.183 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up jq (1.5+dfsg-2) ... 2024-02-13T03:12:11.334 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2024-02-13T03:12:11.395 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.6) ... 2024-02-13T03:12:11.704 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-mako (1.0.7+ds1-1ubuntu0.2) ... 2024-02-13T03:12:12.022 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2024-02-13T03:12:12.148 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pytest (3.3.2-2) ... 2024-02-13T03:12:12.513 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-py (1.5.2-1ubuntu0.1) ... 2024-02-13T03:12:12.516 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-pytest (3.3.2-2) ... 2024-02-13T03:12:12.776 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2024-02-13T03:12:12.840 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.6) ... 2024-02-13T03:12:12.893 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up xmlstarlet (1.6.1-2) ... 2024-02-13T03:12:12.994 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librados2 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:13.095 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rsa (3.4.2-1) ... 2024-02-13T03:12:13.132 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-mako (1.0.7+ds1-1ubuntu0.2) ... 2024-02-13T03:12:13.397 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libcephfs2 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:13.461 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2024-02-13T03:12:13.523 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libcephfs-dev (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:13.612 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pytest (3.3.2-2) ... 2024-02-13T03:12:13.648 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rados (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:13.774 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2024-02-13T03:12:13.979 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-pytest (3.3.2-2) ... 2024-02-13T03:12:14.092 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2024-02-13T03:12:14.231 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2024-02-13T03:12:14.332 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up xmlstarlet (1.6.1-2) ... 2024-02-13T03:12:14.393 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2024-02-13T03:12:14.458 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librados2 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:14.583 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rsa (3.4.2-1) ... 2024-02-13T03:12:14.763 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2024-02-13T03:12:14.893 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libcephfs2 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:15.019 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libcephfs-dev (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:15.145 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rados (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:15.271 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2024-02-13T03:12:15.589 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2024-02-13T03:12:15.608 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-sklearn (0.19.1-3) ... 2024-02-13T03:12:15.890 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2024-02-13T03:12:16.285 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2024-02-13T03:12:16.471 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2024-02-13T03:12:16.765 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libradosstriper1 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:16.890 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cephfs (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:17.147 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-sklearn (0.19.1-3) ... 2024-02-13T03:12:17.191 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libsqlite3-mod-ceph (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:17.317 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librgw2 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:17.468 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-fuse (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:17.686 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-02-13T03:12:17.686 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-02-13T03:12:17.960 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librbd1 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:17.984 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2024-02-13T03:12:18.061 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pecan (1.2.1-2) ... 2024-02-13T03:12:18.143 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2024-02-13T03:12:18.174 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2024-02-13T03:12:18.286 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libradosstriper1 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:18.412 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cephfs (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:18.481 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2024-02-13T03:12:18.704 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libsqlite3-mod-ceph (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:18.822 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librgw2 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:18.931 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-fuse (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:19.137 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-02-13T03:12:19.138 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-02-13T03:12:19.158 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-modules-core (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:19.285 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pastescript (2.0.2-2) ... 2024-02-13T03:12:19.408 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librbd1 (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:19.559 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pecan (1.2.1-2) ... 2024-02-13T03:12:19.594 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rgw (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:19.648 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2024-02-13T03:12:19.673 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2024-02-13T03:12:19.695 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up rbd-fuse (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:19.796 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rbd (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:19.922 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-common (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:19.978 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2024-02-13T03:12:20.465 INFO:teuthology.orchestra.run.smithi161.stdout:Adding group ceph....done 2024-02-13T03:12:20.656 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-modules-core (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:20.765 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pastescript (2.0.2-2) ... 2024-02-13T03:12:20.829 INFO:teuthology.orchestra.run.smithi161.stdout:Adding system user ceph....done 2024-02-13T03:12:21.035 INFO:teuthology.orchestra.run.smithi161.stdout:Setting system user ceph properties....done 2024-02-13T03:12:21.054 INFO:teuthology.orchestra.run.smithi161.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-02-13T03:12:21.074 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rgw (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:21.189 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-02-13T03:12:21.200 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up rbd-fuse (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:21.327 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rbd (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:21.423 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-02-13T03:12:21.478 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-common (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:21.741 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up radosgw (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:21.978 INFO:teuthology.orchestra.run.smithi183.stdout:Adding group ceph....done 2024-02-13T03:12:22.003 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-02-13T03:12:22.003 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-02-13T03:12:22.365 INFO:teuthology.orchestra.run.smithi183.stdout:Adding system user ceph....done 2024-02-13T03:12:22.434 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-test (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:22.557 INFO:teuthology.orchestra.run.smithi183.stdout:Setting system user ceph properties....done 2024-02-13T03:12:22.576 INFO:teuthology.orchestra.run.smithi183.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-02-13T03:12:22.576 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-base (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:22.714 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-02-13T03:12:22.930 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-02-13T03:12:22.943 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-02-13T03:12:23.237 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:23.238 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up radosgw (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:23.427 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-02-13T03:12:23.428 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-02-13T03:12:23.507 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-02-13T03:12:23.507 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-02-13T03:12:23.730 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-diskprediction-local (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:23.873 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-osd (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:23.906 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-test (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:24.015 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-base (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:24.288 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-02-13T03:12:24.289 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-02-13T03:12:24.384 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-02-13T03:12:24.574 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-cephadm (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:24.667 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:24.717 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mds (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:24.891 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-02-13T03:12:24.892 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-02-13T03:12:24.932 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-02-13T03:12:24.932 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-02-13T03:12:25.235 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mon (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:25.345 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-diskprediction-local (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:25.451 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-02-13T03:12:25.451 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-02-13T03:12:25.513 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-osd (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:25.728 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-k8sevents (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:25.829 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-dashboard (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:25.928 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-02-13T03:12:25.928 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-02-13T03:12:25.947 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:26.098 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for systemd (237-3ubuntu10.56) ... 2024-02-13T03:12:26.214 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-cephadm (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:26.340 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mds (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:26.494 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2024-02-13T03:12:26.551 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-02-13T03:12:26.551 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-02-13T03:12:26.833 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mon (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:27.046 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-02-13T03:12:27.172 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-02-13T03:12:27.334 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-k8sevents (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:27.485 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-dashboard (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:27.628 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph (16.2.14-521-g0e714d9a-1bionic) ... 2024-02-13T03:12:27.755 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for systemd (237-3ubuntu10.56) ... 2024-02-13T03:12:27.939 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2024-02-13T03:12:28.287 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2024-02-13T03:12:28.363 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.6) ... 2024-02-13T03:12:28.846 INFO:teuthology.orchestra.run.smithi161.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-02-13T03:12:28.851 DEBUG:teuthology.parallel:result is None 2024-02-13T03:12:29.774 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2024-02-13T03:12:29.858 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.6) ... 2024-02-13T03:12:30.342 INFO:teuthology.orchestra.run.smithi183.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-02-13T03:12:30.346 DEBUG:teuthology.parallel:result is None 2024-02-13T03:12:30.347 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-13T03:12:30.569 DEBUG:teuthology.orchestra.run.smithi161:> dpkg-query -W -f '${Version}' ceph 2024-02-13T03:12:30.594 INFO:teuthology.orchestra.run.smithi161.stdout:16.2.14-521-g0e714d9a-1bionic 2024-02-13T03:12:30.595 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-521-g0e714d9a-1bionic 2024-02-13T03:12:30.595 INFO:teuthology.task.install:The correct ceph version 16.2.14-521-g0e714d9a-1bionic is installed. 2024-02-13T03:12:30.597 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-13T03:12:30.737 DEBUG:teuthology.orchestra.run.smithi183:> dpkg-query -W -f '${Version}' ceph 2024-02-13T03:12:30.760 INFO:teuthology.orchestra.run.smithi183.stdout:16.2.14-521-g0e714d9a-1bionic 2024-02-13T03:12:30.761 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-521-g0e714d9a-1bionic 2024-02-13T03:12:30.761 INFO:teuthology.task.install:The correct ceph version 16.2.14-521-g0e714d9a-1bionic is installed. 2024-02-13T03:12:30.762 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-13T03:12:30.763 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:12:30.763 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-13T03:12:30.794 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-02-13T03:12:30.795 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-13T03:12:30.817 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-13T03:12:30.817 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:12:30.817 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-02-13T03:12:30.851 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-13T03:12:30.907 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-02-13T03:12:30.907 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-02-13T03:12:30.922 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-13T03:12:30.983 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-13T03:12:30.983 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:12:30.984 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-13T03:12:30.998 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-13T03:12:31.058 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-02-13T03:12:31.058 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-13T03:12:31.073 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-13T03:12:31.131 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-13T03:12:31.131 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:12:31.132 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-02-13T03:12:31.146 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-13T03:12:31.202 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-02-13T03:12:31.203 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-02-13T03:12:31.218 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-13T03:12:31.275 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-13T03:12:31.349 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(MON_DOWN\\)', 'missing hit_sets', 'do not have an application enabled', 'application not enabled on pool', 'mons down'], 'mon_bind_msgr2': False, 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1', 'cephadm_mode': 'cephadm-package'} 2024-02-13T03:12:31.350 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-13T03:12:31.350 INFO:tasks.cephadm:Cluster fsid is b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:12:31.350 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-13T03:12:31.350 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.161:6789]', 'mon.c': '[v1:172.21.15.161:6790]', 'mon.b': '[v1:172.21.15.183:6789]'} 2024-02-13T03:12:31.350 INFO:tasks.cephadm:First mon is mon.a on smithi161 2024-02-13T03:12:31.350 INFO:tasks.cephadm:First mgr is y 2024-02-13T03:12:31.351 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-13T03:12:31.351 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-02-13T03:12:31.367 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-02-13T03:12:31.394 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-02-13T03:12:31.423 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-02-13T03:12:31.440 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-02-13T03:12:31.484 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-02-13T03:12:31.499 INFO:tasks.cephadm:Writing seed config... 2024-02-13T03:12:31.500 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-13T03:12:31.500 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-13T03:12:31.500 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-13T03:12:31.500 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-02-13T03:12:31.501 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-02-13T03:12:31.501 INFO:tasks.cephadm: override: [global] ms type = async 2024-02-13T03:12:31.501 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-13T03:12:31.501 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-13T03:12:31.501 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-13T03:12:31.501 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-02-13T03:12:31.501 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-13T03:12:31.501 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-13T03:12:31.501 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-13T03:12:31.502 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:12:31.502 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-13T03:12:31.534 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b9b3ef98-ca1d-11ee-95b9-87774f69a715 mon election default strategy = 1 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-02-13T03:12:31.534 DEBUG:teuthology.orchestra.run.smithi161:mon.a> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mon.a.service 2024-02-13T03:12:31.578 DEBUG:teuthology.orchestra.run.smithi161:mgr.y> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mgr.y.service 2024-02-13T03:12:31.621 INFO:tasks.cephadm:Bootstrapping... 2024-02-13T03:12:31.622 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -v bootstrap --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.161:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-13T03:12:31.950 INFO:journalctl@ceph.mgr.y.smithi161.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:12:32.024 INFO:journalctl@ceph.mon.a.smithi161.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:12:32.080 INFO:teuthology.orchestra.run.smithi161.stderr:-------------------------------------------------------------------------------- 2024-02-13T03:12:32.080 INFO:teuthology.orchestra.run.smithi161.stderr:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1', '-v', 'bootstrap', '--fsid', 'b9b3ef98-ca1d-11ee-95b9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.161:6789]', '--skip-admin-label'] 2024-02-13T03:12:32.080 INFO:teuthology.orchestra.run.smithi161.stderr:Verifying podman|docker is present... 2024-02-13T03:12:32.081 INFO:teuthology.orchestra.run.smithi161.stderr:Verifying lvm2 is present... 2024-02-13T03:12:32.081 INFO:teuthology.orchestra.run.smithi161.stderr:Verifying time synchronization is in place... 2024-02-13T03:12:32.088 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-13T03:12:32.095 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: inactive 2024-02-13T03:12:32.102 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Failed to get unit file state for chronyd.service: No such file or directory 2024-02-13T03:12:32.109 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: inactive 2024-02-13T03:12:32.119 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: enabled 2024-02-13T03:12:32.127 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: inactive 2024-02-13T03:12:32.136 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Failed to get unit file state for ntpd.service: No such file or directory 2024-02-13T03:12:32.143 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: inactive 2024-02-13T03:12:32.153 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: enabled 2024-02-13T03:12:32.162 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: active 2024-02-13T03:12:32.164 INFO:teuthology.orchestra.run.smithi161.stderr:Unit ntp.service is enabled and running 2024-02-13T03:12:32.164 INFO:teuthology.orchestra.run.smithi161.stderr:Repeating the final host check... 2024-02-13T03:12:32.164 INFO:teuthology.orchestra.run.smithi161.stderr:docker (/usr/bin/docker) is present 2024-02-13T03:12:32.164 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl is present 2024-02-13T03:12:32.164 INFO:teuthology.orchestra.run.smithi161.stderr:lvcreate is present 2024-02-13T03:12:32.171 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-13T03:12:32.177 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: inactive 2024-02-13T03:12:32.186 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Failed to get unit file state for chronyd.service: No such file or directory 2024-02-13T03:12:32.194 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: inactive 2024-02-13T03:12:32.203 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: enabled 2024-02-13T03:12:32.212 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: inactive 2024-02-13T03:12:32.220 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Failed to get unit file state for ntpd.service: No such file or directory 2024-02-13T03:12:32.227 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: inactive 2024-02-13T03:12:32.234 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: enabled 2024-02-13T03:12:32.242 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: active 2024-02-13T03:12:32.243 INFO:teuthology.orchestra.run.smithi161.stderr:Unit ntp.service is enabled and running 2024-02-13T03:12:32.243 INFO:teuthology.orchestra.run.smithi161.stderr:Host looks OK 2024-02-13T03:12:32.243 INFO:teuthology.orchestra.run.smithi161.stderr:Cluster fsid: b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:12:32.243 INFO:teuthology.orchestra.run.smithi161.stderr:Acquiring lock 140434438073704 on /run/cephadm/b9b3ef98-ca1d-11ee-95b9-87774f69a715.lock 2024-02-13T03:12:32.244 INFO:teuthology.orchestra.run.smithi161.stderr:Lock 140434438073704 acquired on /run/cephadm/b9b3ef98-ca1d-11ee-95b9-87774f69a715.lock 2024-02-13T03:12:32.244 INFO:teuthology.orchestra.run.smithi161.stderr:Verifying IP 172.21.15.161 port 6789 ... 2024-02-13T03:12:32.244 INFO:teuthology.orchestra.run.smithi161.stderr:Base mon IP(s) is [172.21.15.161:6789], mon addrv is [v1:172.21.15.161:6789] 2024-02-13T03:12:32.249 INFO:teuthology.orchestra.run.smithi161.stderr:/sbin/ip: default via 172.21.15.254 dev enp3s0f1 2024-02-13T03:12:32.249 INFO:teuthology.orchestra.run.smithi161.stderr:/sbin/ip: default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.161 metric 100 2024-02-13T03:12:32.249 INFO:teuthology.orchestra.run.smithi161.stderr:/sbin/ip: 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 2024-02-13T03:12:32.249 INFO:teuthology.orchestra.run.smithi161.stderr:/sbin/ip: 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.161 2024-02-13T03:12:32.249 INFO:teuthology.orchestra.run.smithi161.stderr:/sbin/ip: 172.21.15.254 dev enp3s0f1 proto dhcp scope link src 172.21.15.161 metric 100 2024-02-13T03:12:32.253 INFO:teuthology.orchestra.run.smithi161.stderr:/sbin/ip: fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-13T03:12:32.253 INFO:teuthology.orchestra.run.smithi161.stderr:/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-02-13T03:12:32.257 INFO:teuthology.orchestra.run.smithi161.stderr:/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-13T03:12:32.257 INFO:teuthology.orchestra.run.smithi161.stderr:/sbin/ip: inet6 ::1/128 scope host 2024-02-13T03:12:32.257 INFO:teuthology.orchestra.run.smithi161.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-13T03:12:32.258 INFO:teuthology.orchestra.run.smithi161.stderr:/sbin/ip: 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-13T03:12:32.258 INFO:teuthology.orchestra.run.smithi161.stderr:/sbin/ip: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-02-13T03:12:32.258 INFO:teuthology.orchestra.run.smithi161.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-13T03:12:32.259 INFO:teuthology.orchestra.run.smithi161.stderr:Mon IP `172.21.15.161` is in CIDR network `172.21.0.0/20` 2024-02-13T03:12:32.259 INFO:teuthology.orchestra.run.smithi161.stderr:Mon IP `172.21.15.161` is in CIDR network `172.21.15.254/32` 2024-02-13T03:12:32.259 INFO:teuthology.orchestra.run.smithi161.stderr:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.15.254/32'] 2024-02-13T03:12:32.260 INFO:teuthology.orchestra.run.smithi161.stderr:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-13T03:12:32.261 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1... 2024-02-13T03:12:32.633 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/docker: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1: Pulling from ceph-ci/ceph 2024-02-13T03:12:32.641 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/docker: 58df4e03ba06: Pulling fs layer 2024-02-13T03:12:32.641 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/docker: 955bfa9dba4a: Pulling fs layer 2024-02-13T03:12:33.401 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/docker: 58df4e03ba06: Verifying Checksum 2024-02-13T03:12:33.402 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/docker: 58df4e03ba06: Download complete 2024-02-13T03:12:35.288 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/docker: 955bfa9dba4a: Verifying Checksum 2024-02-13T03:12:35.289 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/docker: 955bfa9dba4a: Download complete 2024-02-13T03:12:35.829 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/docker: 58df4e03ba06: Pull complete 2024-02-13T03:12:45.618 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/docker: 955bfa9dba4a: Pull complete 2024-02-13T03:12:45.730 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/docker: Digest: sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc 2024-02-13T03:12:45.769 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/docker: Status: Downloaded newer image for quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-13T03:12:45.788 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/docker: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-13T03:12:55.565 INFO:teuthology.orchestra.run.smithi161.stderr:ceph: ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable) 2024-02-13T03:12:56.107 INFO:teuthology.orchestra.run.smithi161.stderr:Ceph version: ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable) 2024-02-13T03:12:56.107 INFO:teuthology.orchestra.run.smithi161.stderr:Extracting ceph user uid/gid from container image... 2024-02-13T03:12:56.898 INFO:teuthology.orchestra.run.smithi161.stderr:stat: 167 167 2024-02-13T03:12:57.370 INFO:teuthology.orchestra.run.smithi161.stderr:Creating initial keys... 2024-02-13T03:12:58.150 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-authtool: AQA63spl8bT2CBAAOH8ooqyVotLZdYmRgOPmew== 2024-02-13T03:12:59.447 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-authtool: AQA73spldYehGhAA5EhJ7hbkuKmIgEreoG78Kw== 2024-02-13T03:13:00.678 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-authtool: AQA83splu7lqKBAAOps3cwDDuyQaQop6bJu17w== 2024-02-13T03:13:01.127 INFO:teuthology.orchestra.run.smithi161.stderr:Creating initial monmap... 2024-02-13T03:13:01.971 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-13T03:13:01.971 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:13:01.971 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-13T03:13:02.497 INFO:teuthology.orchestra.run.smithi161.stderr:monmaptool for a [v1:172.21.15.161:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-13T03:13:02.497 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/monmaptool: set fsid to b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:13:02.498 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-13T03:13:02.498 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-02-13T03:13:02.498 INFO:teuthology.orchestra.run.smithi161.stderr:Creating mon... 2024-02-13T03:13:03.308 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.302+0000 7f2568ce3880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-13T03:13:03.308 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.302+0000 7f2568ce3880 1 imported monmap: 2024-02-13T03:13:03.309 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: epoch 0 2024-02-13T03:13:03.309 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:13:03.330 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: last_changed 2024-02-13T03:13:01.970936+0000 2024-02-13T03:13:03.330 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: created 2024-02-13T03:13:01.970936+0000 2024-02-13T03:13:03.330 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: min_mon_release 0 (unknown) 2024-02-13T03:13:03.330 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: election_strategy: 1 2024-02-13T03:13:03.330 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: 0: v1:172.21.15.161:6789/0 mon.a 2024-02-13T03:13:03.330 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: 2024-02-13T03:13:03.330 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.302+0000 7f2568ce3880 0 /usr/bin/ceph-mon: set fsid to b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:13:03.330 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: RocksDB version: 6.8.1 2024-02-13T03:13:03.330 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: 2024-02-13T03:13:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-13T03:13:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Compile date Feb 8 2024 2024-02-13T03:13:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: DB SUMMARY 2024-02-13T03:13:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: 2024-02-13T03:13:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2024-02-13T03:13:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: 2024-02-13T03:13:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2024-02-13T03:13:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: 2024-02-13T03:13:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.error_if_exists: 0 2024-02-13T03:13:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.create_if_missing: 1 2024-02-13T03:13:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.paranoid_checks: 1 2024-02-13T03:13:03.331 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.env: 0x55ee1589c080 2024-02-13T03:13:03.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.fs: Posix File System 2024-02-13T03:13:03.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.info_log: 0x55ee17bdfa80 2024-02-13T03:13:03.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-13T03:13:03.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.statistics: (nil) 2024-02-13T03:13:03.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.use_fsync: 0 2024-02-13T03:13:03.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_log_file_size: 0 2024-02-13T03:13:03.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-13T03:13:03.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-13T03:13:03.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-13T03:13:03.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-13T03:13:03.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.allow_fallocate: 1 2024-02-13T03:13:03.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-13T03:13:03.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-13T03:13:03.333 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.use_direct_reads: 0 2024-02-13T03:13:03.333 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-13T03:13:03.333 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-13T03:13:03.333 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.db_log_dir: 2024-02-13T03:13:03.333 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-02-13T03:13:03.333 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-13T03:13:03.333 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_subcompactions: 1 2024-02-13T03:13:03.333 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_background_flushes: -1 2024-02-13T03:13:03.333 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-13T03:13:03.333 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-13T03:13:03.333 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-13T03:13:03.333 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-13T03:13:03.333 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.write_buffer_manager: 0x55ee17be5980 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.unordered_write: 0 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-13T03:13:03.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-13T03:13:03.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-13T03:13:03.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-13T03:13:03.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.row_cache: None 2024-02-13T03:13:03.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.wal_filter: None 2024-02-13T03:13:03.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-13T03:13:03.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-13T03:13:03.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.preserve_deletes: 0 2024-02-13T03:13:03.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.two_write_queues: 0 2024-02-13T03:13:03.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-13T03:13:03.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.atomic_flush: 0 2024-02-13T03:13:03.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-13T03:13:03.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-13T03:13:03.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-13T03:13:03.336 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.log_readahead_size: 0 2024-02-13T03:13:03.336 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-13T03:13:03.336 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_background_jobs: 2 2024-02-13T03:13:03.336 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_background_compactions: -1 2024-02-13T03:13:03.336 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-13T03:13:03.336 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-13T03:13:03.336 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-13T03:13:03.336 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-13T03:13:03.336 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-13T03:13:03.336 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-13T03:13:03.336 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-13T03:13:03.336 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-13T03:13:03.336 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_open_files: -1 2024-02-13T03:13:03.337 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-13T03:13:03.337 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-13T03:13:03.337 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-13T03:13:03.337 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-13T03:13:03.337 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Compression algorithms supported: 2024-02-13T03:13:03.337 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-13T03:13:03.337 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: kZSTD supported: 0 2024-02-13T03:13:03.337 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: kXpressCompression supported: 0 2024-02-13T03:13:03.337 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-13T03:13:03.337 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: kLZ4Compression supported: 1 2024-02-13T03:13:03.337 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: kBZip2Compression supported: 0 2024-02-13T03:13:03.337 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: kZlibCompression supported: 1 2024-02-13T03:13:03.337 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: kSnappyCompression supported: 1 2024-02-13T03:13:03.338 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-13T03:13:03.338 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: [db_impl/db_impl_open.cc:273] Creating manifest 1 2024-02-13T03:13:03.338 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: 2024-02-13T03:13:03.338 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2024-02-13T03:13:03.338 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: 2024-02-13T03:13:03.338 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-13T03:13:03.338 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: 2024-02-13T03:13:03.338 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-13T03:13:03.338 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.merge_operator: 2024-02-13T03:13:03.338 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compaction_filter: None 2024-02-13T03:13:03.338 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-13T03:13:03.338 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-13T03:13:03.338 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-13T03:13:03.339 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ee17af3d30) 2024-02-13T03:13:03.339 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks: 1 2024-02-13T03:13:03.339 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-13T03:13:03.339 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-13T03:13:03.339 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: pin_top_level_index_and_filter: 1 2024-02-13T03:13:03.339 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: index_type: 0 2024-02-13T03:13:03.339 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: data_block_index_type: 0 2024-02-13T03:13:03.339 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: index_shortening: 1 2024-02-13T03:13:03.339 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: data_block_hash_table_util_ratio: 0.750000 2024-02-13T03:13:03.339 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: hash_index_allow_collision: 1 2024-02-13T03:13:03.339 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: checksum: 1 2024-02-13T03:13:03.339 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: no_block_cache: 0 2024-02-13T03:13:03.339 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: block_cache: 0x55ee17b2ad10 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: block_cache_name: BinnedLRUCache 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: block_cache_options: 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: capacity : 536870912 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: num_shard_bits : 4 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: strict_capacity_limit : 0 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: high_pri_pool_ratio: 0.000 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: block_cache_compressed: (nil) 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: persistent_cache: (nil) 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: block_size: 4096 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: block_size_deviation: 10 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: block_restart_interval: 16 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: index_block_restart_interval: 1 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: metadata_block_size: 4096 2024-02-13T03:13:03.340 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: partition_filters: 0 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: use_delta_encoding: 1 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: whole_key_filtering: 1 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: verify_compression: 0 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: read_amp_bytes_per_bit: 0 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: format_version: 2 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: enable_index_compression: 1 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: block_align: 0 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compression: NoCompression 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-13T03:13:03.341 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.num_levels: 7 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-13T03:13:03.342 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-13T03:13:03.343 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-13T03:13:03.344 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-13T03:13:03.345 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-13T03:13:03.345 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-13T03:13:03.345 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-13T03:13:03.345 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-13T03:13:03.345 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-13T03:13:03.345 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-13T03:13:03.345 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-13T03:13:03.345 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.table_properties_collectors: 2024-02-13T03:13:03.345 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.inplace_update_support: 0 2024-02-13T03:13:03.345 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-13T03:13:03.345 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-13T03:13:03.345 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-13T03:13:03.345 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-13T03:13:03.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.bloom_locality: 0 2024-02-13T03:13:03.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.max_successive_merges: 0 2024-02-13T03:13:03.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-13T03:13:03.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-13T03:13:03.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-13T03:13:03.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-13T03:13:03.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.ttl: 2592000 2024-02-13T03:13:03.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.306+0000 7f2568ce3880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-13T03:13:03.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.310+0000 7f2568ce3880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-13T03:13:03.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: 2024-02-13T03:13:03.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.310+0000 7f2568ce3880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-13T03:13:03.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: 2024-02-13T03:13:03.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.310+0000 7f2568ce3880 4 rocksdb: DB pointer 0x55ee17bf3800 2024-02-13T03:13:03.347 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.314+0000 7f2568ce3880 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work 2024-02-13T03:13:03.347 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.318+0000 7f2568ce3880 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete 2024-02-13T03:13:03.347 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph-mon: debug 2024-02-13T03:13:03.318+0000 7f2568ce3880 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2024-02-13T03:13:03.828 INFO:teuthology.orchestra.run.smithi161.stderr:create mon.a on 2024-02-13T03:13:03.846 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-02-13T03:13:03.972 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /etc/systemd/system/ceph.target. 2024-02-13T03:13:04.102 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715.target -> /etc/systemd/system/ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715.target. 2024-02-13T03:13:04.102 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715.target -> /etc/systemd/system/ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715.target. 2024-02-13T03:13:04.367 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Failed to reset failed state of unit ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mon.a.service: Unit ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mon.a.service is not loaded. 2024-02-13T03:13:04.376 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Created symlink /etc/systemd/system/ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715.target.wants/ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mon.a.service -> /etc/systemd/system/ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@.service. 2024-02-13T03:13:04.498 INFO:teuthology.orchestra.run.smithi161.stderr:firewalld does not appear to be present 2024-02-13T03:13:04.498 INFO:teuthology.orchestra.run.smithi161.stderr:Not possible to enable service . firewalld.service is not available 2024-02-13T03:13:04.498 INFO:teuthology.orchestra.run.smithi161.stderr:Waiting for mon to start... 2024-02-13T03:13:04.498 INFO:teuthology.orchestra.run.smithi161.stderr:Waiting for mon... 2024-02-13T03:13:04.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:04 smithi161 systemd[1]: Started Ceph mon.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:13:05.924 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:05 smithi161 bash[19121]: cluster 2024-02-13T03:13:05.823220+0000 mon.a (mon.0) 0 : [INF] mkfs b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:13:05.924 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:05 smithi161 bash[19121]: cluster 2024-02-13T03:13:05.805007+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-13T03:13:05.967 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.962+0000 7ff516560700 1 Processor -- start 2024-02-13T03:13:05.967 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.962+0000 7ff516560700 1 -- start start 2024-02-13T03:13:05.967 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.962+0000 7ff516560700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff5100febc0 con 0x7ff5100fbd00 2024-02-13T03:13:05.968 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.962+0000 7ff50ffff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff5100fbd00 0x7ff5100fc120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:36906/0 (socket says 172.21.15.161:36906) 2024-02-13T03:13:05.968 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.962+0000 7ff50ffff700 1 -- 172.21.15.161:0/378649493 learned_addr learned my addr 172.21.15.161:0/378649493 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.962+0000 7ff50effd700 1 -- 172.21.15.161:0/378649493 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449272801 0 0) 0x7ff5100febc0 con 0x7ff5100fbd00 2024-02-13T03:13:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.962+0000 7ff50effd700 1 -- 172.21.15.161:0/378649493 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4f80033f0 con 0x7ff5100fbd00 2024-02-13T03:13:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.962+0000 7ff50effd700 1 -- 172.21.15.161:0/378649493 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3187068489 0 0) 0x7ff4f80033f0 con 0x7ff5100fbd00 2024-02-13T03:13:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.962+0000 7ff50effd700 1 -- 172.21.15.161:0/378649493 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100ffd60 con 0x7ff5100fbd00 2024-02-13T03:13:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.962+0000 7ff50effd700 1 -- 172.21.15.161:0/378649493 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7ff500001590 con 0x7ff5100fbd00 2024-02-13T03:13:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.962+0000 7ff50effd700 1 -- 172.21.15.161:0/378649493 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff5000018f0 con 0x7ff5100fbd00 2024-02-13T03:13:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.962+0000 7ff50effd700 1 -- 172.21.15.161:0/378649493 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7ff500001c40 con 0x7ff5100fbd00 2024-02-13T03:13:05.969 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.966+0000 7ff516560700 1 -- 172.21.15.161:0/378649493 >> v1:172.21.15.161:6789/0 conn(0x7ff5100fbd00 legacy=0x7ff5100fc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:05.970 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.966+0000 7ff516560700 1 -- 172.21.15.161:0/378649493 shutdown_connections 2024-02-13T03:13:05.970 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.966+0000 7ff516560700 1 -- 172.21.15.161:0/378649493 >> 172.21.15.161:0/378649493 conn(0x7ff5100f5e30 msgr2=0x7ff5100f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:05.971 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.966+0000 7ff516560700 1 -- 172.21.15.161:0/378649493 shutdown_connections 2024-02-13T03:13:05.971 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.966+0000 7ff516560700 1 -- 172.21.15.161:0/378649493 wait complete. 2024-02-13T03:13:05.971 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.966+0000 7ff516560700 1 Processor -- start 2024-02-13T03:13:05.972 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.966+0000 7ff516560700 1 -- start start 2024-02-13T03:13:05.972 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.966+0000 7ff516560700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff5100f7e90 con 0x7ff5100fbd00 2024-02-13T03:13:05.972 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.966+0000 7ff50ffff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff5100fbd00 0x7ff5100f6380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:36910/0 (socket says 172.21.15.161:36910) 2024-02-13T03:13:05.973 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.966+0000 7ff50ffff700 1 -- 172.21.15.161:0/3010011137 learned_addr learned my addr 172.21.15.161:0/3010011137 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:05.973 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.966+0000 7ff50d7fa700 1 -- 172.21.15.161:0/3010011137 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339230231 0 0) 0x7ff5100f7e90 con 0x7ff5100fbd00 2024-02-13T03:13:05.973 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.966+0000 7ff50d7fa700 1 -- 172.21.15.161:0/3010011137 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4f8003c30 con 0x7ff5100fbd00 2024-02-13T03:13:05.973 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.970+0000 7ff50d7fa700 1 -- 172.21.15.161:0/3010011137 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 490580623 0 0) 0x7ff4f8003c30 con 0x7ff5100fbd00 2024-02-13T03:13:05.973 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.970+0000 7ff50d7fa700 1 -- 172.21.15.161:0/3010011137 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff5100f7e90 con 0x7ff5100fbd00 2024-02-13T03:13:05.974 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.970+0000 7ff50d7fa700 1 -- 172.21.15.161:0/3010011137 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7ff500001550 con 0x7ff5100fbd00 2024-02-13T03:13:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.970+0000 7ff50d7fa700 1 -- 172.21.15.161:0/3010011137 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 641101222 0 0) 0x7ff5100f7e90 con 0x7ff5100fbd00 2024-02-13T03:13:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.970+0000 7ff50d7fa700 1 -- 172.21.15.161:0/3010011137 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100f8020 con 0x7ff5100fbd00 2024-02-13T03:13:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.970+0000 7ff516560700 1 -- 172.21.15.161:0/3010011137 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5100f82d0 con 0x7ff5100fbd00 2024-02-13T03:13:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.970+0000 7ff516560700 1 -- 172.21.15.161:0/3010011137 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5100f6d90 con 0x7ff5100fbd00 2024-02-13T03:13:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.970+0000 7ff50d7fa700 1 -- 172.21.15.161:0/3010011137 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff500004b80 con 0x7ff5100fbd00 2024-02-13T03:13:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.970+0000 7ff50d7fa700 1 -- 172.21.15.161:0/3010011137 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7ff500004e90 con 0x7ff5100fbd00 2024-02-13T03:13:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.970+0000 7ff50d7fa700 1 -- 172.21.15.161:0/3010011137 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 1) v1 ==== 495+0+0 (unknown 2835230805 0 0) 0x7ff500005290 con 0x7ff5100fbd00 2024-02-13T03:13:05.975 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.970+0000 7ff50d7fa700 1 -- 172.21.15.161:0/3010011137 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 1315227326 0 0) 0x7ff500006270 con 0x7ff5100fbd00 2024-02-13T03:13:05.976 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.970+0000 7ff516560700 1 -- 172.21.15.161:0/3010011137 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4fc004f50 con 0x7ff5100fbd00 2024-02-13T03:13:05.980 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:05.974+0000 7ff50d7fa700 1 -- 172.21.15.161:0/3010011137 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (unknown 1092875540 0 259193764) 0x7ff5100f82d0 con 0x7ff5100fbd00 2024-02-13T03:13:06.113 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:06.110+0000 7ff516560700 1 -- 172.21.15.161:0/3010011137 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff4fc005a10 con 0x7ff5100fbd00 2024-02-13T03:13:06.114 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:06.110+0000 7ff50d7fa700 1 -- 172.21.15.161:0/3010011137 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (unknown 1155462804 0 1875426020) 0x7ff5000064b0 con 0x7ff5100fbd00 2024-02-13T03:13:06.114 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: cluster: 2024-02-13T03:13:06.114 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: id: b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:13:06.114 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: health: HEALTH_OK 2024-02-13T03:13:06.114 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:06.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: services: 2024-02-13T03:13:06.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: mon: 1 daemons, quorum a (age 0.295548s) 2024-02-13T03:13:06.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: mgr: no daemons active 2024-02-13T03:13:06.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2024-02-13T03:13:06.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:06.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: data: 2024-02-13T03:13:06.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2024-02-13T03:13:06.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2024-02-13T03:13:06.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2024-02-13T03:13:06.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: pgs: 2024-02-13T03:13:06.117 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:06.117 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:06.110+0000 7ff516560700 1 -- 172.21.15.161:0/3010011137 >> v1:172.21.15.161:6789/0 conn(0x7ff5100fbd00 legacy=0x7ff5100f6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:06.117 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:06.110+0000 7ff516560700 1 -- 172.21.15.161:0/3010011137 shutdown_connections 2024-02-13T03:13:06.117 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:06.110+0000 7ff516560700 1 -- 172.21.15.161:0/3010011137 >> 172.21.15.161:0/3010011137 conn(0x7ff5100f5e30 msgr2=0x7ff5100fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:06.118 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:06.110+0000 7ff516560700 1 -- 172.21.15.161:0/3010011137 shutdown_connections 2024-02-13T03:13:06.118 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:06.110+0000 7ff516560700 1 -- 172.21.15.161:0/3010011137 wait complete. 2024-02-13T03:13:06.655 INFO:teuthology.orchestra.run.smithi161.stderr:mon is available 2024-02-13T03:13:06.656 INFO:teuthology.orchestra.run.smithi161.stderr:Assimilating anything we can from ceph.conf... 2024-02-13T03:13:07.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:06 smithi161 bash[19121]: cluster 2024-02-13T03:13:05.818470+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-13T03:13:07.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:06 smithi161 bash[19121]: cluster 2024-02-13T03:13:05.820135+0000 mon.a (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {a=v1:172.21.15.161:6789/0} removed_ranks: {} 2024-02-13T03:13:07.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:06 smithi161 bash[19121]: cluster 2024-02-13T03:13:05.830249+0000 mon.a (mon.0) 4 : cluster [DBG] fsmap 2024-02-13T03:13:07.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:06 smithi161 bash[19121]: cluster 2024-02-13T03:13:05.836303+0000 mon.a (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-13T03:13:07.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:06 smithi161 bash[19121]: cluster 2024-02-13T03:13:05.837319+0000 mon.a (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2024-02-13T03:13:07.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:06 smithi161 bash[19121]: audit 2024-02-13T03:13:06.113952+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? v1:172.21.15.161:0/3010011137' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-13T03:13:07.667 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6ebadec700 1 Processor -- start 2024-02-13T03:13:07.667 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6ebadec700 1 -- start start 2024-02-13T03:13:07.668 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6ebadec700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6eb40fc8a0 con 0x7f6eb40fbd70 2024-02-13T03:13:07.668 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6eb8b88700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f6eb40fbd70 0x7f6eb40fc190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34602/0 (socket says 172.21.15.161:34602) 2024-02-13T03:13:07.668 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6eb8b88700 1 -- 172.21.15.161:0/3253042740 learned_addr learned my addr 172.21.15.161:0/3253042740 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:07.669 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6eb37fe700 1 -- 172.21.15.161:0/3253042740 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122155552 0 0) 0x7f6eb40fc8a0 con 0x7f6eb40fbd70 2024-02-13T03:13:07.669 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6eb37fe700 1 -- 172.21.15.161:0/3253042740 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ea00033f0 con 0x7f6eb40fbd70 2024-02-13T03:13:07.669 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6eb37fe700 1 -- 172.21.15.161:0/3253042740 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 243998145 0 0) 0x7f6ea00033f0 con 0x7f6eb40fbd70 2024-02-13T03:13:07.669 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6eb37fe700 1 -- 172.21.15.161:0/3253042740 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb40fca30 con 0x7f6eb40fbd70 2024-02-13T03:13:07.669 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6eb37fe700 1 -- 172.21.15.161:0/3253042740 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f6eac001590 con 0x7f6eb40fbd70 2024-02-13T03:13:07.669 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6eb37fe700 1 -- 172.21.15.161:0/3253042740 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6eac0018f0 con 0x7f6eb40fbd70 2024-02-13T03:13:07.671 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6eb37fe700 1 -- 172.21.15.161:0/3253042740 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f6eac001c40 con 0x7f6eb40fbd70 2024-02-13T03:13:07.671 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6ebadec700 1 -- 172.21.15.161:0/3253042740 >> v1:172.21.15.161:6789/0 conn(0x7f6eb40fbd70 legacy=0x7f6eb40fc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:07.671 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6ebadec700 1 -- 172.21.15.161:0/3253042740 shutdown_connections 2024-02-13T03:13:07.671 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6ebadec700 1 -- 172.21.15.161:0/3253042740 >> 172.21.15.161:0/3253042740 conn(0x7f6eb40f55c0 msgr2=0x7f6eb40f79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:07.671 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6ebadec700 1 -- 172.21.15.161:0/3253042740 shutdown_connections 2024-02-13T03:13:07.671 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.662+0000 7f6ebadec700 1 -- 172.21.15.161:0/3253042740 wait complete. 2024-02-13T03:13:07.671 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6ebadec700 1 Processor -- start 2024-02-13T03:13:07.671 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6ebadec700 1 -- start start 2024-02-13T03:13:07.671 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6ebadec700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6eb410c320 con 0x7f6eb40fbd70 2024-02-13T03:13:07.671 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6eb8b88700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f6eb40fbd70 0x7f6eb410bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34618/0 (socket says 172.21.15.161:34618) 2024-02-13T03:13:07.671 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6eb8b88700 1 -- 172.21.15.161:0/1298475648 learned_addr learned my addr 172.21.15.161:0/1298475648 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:07.673 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740995658 0 0) 0x7f6eb410c320 con 0x7f6eb40fbd70 2024-02-13T03:13:07.673 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ea0003c30 con 0x7f6eb40fbd70 2024-02-13T03:13:07.673 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 239693362 0 0) 0x7f6ea0003c30 con 0x7f6eb40fbd70 2024-02-13T03:13:07.673 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6eb410c320 con 0x7f6eb40fbd70 2024-02-13T03:13:07.673 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f6eac001b80 con 0x7f6eb40fbd70 2024-02-13T03:13:07.673 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2209748635 0 0) 0x7f6eb410c320 con 0x7f6eb40fbd70 2024-02-13T03:13:07.673 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb410c4b0 con 0x7f6eb40fbd70 2024-02-13T03:13:07.674 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6ebadec700 1 -- 172.21.15.161:0/1298475648 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb410c760 con 0x7f6eb40fbd70 2024-02-13T03:13:07.674 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6ebadec700 1 -- 172.21.15.161:0/1298475648 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb4110450 con 0x7f6eb40fbd70 2024-02-13T03:13:07.674 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6eac004b80 con 0x7f6eb40fbd70 2024-02-13T03:13:07.674 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f6eac004e90 con 0x7f6eb40fbd70 2024-02-13T03:13:07.674 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 1) v1 ==== 495+0+0 (unknown 2835230805 0 0) 0x7f6eac005310 con 0x7f6eb40fbd70 2024-02-13T03:13:07.674 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.666+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 1315227326 0 0) 0x7f6eac0062b0 con 0x7f6eb40fbd70 2024-02-13T03:13:07.674 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.670+0000 7f6eab7fe700 1 -- 172.21.15.161:0/1298475648 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e98004f20 con 0x7f6eb40fbd70 2024-02-13T03:13:07.678 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.674+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (unknown 1092875540 0 259193764) 0x7f6eac006590 con 0x7f6eb40fbd70 2024-02-13T03:13:07.805 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.798+0000 7f6eab7fe700 1 -- 172.21.15.161:0/1298475648 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f6e98005750 con 0x7f6eb40fbd70 2024-02-13T03:13:07.812 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.806+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 <== mon.0 v1:172.21.15.161:6789/0 10 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f6eac0068b0 con 0x7f6eb40fbd70 2024-02-13T03:13:07.812 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.806+0000 7f6eb1ffb700 1 -- 172.21.15.161:0/1298475648 <== mon.0 v1:172.21.15.161:6789/0 11 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+343 (unknown 1213389831 0 4014594122) 0x7f6eac007640 con 0x7f6eb40fbd70 2024-02-13T03:13:07.812 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: [global] 2024-02-13T03:13:07.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: fsid = b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:13:07.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-13T03:13:07.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: mon_host = [v1:172.21.15.161:6789] 2024-02-13T03:13:07.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-13T03:13:07.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-13T03:13:07.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-13T03:13:07.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-13T03:13:07.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: [mgr] 2024-02-13T03:13:07.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-13T03:13:07.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: [osd] 2024-02-13T03:13:07.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-13T03:13:07.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-13T03:13:07.816 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.810+0000 7f6eab7fe700 1 -- 172.21.15.161:0/1298475648 >> v1:172.21.15.161:6789/0 conn(0x7f6eb40fbd70 legacy=0x7f6eb410bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:07.817 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.810+0000 7f6eab7fe700 1 -- 172.21.15.161:0/1298475648 shutdown_connections 2024-02-13T03:13:07.817 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.810+0000 7f6eab7fe700 1 -- 172.21.15.161:0/1298475648 >> 172.21.15.161:0/1298475648 conn(0x7f6eb40f55c0 msgr2=0x7f6eb40f79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:07.817 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.810+0000 7f6eab7fe700 1 -- 172.21.15.161:0/1298475648 shutdown_connections 2024-02-13T03:13:07.817 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:07.810+0000 7f6eab7fe700 1 -- 172.21.15.161:0/1298475648 wait complete. 2024-02-13T03:13:08.174 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:07 smithi161 bash[19121]: audit 2024-02-13T03:13:07.805748+0000 mon.a (mon.0) 8 : audit [INF] from='client.? v1:172.21.15.161:0/1298475648' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-13T03:13:08.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:07 smithi161 bash[19121]: audit 2024-02-13T03:13:07.812315+0000 mon.a (mon.0) 9 : audit [INF] from='client.? v1:172.21.15.161:0/1298475648' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-13T03:13:08.327 INFO:teuthology.orchestra.run.smithi161.stderr:Generating new minimal ceph.conf... 2024-02-13T03:13:09.275 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.270+0000 7f5193def700 1 Processor -- start 2024-02-13T03:13:09.276 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.270+0000 7f5193def700 1 -- start start 2024-02-13T03:13:09.276 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.270+0000 7f5193def700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f518c0fc820 con 0x7f518c0fbcf0 2024-02-13T03:13:09.276 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.270+0000 7f5191b8b700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f518c0fbcf0 0x7f518c0fc110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34630/0 (socket says 172.21.15.161:34630) 2024-02-13T03:13:09.276 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.270+0000 7f5191b8b700 1 -- 172.21.15.161:0/3043701613 learned_addr learned my addr 172.21.15.161:0/3043701613 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:09.276 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.270+0000 7f5190b89700 1 -- 172.21.15.161:0/3043701613 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905652528 0 0) 0x7f518c0fc820 con 0x7f518c0fbcf0 2024-02-13T03:13:09.277 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.270+0000 7f5190b89700 1 -- 172.21.15.161:0/3043701613 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51740033f0 con 0x7f518c0fbcf0 2024-02-13T03:13:09.277 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.270+0000 7f5190b89700 1 -- 172.21.15.161:0/3043701613 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3496212022 0 0) 0x7f51740033f0 con 0x7f518c0fbcf0 2024-02-13T03:13:09.278 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.270+0000 7f5190b89700 1 -- 172.21.15.161:0/3043701613 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0ffc60 con 0x7f518c0fbcf0 2024-02-13T03:13:09.278 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.270+0000 7f5190b89700 1 -- 172.21.15.161:0/3043701613 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f517c001590 con 0x7f518c0fbcf0 2024-02-13T03:13:09.278 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.270+0000 7f5190b89700 1 -- 172.21.15.161:0/3043701613 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f517c001d00 con 0x7f518c0fbcf0 2024-02-13T03:13:09.278 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.270+0000 7f5190b89700 1 -- 172.21.15.161:0/3043701613 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f517c005700 con 0x7f518c0fbcf0 2024-02-13T03:13:09.278 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f5193def700 1 -- 172.21.15.161:0/3043701613 >> v1:172.21.15.161:6789/0 conn(0x7f518c0fbcf0 legacy=0x7f518c0fc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:09.278 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f5193def700 1 -- 172.21.15.161:0/3043701613 shutdown_connections 2024-02-13T03:13:09.279 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f5193def700 1 -- 172.21.15.161:0/3043701613 >> 172.21.15.161:0/3043701613 conn(0x7f518c0f55c0 msgr2=0x7f518c0f79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:09.279 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f5193def700 1 -- 172.21.15.161:0/3043701613 shutdown_connections 2024-02-13T03:13:09.279 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f5193def700 1 -- 172.21.15.161:0/3043701613 wait complete. 2024-02-13T03:13:09.279 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f5193def700 1 Processor -- start 2024-02-13T03:13:09.279 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f5193def700 1 -- start start 2024-02-13T03:13:09.279 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f5193def700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f518c10a860 con 0x7f518c0fbcf0 2024-02-13T03:13:09.280 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f5191b8b700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f518c0fbcf0 0x7f518c10a150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34640/0 (socket says 172.21.15.161:34640) 2024-02-13T03:13:09.281 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f5191b8b700 1 -- 172.21.15.161:0/1336438138 learned_addr learned my addr 172.21.15.161:0/1336438138 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:09.281 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f518affd700 1 -- 172.21.15.161:0/1336438138 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037303589 0 0) 0x7f518c10a860 con 0x7f518c0fbcf0 2024-02-13T03:13:09.281 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f518affd700 1 -- 172.21.15.161:0/1336438138 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5174005210 con 0x7f518c0fbcf0 2024-02-13T03:13:09.282 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f518affd700 1 -- 172.21.15.161:0/1336438138 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1169452296 0 0) 0x7f5174005210 con 0x7f518c0fbcf0 2024-02-13T03:13:09.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f518affd700 1 -- 172.21.15.161:0/1336438138 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f518c10a860 con 0x7f518c0fbcf0 2024-02-13T03:13:09.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f518affd700 1 -- 172.21.15.161:0/1336438138 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f517c000fe0 con 0x7f518c0fbcf0 2024-02-13T03:13:09.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f518affd700 1 -- 172.21.15.161:0/1336438138 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1527239577 0 0) 0x7f518c10a860 con 0x7f518c0fbcf0 2024-02-13T03:13:09.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f518affd700 1 -- 172.21.15.161:0/1336438138 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c10a9f0 con 0x7f518c0fbcf0 2024-02-13T03:13:09.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f5193def700 1 -- 172.21.15.161:0/1336438138 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c10aca0 con 0x7f518c0fbcf0 2024-02-13T03:13:09.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f5193def700 1 -- 172.21.15.161:0/1336438138 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c10e990 con 0x7f518c0fbcf0 2024-02-13T03:13:09.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f518affd700 1 -- 172.21.15.161:0/1336438138 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f517c004b10 con 0x7f518c0fbcf0 2024-02-13T03:13:09.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.274+0000 7f518affd700 1 -- 172.21.15.161:0/1336438138 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f517c0060c0 con 0x7f518c0fbcf0 2024-02-13T03:13:09.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.278+0000 7f518affd700 1 -- 172.21.15.161:0/1336438138 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 1) v1 ==== 495+0+0 (unknown 2835230805 0 0) 0x7f517c006560 con 0x7f518c0fbcf0 2024-02-13T03:13:09.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.278+0000 7f518affd700 1 -- 172.21.15.161:0/1336438138 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 1315227326 0 0) 0x7f517c0075a0 con 0x7f518c0fbcf0 2024-02-13T03:13:09.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.278+0000 7f5193def700 1 -- 172.21.15.161:0/1336438138 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5178004f50 con 0x7f518c0fbcf0 2024-02-13T03:13:09.285 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.282+0000 7f518affd700 1 -- 172.21.15.161:0/1336438138 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (unknown 1092875540 0 259193764) 0x7f517c0059a0 con 0x7f518c0fbcf0 2024-02-13T03:13:09.417 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.410+0000 7f5193def700 1 -- 172.21.15.161:0/1336438138 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f5178005200 con 0x7f518c0fbcf0 2024-02-13T03:13:09.418 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.414+0000 7f518affd700 1 -- 172.21.15.161:0/1336438138 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+148 (unknown 1452402520 0 2662391018) 0x7f517c001430 con 0x7f518c0fbcf0 2024-02-13T03:13:09.419 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.414+0000 7f5193def700 1 -- 172.21.15.161:0/1336438138 >> v1:172.21.15.161:6789/0 conn(0x7f518c0fbcf0 legacy=0x7f518c10a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:09.419 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.414+0000 7f5193def700 1 -- 172.21.15.161:0/1336438138 shutdown_connections 2024-02-13T03:13:09.419 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.414+0000 7f5193def700 1 -- 172.21.15.161:0/1336438138 >> 172.21.15.161:0/1336438138 conn(0x7f518c0f55c0 msgr2=0x7f518c0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:09.419 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.414+0000 7f5193def700 1 -- 172.21.15.161:0/1336438138 shutdown_connections 2024-02-13T03:13:09.419 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:09.414+0000 7f5193def700 1 -- 172.21.15.161:0/1336438138 wait complete. 2024-02-13T03:13:09.920 INFO:teuthology.orchestra.run.smithi161.stderr:Restarting the monitor... 2024-02-13T03:13:09.924 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:09 smithi161 bash[19121]: audit 2024-02-13T03:13:09.417872+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? v1:172.21.15.161:0/1336438138' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:13:10.386 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:09 smithi161 systemd[1]: Stopping Ceph mon.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:13:10.386 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:09 smithi161 bash[19644]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-mon.a 2024-02-13T03:13:10.386 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:10 smithi161 bash[19121]: debug 2024-02-13T03:13:10.098+0000 7f3a692b6700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-13T03:13:10.386 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:10 smithi161 bash[19121]: debug 2024-02-13T03:13:10.098+0000 7f3a692b6700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-13T03:13:10.866 INFO:teuthology.orchestra.run.smithi161.stderr:Setting public_network to 172.21.15.254/32,172.21.0.0/20 in mon config section 2024-02-13T03:13:11.174 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:10 smithi161 bash[19644]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-mon-a 2024-02-13T03:13:11.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:10 smithi161 systemd[1]: Stopped Ceph mon.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:13:11.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:10 smithi161 systemd[1]: Started Ceph mon.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:13:12.316 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.310+0000 7f66b8d06700 1 Processor -- start 2024-02-13T03:13:12.316 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.310+0000 7f66b8d06700 1 -- start start 2024-02-13T03:13:12.316 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.310+0000 7f66b8d06700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f66b40fd890 con 0x7f66b40fcd60 2024-02-13T03:13:12.317 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.310+0000 7f66b259c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f66b40fcd60 0x7f66b40fd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34648/0 (socket says 172.21.15.161:34648) 2024-02-13T03:13:12.317 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.310+0000 7f66b259c700 1 -- 172.21.15.161:0/4137638402 learned_addr learned my addr 172.21.15.161:0/4137638402 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:12.317 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.314+0000 7f66b159a700 1 -- 172.21.15.161:0/4137638402 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361900808 0 0) 0x7f66b40fd890 con 0x7f66b40fcd60 2024-02-13T03:13:12.318 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.314+0000 7f66b159a700 1 -- 172.21.15.161:0/4137638402 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f669c0033f0 con 0x7f66b40fcd60 2024-02-13T03:13:12.318 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.314+0000 7f66b159a700 1 -- 172.21.15.161:0/4137638402 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3009137333 0 0) 0x7f669c0033f0 con 0x7f66b40fcd60 2024-02-13T03:13:12.318 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.314+0000 7f66b159a700 1 -- 172.21.15.161:0/4137638402 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40fda20 con 0x7f66b40fcd60 2024-02-13T03:13:12.318 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.314+0000 7f66b159a700 1 -- 172.21.15.161:0/4137638402 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f66a4001590 con 0x7f66b40fcd60 2024-02-13T03:13:12.319 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.314+0000 7f66b159a700 1 -- 172.21.15.161:0/4137638402 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f66a4001d00 con 0x7f66b40fcd60 2024-02-13T03:13:12.319 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.314+0000 7f66b159a700 1 -- 172.21.15.161:0/4137638402 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f66a4005700 con 0x7f66b40fcd60 2024-02-13T03:13:12.320 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.314+0000 7f66b8d06700 1 -- 172.21.15.161:0/4137638402 >> v1:172.21.15.161:6789/0 conn(0x7f66b40fcd60 legacy=0x7f66b40fd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:12.320 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.314+0000 7f66b8d06700 1 -- 172.21.15.161:0/4137638402 shutdown_connections 2024-02-13T03:13:12.320 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.314+0000 7f66b8d06700 1 -- 172.21.15.161:0/4137638402 >> 172.21.15.161:0/4137638402 conn(0x7f66b40f55a0 msgr2=0x7f66b40f79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:12.320 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.314+0000 7f66b8d06700 1 -- 172.21.15.161:0/4137638402 shutdown_connections 2024-02-13T03:13:12.320 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.314+0000 7f66b8d06700 1 -- 172.21.15.161:0/4137638402 wait complete. 2024-02-13T03:13:12.321 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66b8d06700 1 Processor -- start 2024-02-13T03:13:12.321 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66b8d06700 1 -- start start 2024-02-13T03:13:12.322 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66b8d06700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f66b410a760 con 0x7f66b40fcd60 2024-02-13T03:13:12.322 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66b259c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f66b40fcd60 0x7f66b410a050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34656/0 (socket says 172.21.15.161:34656) 2024-02-13T03:13:12.322 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66b259c700 1 -- 172.21.15.161:0/1723331036 learned_addr learned my addr 172.21.15.161:0/1723331036 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:12.323 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66a37fe700 1 -- 172.21.15.161:0/1723331036 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161759713 0 0) 0x7f66b410a760 con 0x7f66b40fcd60 2024-02-13T03:13:12.323 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66a37fe700 1 -- 172.21.15.161:0/1723331036 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f669c005210 con 0x7f66b40fcd60 2024-02-13T03:13:12.323 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66a37fe700 1 -- 172.21.15.161:0/1723331036 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 812115861 0 0) 0x7f669c005210 con 0x7f66b40fcd60 2024-02-13T03:13:12.324 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66a37fe700 1 -- 172.21.15.161:0/1723331036 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f66b410a760 con 0x7f66b40fcd60 2024-02-13T03:13:12.324 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66a37fe700 1 -- 172.21.15.161:0/1723331036 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f66a4000fe0 con 0x7f66b40fcd60 2024-02-13T03:13:12.324 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66a37fe700 1 -- 172.21.15.161:0/1723331036 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3347067867 0 0) 0x7f66b410a760 con 0x7f66b40fcd60 2024-02-13T03:13:12.325 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66a37fe700 1 -- 172.21.15.161:0/1723331036 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b410a8f0 con 0x7f66b40fcd60 2024-02-13T03:13:12.325 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66b8d06700 1 -- 172.21.15.161:0/1723331036 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b410aba0 con 0x7f66b40fcd60 2024-02-13T03:13:12.325 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66b8d06700 1 -- 172.21.15.161:0/1723331036 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b410e890 con 0x7f66b40fcd60 2024-02-13T03:13:12.325 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66a37fe700 1 -- 172.21.15.161:0/1723331036 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f66a4004bd0 con 0x7f66b40fcd60 2024-02-13T03:13:12.325 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66a37fe700 1 -- 172.21.15.161:0/1723331036 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f66a40061a0 con 0x7f66b40fcd60 2024-02-13T03:13:12.325 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66a37fe700 1 -- 172.21.15.161:0/1723331036 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 1) v1 ==== 495+0+0 (unknown 2835230805 0 0) 0x7f66a40066a0 con 0x7f66b40fcd60 2024-02-13T03:13:12.326 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.318+0000 7f66a37fe700 1 -- 172.21.15.161:0/1723331036 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 1315227326 0 0) 0x7f66a4006cb0 con 0x7f66b40fcd60 2024-02-13T03:13:12.326 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.322+0000 7f66b8d06700 1 -- 172.21.15.161:0/1723331036 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6694004f50 con 0x7f66b40fcd60 2024-02-13T03:13:12.329 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.326+0000 7f66a37fe700 1 -- 172.21.15.161:0/1723331036 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (unknown 1092875540 0 259193764) 0x7f66a40059f0 con 0x7f66b40fcd60 2024-02-13T03:13:12.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.074+0000 7f589c962880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-13T03:13:12.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.074+0000 7f589c962880 0 ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable), process ceph-mon, pid 7 2024-02-13T03:13:12.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.074+0000 7f589c962880 0 pidfile_write: ignore empty --pid-file 2024-02-13T03:13:12.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 0 load: jerasure load: lrc load: isa 2024-02-13T03:13:12.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: RocksDB version: 6.8.1 2024-02-13T03:13:12.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-13T03:13:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Compile date Feb 8 2024 2024-02-13T03:13:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: DB SUMMARY 2024-02-13T03:13:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: CURRENT file: CURRENT 2024-02-13T03:13:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-13T03:13:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 131 Bytes 2024-02-13T03:13:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000004.sst 2024-02-13T03:13:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000006.log size: 85985 ; 2024-02-13T03:13:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.error_if_exists: 0 2024-02-13T03:13:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.create_if_missing: 0 2024-02-13T03:13:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.paranoid_checks: 1 2024-02-13T03:13:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.env: 0x55f32b859080 2024-02-13T03:13:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.fs: Posix File System 2024-02-13T03:13:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.info_log: 0x55f32db261c0 2024-02-13T03:13:12.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-13T03:13:12.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.statistics: (nil) 2024-02-13T03:13:12.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.use_fsync: 0 2024-02-13T03:13:12.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_log_file_size: 0 2024-02-13T03:13:12.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-13T03:13:12.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-13T03:13:12.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-13T03:13:12.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-13T03:13:12.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.allow_fallocate: 1 2024-02-13T03:13:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-13T03:13:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-13T03:13:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.use_direct_reads: 0 2024-02-13T03:13:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-13T03:13:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-13T03:13:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.db_log_dir: 2024-02-13T03:13:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-02-13T03:13:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-13T03:13:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_subcompactions: 1 2024-02-13T03:13:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_background_flushes: -1 2024-02-13T03:13:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-13T03:13:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-13T03:13:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-13T03:13:12.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-13T03:13:12.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-13T03:13:12.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-13T03:13:12.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-13T03:13:12.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.write_buffer_manager: 0x55f32e81a390 2024-02-13T03:13:12.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-13T03:13:12.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-13T03:13:12.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-13T03:13:12.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-13T03:13:12.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-13T03:13:12.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-13T03:13:12.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-13T03:13:12.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.unordered_write: 0 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.row_cache: None 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.wal_filter: None 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.preserve_deletes: 0 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.two_write_queues: 0 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-13T03:13:12.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.atomic_flush: 0 2024-02-13T03:13:12.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-13T03:13:12.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-13T03:13:12.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-13T03:13:12.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.log_readahead_size: 0 2024-02-13T03:13:12.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-13T03:13:12.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_background_jobs: 2 2024-02-13T03:13:12.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_background_compactions: -1 2024-02-13T03:13:12.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-13T03:13:12.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-13T03:13:12.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-13T03:13:12.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-13T03:13:12.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-13T03:13:12.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-13T03:13:12.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-13T03:13:12.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-13T03:13:12.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_open_files: -1 2024-02-13T03:13:12.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-13T03:13:12.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-13T03:13:12.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-13T03:13:12.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-13T03:13:12.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Compression algorithms supported: 2024-02-13T03:13:12.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-13T03:13:12.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: kZSTD supported: 0 2024-02-13T03:13:12.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: kXpressCompression supported: 0 2024-02-13T03:13:12.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: kLZ4Compression supported: 1 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: kBZip2Compression supported: 0 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: kZlibCompression supported: 1 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: kSnappyCompression supported: 1 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.merge_operator: 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compaction_filter: None 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-13T03:13:12.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f32dac9d28) 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: cache_index_and_filter_blocks: 1 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: pin_top_level_index_and_filter: 1 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: index_type: 0 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: data_block_index_type: 0 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: index_shortening: 1 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: data_block_hash_table_util_ratio: 0.750000 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: hash_index_allow_collision: 1 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: checksum: 1 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: no_block_cache: 0 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: block_cache: 0x55f32db00f10 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: block_cache_name: BinnedLRUCache 2024-02-13T03:13:12.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: block_cache_options: 2024-02-13T03:13:12.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: capacity : 536870912 2024-02-13T03:13:12.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: num_shard_bits : 4 2024-02-13T03:13:12.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: strict_capacity_limit : 0 2024-02-13T03:13:12.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: high_pri_pool_ratio: 0.000 2024-02-13T03:13:12.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: block_cache_compressed: (nil) 2024-02-13T03:13:12.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: persistent_cache: (nil) 2024-02-13T03:13:12.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: block_size: 4096 2024-02-13T03:13:12.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: block_size_deviation: 10 2024-02-13T03:13:12.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: block_restart_interval: 16 2024-02-13T03:13:12.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: index_block_restart_interval: 1 2024-02-13T03:13:12.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: metadata_block_size: 4096 2024-02-13T03:13:12.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: partition_filters: 0 2024-02-13T03:13:12.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: use_delta_encoding: 1 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: whole_key_filtering: 1 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: verify_compression: 0 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: read_amp_bytes_per_bit: 0 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: format_version: 2 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: enable_index_compression: 1 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: block_align: 0 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compression: NoCompression 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.num_levels: 7 2024-02-13T03:13:12.437 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-13T03:13:12.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-13T03:13:12.439 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-13T03:13:12.440 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-13T03:13:12.440 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-13T03:13:12.440 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-13T03:13:12.440 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-13T03:13:12.440 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-13T03:13:12.440 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-13T03:13:12.440 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-13T03:13:12.440 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-13T03:13:12.440 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-13T03:13:12.440 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-13T03:13:12.440 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-13T03:13:12.440 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.table_properties_collectors: 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.inplace_update_support: 0 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.bloom_locality: 0 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.max_successive_merges: 0 2024-02-13T03:13:12.441 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.ttl: 2592000 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.078+0000 7f589c962880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.082+0000 7f589c962880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.082+0000 7f589c962880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 4 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.082+0000 7f589c962880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707793992086869, "job": 1, "event": "recovery_started", "log_files": [6]} 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.082+0000 7f589c962880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #6 mode 2 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.082+0000 7f589c962880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.086+0000 7f589c962880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707793992090037, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 82410, "table_properties": {"data_size": 80684, "index_size": 331, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 581, "raw_key_size": 10041, "raw_average_key_size": 47, "raw_value_size": 74987, "raw_average_value_size": 357, "num_data_blocks": 11, "num_entries": 210, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1707793992, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.086+0000 7f589c962880 4 rocksdb: [version_set.cc:3826] Creating manifest 8 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.086+0000 7f589c962880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707793992091329, "job": 1, "event": "recovery_finished"} 2024-02-13T03:13:12.442 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.090+0000 7f589c962880 4 rocksdb: DB pointer 0x55f32dbc9800 2024-02-13T03:13:12.443 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.090+0000 7f5883753700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-13T03:13:12.443 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.090+0000 7f5883753700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-13T03:13:12.443 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: ** DB Stats ** 2024-02-13T03:13:12.443 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T03:13:12.443 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-13T03:13:12.443 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T03:13:12.443 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T03:13:12.443 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-13T03:13:12.443 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-13T03:13:12.443 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T03:13:12.444 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: ** Compaction Stats [default] ** 2024-02-13T03:13:12.444 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-13T03:13:12.444 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T03:13:12.444 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: L0 2/0 82.10 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 39.7 0.00 0.00 1 0.002 0 0 2024-02-13T03:13:12.444 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Sum 2/0 82.10 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 39.7 0.00 0.00 1 0.002 0 0 2024-02-13T03:13:12.444 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 39.7 0.00 0.00 1 0.002 0 0 2024-02-13T03:13:12.444 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: ** Compaction Stats [default] ** 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 39.7 0.00 0.00 1 0.002 0 0 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: AddFile(Total Files): cumulative 0, interval 0 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: AddFile(Keys): cumulative 0, interval 0 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Cumulative compaction: 0.00 GB write, 7.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Interval compaction: 0.00 GB write, 7.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: ** File Read Latency Histogram By Level [default] ** 2024-02-13T03:13:12.445 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: ** Compaction Stats [default] ** 2024-02-13T03:13:12.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-13T03:13:12.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T03:13:12.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: L0 2/0 82.10 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 39.7 0.00 0.00 1 0.002 0 0 2024-02-13T03:13:12.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Sum 2/0 82.10 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 39.7 0.00 0.00 1 0.002 0 0 2024-02-13T03:13:12.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-13T03:13:12.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: ** Compaction Stats [default] ** 2024-02-13T03:13:12.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-13T03:13:12.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T03:13:12.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 39.7 0.00 0.00 1 0.002 0 0 2024-02-13T03:13:12.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T03:13:12.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-13T03:13:12.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-13T03:13:12.446 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: AddFile(Total Files): cumulative 0, interval 0 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: AddFile(Keys): cumulative 0, interval 0 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Cumulative compaction: 0.00 GB write, 7.81 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: ** File Read Latency Histogram By Level [default] ** 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.090+0000 7f589c962880 0 starting mon.a rank 0 at public addrs v1:172.21.15.161:6789/0 at bind addrs v1:172.21.15.161:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.090+0000 7f589c962880 1 mon.a@-1(???) e1 preinit fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.090+0000 7f589c962880 0 mon.a@-1(???).mds e1 new map 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.090+0000 7f589c962880 0 mon.a@-1(???).mds e1 print_map 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: e1 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: legacy client fscid: -1 2024-02-13T03:13:12.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: 2024-02-13T03:13:12.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: No filesystems configured 2024-02-13T03:13:12.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.090+0000 7f589c962880 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-13T03:13:12.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.090+0000 7f589c962880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T03:13:12.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: audit 2024-02-13T03:13:09.417872+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? v1:172.21.15.161:0/1336438138' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:13:12.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: debug 2024-02-13T03:13:12.090+0000 7f589c962880 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-13T03:13:12.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: cluster 2024-02-13T03:13:12.100720+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-13T03:13:12.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: cluster 2024-02-13T03:13:12.100796+0000 mon.a (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {a=v1:172.21.15.161:6789/0} removed_ranks: {} 2024-02-13T03:13:12.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: cluster 2024-02-13T03:13:12.101252+0000 mon.a (mon.0) 3 : cluster [DBG] fsmap 2024-02-13T03:13:12.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: cluster 2024-02-13T03:13:12.101302+0000 mon.a (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-13T03:13:12.448 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:12 smithi161 bash[19769]: cluster 2024-02-13T03:13:12.102952+0000 mon.a (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2024-02-13T03:13:12.459 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.454+0000 7f66b8d06700 1 -- 172.21.15.161:0/1723331036 --> v1:172.21.15.161:6789/0 -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f6694005200 con 0x7f66b40fcd60 2024-02-13T03:13:12.467 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.462+0000 7f66a37fe700 1 -- 172.21.15.161:0/1723331036 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 141+0+0 (unknown 223993786 0 0) 0x7f66a4001430 con 0x7f66b40fcd60 2024-02-13T03:13:12.468 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.462+0000 7f66b8d06700 1 -- 172.21.15.161:0/1723331036 >> v1:172.21.15.161:6789/0 conn(0x7f66b40fcd60 legacy=0x7f66b410a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:12.468 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.462+0000 7f66b8d06700 1 -- 172.21.15.161:0/1723331036 shutdown_connections 2024-02-13T03:13:12.468 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.462+0000 7f66b8d06700 1 -- 172.21.15.161:0/1723331036 >> 172.21.15.161:0/1723331036 conn(0x7f66b40f55a0 msgr2=0x7f66b40f61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:12.468 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.462+0000 7f66b8d06700 1 -- 172.21.15.161:0/1723331036 shutdown_connections 2024-02-13T03:13:12.469 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:12.462+0000 7f66b8d06700 1 -- 172.21.15.161:0/1723331036 wait complete. 2024-02-13T03:13:13.064 INFO:teuthology.orchestra.run.smithi161.stderr:Wrote config to /etc/ceph/ceph.conf 2024-02-13T03:13:13.064 INFO:teuthology.orchestra.run.smithi161.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-13T03:13:13.065 INFO:teuthology.orchestra.run.smithi161.stderr:Creating mgr... 2024-02-13T03:13:13.065 INFO:teuthology.orchestra.run.smithi161.stderr:Verifying port 9283 ... 2024-02-13T03:13:13.190 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Failed to reset failed state of unit ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mgr.y.service: Unit ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mgr.y.service is not loaded. 2024-02-13T03:13:13.199 INFO:teuthology.orchestra.run.smithi161.stderr:systemctl: Created symlink /etc/systemd/system/ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715.target.wants/ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mgr.y.service -> /etc/systemd/system/ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@.service. 2024-02-13T03:13:13.318 INFO:teuthology.orchestra.run.smithi161.stderr:firewalld does not appear to be present 2024-02-13T03:13:13.318 INFO:teuthology.orchestra.run.smithi161.stderr:Not possible to enable service . firewalld.service is not available 2024-02-13T03:13:13.318 INFO:teuthology.orchestra.run.smithi161.stderr:firewalld does not appear to be present 2024-02-13T03:13:13.318 INFO:teuthology.orchestra.run.smithi161.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2024-02-13T03:13:13.318 INFO:teuthology.orchestra.run.smithi161.stderr:Waiting for mgr to start... 2024-02-13T03:13:13.319 INFO:teuthology.orchestra.run.smithi161.stderr:Waiting for mgr... 2024-02-13T03:13:13.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:13 smithi161 systemd[1]: Started Ceph mgr.y for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:13:13.924 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:13 smithi161 bash[19769]: audit 2024-02-13T03:13:12.466337+0000 mon.a (mon.0) 6 : audit [INF] from='client.? v1:172.21.15.161:0/1723331036' entity='client.admin' 2024-02-13T03:13:14.955 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f880303a700 1 Processor -- start 2024-02-13T03:13:14.956 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f880303a700 1 -- start start 2024-02-13T03:13:14.956 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f880303a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f87fc0fc220 con 0x7f87fc06fc70 2024-02-13T03:13:14.956 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f8800dd6700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f87fc06fc70 0x7f87fc100830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34680/0 (socket says 172.21.15.161:34680) 2024-02-13T03:13:14.956 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f8800dd6700 1 -- 172.21.15.161:0/4185304937 learned_addr learned my addr 172.21.15.161:0/4185304937 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:14.956 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f87fb7fe700 1 -- 172.21.15.161:0/4185304937 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441916635 0 0) 0x7f87fc0fc220 con 0x7f87fc06fc70 2024-02-13T03:13:14.956 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f87fb7fe700 1 -- 172.21.15.161:0/4185304937 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87e80033f0 con 0x7f87fc06fc70 2024-02-13T03:13:14.957 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f87fb7fe700 1 -- 172.21.15.161:0/4185304937 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 80016696 0 0) 0x7f87e80033f0 con 0x7f87fc06fc70 2024-02-13T03:13:14.957 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f87fb7fe700 1 -- 172.21.15.161:0/4185304937 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc102f60 con 0x7f87fc06fc70 2024-02-13T03:13:14.957 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f87fb7fe700 1 -- 172.21.15.161:0/4185304937 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f87ec001590 con 0x7f87fc06fc70 2024-02-13T03:13:14.957 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f87fb7fe700 1 -- 172.21.15.161:0/4185304937 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f87ec001c30 con 0x7f87fc06fc70 2024-02-13T03:13:14.957 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f87fb7fe700 1 -- 172.21.15.161:0/4185304937 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f87ec0055d0 con 0x7f87fc06fc70 2024-02-13T03:13:14.960 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f880303a700 1 -- 172.21.15.161:0/4185304937 >> v1:172.21.15.161:6789/0 conn(0x7f87fc06fc70 legacy=0x7f87fc100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f880303a700 1 -- 172.21.15.161:0/4185304937 shutdown_connections 2024-02-13T03:13:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f880303a700 1 -- 172.21.15.161:0/4185304937 >> 172.21.15.161:0/4185304937 conn(0x7f87fc0f5e30 msgr2=0x7f87fc0f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f880303a700 1 -- 172.21.15.161:0/4185304937 shutdown_connections 2024-02-13T03:13:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.950+0000 7f880303a700 1 -- 172.21.15.161:0/4185304937 wait complete. 2024-02-13T03:13:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f880303a700 1 Processor -- start 2024-02-13T03:13:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f880303a700 1 -- start start 2024-02-13T03:13:14.961 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f880303a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f87fc10ca80 con 0x7f87fc06fc70 2024-02-13T03:13:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f8800dd6700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f87fc06fc70 0x7f87fc10c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34696/0 (socket says 172.21.15.161:34696) 2024-02-13T03:13:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f8800dd6700 1 -- 172.21.15.161:0/2827512459 learned_addr learned my addr 172.21.15.161:0/2827512459 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f87f9ffb700 1 -- 172.21.15.161:0/2827512459 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144941590 0 0) 0x7f87fc10ca80 con 0x7f87fc06fc70 2024-02-13T03:13:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f87f9ffb700 1 -- 172.21.15.161:0/2827512459 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87e8005260 con 0x7f87fc06fc70 2024-02-13T03:13:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f87f9ffb700 1 -- 172.21.15.161:0/2827512459 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3547735658 0 0) 0x7f87e8005260 con 0x7f87fc06fc70 2024-02-13T03:13:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f87f9ffb700 1 -- 172.21.15.161:0/2827512459 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87fc10ca80 con 0x7f87fc06fc70 2024-02-13T03:13:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f87f9ffb700 1 -- 172.21.15.161:0/2827512459 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f87ec000ed0 con 0x7f87fc06fc70 2024-02-13T03:13:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f87f9ffb700 1 -- 172.21.15.161:0/2827512459 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2620595812 0 0) 0x7f87fc10ca80 con 0x7f87fc06fc70 2024-02-13T03:13:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f87f9ffb700 1 -- 172.21.15.161:0/2827512459 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc10cc10 con 0x7f87fc06fc70 2024-02-13T03:13:14.962 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f880303a700 1 -- 172.21.15.161:0/2827512459 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87fc10cec0 con 0x7f87fc06fc70 2024-02-13T03:13:14.963 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f880303a700 1 -- 172.21.15.161:0/2827512459 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc110bb0 con 0x7f87fc06fc70 2024-02-13T03:13:14.964 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f87f9ffb700 1 -- 172.21.15.161:0/2827512459 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f87ec0016b0 con 0x7f87fc06fc70 2024-02-13T03:13:14.964 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f87f9ffb700 1 -- 172.21.15.161:0/2827512459 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f87ec006200 con 0x7f87fc06fc70 2024-02-13T03:13:14.964 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f87f9ffb700 1 -- 172.21.15.161:0/2827512459 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 1) v1 ==== 495+0+0 (unknown 2835230805 0 0) 0x7f87ec0066a0 con 0x7f87fc06fc70 2024-02-13T03:13:14.964 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.954+0000 7f87f9ffb700 1 -- 172.21.15.161:0/2827512459 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 1315227326 0 0) 0x7f87ec007600 con 0x7f87fc06fc70 2024-02-13T03:13:14.964 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.958+0000 7f880303a700 1 -- 172.21.15.161:0/2827512459 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc004f50 con 0x7f87fc06fc70 2024-02-13T03:13:14.967 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:14.962+0000 7f87f9ffb700 1 -- 172.21.15.161:0/2827512459 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (unknown 1092875540 0 259193764) 0x7f87ec0058c0 con 0x7f87fc06fc70 2024-02-13T03:13:15.101 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:15.094+0000 7f880303a700 1 -- 172.21.15.161:0/2827512459 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f87dc005200 con 0x7f87fc06fc70 2024-02-13T03:13:15.101 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:15.098+0000 7f87f9ffb700 1 -- 172.21.15.161:0/2827512459 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (unknown 4201413639 0 1702239654) 0x7f87ec001820 con 0x7f87fc06fc70 2024-02-13T03:13:15.102 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:15.102 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: { 2024-02-13T03:13:15.102 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "fsid": "b9b3ef98-ca1d-11ee-95b9-87774f69a715", 2024-02-13T03:13:15.103 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "health": { 2024-02-13T03:13:15.103 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-13T03:13:15.103 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "checks": {}, 2024-02-13T03:13:15.103 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "mutes": [] 2024-02-13T03:13:15.103 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:15.103 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-13T03:13:15.108 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "quorum": [ 2024-02-13T03:13:15.108 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 0 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: ], 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "a" 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: ], 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "quorum_age": 3, 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "monmap": { 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "osdmap": { 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-13T03:13:15.109 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-13T03:13:15.110 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-13T03:13:15.110 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-13T03:13:15.110 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-13T03:13:15.110 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-13T03:13:15.110 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:15.110 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "pgmap": { 2024-02-13T03:13:15.110 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-13T03:13:15.110 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-13T03:13:15.110 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-13T03:13:15.110 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-13T03:13:15.110 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-13T03:13:15.110 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-13T03:13:15.110 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "fsmap": { 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "available": false, 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "modules": [ 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "iostat", 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "nfs", 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "restful" 2024-02-13T03:13:15.111 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: ], 2024-02-13T03:13:15.112 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "services": {} 2024-02-13T03:13:15.112 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:15.112 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "servicemap": { 2024-02-13T03:13:15.112 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-13T03:13:15.112 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "modified": "2024-02-13T03:13:05.823965+0000", 2024-02-13T03:13:15.112 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "services": {} 2024-02-13T03:13:15.112 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:15.112 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-13T03:13:15.112 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: } 2024-02-13T03:13:15.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:15.098+0000 7f880303a700 1 -- 172.21.15.161:0/2827512459 >> v1:172.21.15.161:6789/0 conn(0x7f87fc06fc70 legacy=0x7f87fc10c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:15.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:15.098+0000 7f880303a700 1 -- 172.21.15.161:0/2827512459 shutdown_connections 2024-02-13T03:13:15.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:15.098+0000 7f880303a700 1 -- 172.21.15.161:0/2827512459 >> 172.21.15.161:0/2827512459 conn(0x7f87fc0f5e30 msgr2=0x7f87fc0f6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:15.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:15.102+0000 7f880303a700 1 -- 172.21.15.161:0/2827512459 shutdown_connections 2024-02-13T03:13:15.116 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:15.102+0000 7f880303a700 1 -- 172.21.15.161:0/2827512459 wait complete. 2024-02-13T03:13:15.424 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:15 smithi161 bash[19769]: audit 2024-02-13T03:13:15.101496+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? v1:172.21.15.161:0/2827512459' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-13T03:13:15.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:15 smithi161 bash[20075]: debug 2024-02-13T03:13:15.014+0000 7f2aeede9680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T03:13:15.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:15 smithi161 bash[20075]: debug 2024-02-13T03:13:15.122+0000 7f2aeede9680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T03:13:15.673 INFO:teuthology.orchestra.run.smithi161.stderr:mgr not available, waiting (1/15)... 2024-02-13T03:13:15.924 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:15 smithi161 bash[20075]: debug 2024-02-13T03:13:15.562+0000 7f2aeede9680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T03:13:16.785 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:16 smithi161 bash[20075]: debug 2024-02-13T03:13:16.454+0000 7f2aeede9680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T03:13:16.785 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:16 smithi161 bash[20075]: debug 2024-02-13T03:13:16.542+0000 7f2aeede9680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T03:13:17.174 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:16 smithi161 bash[20075]: debug 2024-02-13T03:13:16.778+0000 7f2aeede9680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T03:13:18.131 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:17 smithi161 bash[20075]: debug 2024-02-13T03:13:17.874+0000 7f2aeede9680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T03:13:18.131 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:18 smithi161 bash[20075]: debug 2024-02-13T03:13:18.030+0000 7f2aeede9680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T03:13:18.424 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:18 smithi161 bash[20075]: debug 2024-02-13T03:13:18.126+0000 7f2aeede9680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T03:13:18.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:18 smithi161 bash[20075]: debug 2024-02-13T03:13:18.234+0000 7f2aeede9680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T03:13:18.826 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:18 smithi161 bash[20075]: debug 2024-02-13T03:13:18.454+0000 7f2aeede9680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T03:13:18.870 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.862+0000 7f081881d700 1 Processor -- start 2024-02-13T03:13:18.870 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.862+0000 7f081881d700 1 -- start start 2024-02-13T03:13:18.870 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.862+0000 7f081881d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f08140720e0 con 0x7f0814071cc0 2024-02-13T03:13:18.870 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.862+0000 7f081259c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0814071cc0 0x7f0814070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53590/0 (socket says 172.21.15.161:53590) 2024-02-13T03:13:18.870 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.862+0000 7f081259c700 1 -- 172.21.15.161:0/350307628 learned_addr learned my addr 172.21.15.161:0/350307628 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:18.870 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081159a700 1 -- 172.21.15.161:0/350307628 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707941790 0 0) 0x7f08140720e0 con 0x7f0814071cc0 2024-02-13T03:13:18.870 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081159a700 1 -- 172.21.15.161:0/350307628 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07fc0033f0 con 0x7f0814071cc0 2024-02-13T03:13:18.870 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081159a700 1 -- 172.21.15.161:0/350307628 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4029312076 0 0) 0x7f07fc0033f0 con 0x7f0814071cc0 2024-02-13T03:13:18.871 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081159a700 1 -- 172.21.15.161:0/350307628 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0814070970 con 0x7f0814071cc0 2024-02-13T03:13:18.871 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081159a700 1 -- 172.21.15.161:0/350307628 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f0804001590 con 0x7f0814071cc0 2024-02-13T03:13:18.872 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081159a700 1 -- 172.21.15.161:0/350307628 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f0804001c30 con 0x7f0814071cc0 2024-02-13T03:13:18.872 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081159a700 1 -- 172.21.15.161:0/350307628 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f0804005630 con 0x7f0814071cc0 2024-02-13T03:13:18.872 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081881d700 1 -- 172.21.15.161:0/350307628 >> v1:172.21.15.161:6789/0 conn(0x7f0814071cc0 legacy=0x7f0814070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:18.872 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081881d700 1 -- 172.21.15.161:0/350307628 shutdown_connections 2024-02-13T03:13:18.872 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081881d700 1 -- 172.21.15.161:0/350307628 >> 172.21.15.161:0/350307628 conn(0x7f08140f6f20 msgr2=0x7f08140f9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:18.872 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081881d700 1 -- 172.21.15.161:0/350307628 shutdown_connections 2024-02-13T03:13:18.872 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081881d700 1 -- 172.21.15.161:0/350307628 wait complete. 2024-02-13T03:13:18.874 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081881d700 1 Processor -- start 2024-02-13T03:13:18.874 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081881d700 1 -- start start 2024-02-13T03:13:18.874 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.866+0000 7f081881d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f081410c9f0 con 0x7f0814071cc0 2024-02-13T03:13:18.874 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f081259c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0814071cc0 0x7f081410c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53600/0 (socket says 172.21.15.161:53600) 2024-02-13T03:13:18.874 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f081259c700 1 -- 172.21.15.161:0/1312342731 learned_addr learned my addr 172.21.15.161:0/1312342731 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:18.874 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f08037fe700 1 -- 172.21.15.161:0/1312342731 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 730356418 0 0) 0x7f081410c9f0 con 0x7f0814071cc0 2024-02-13T03:13:18.874 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f08037fe700 1 -- 172.21.15.161:0/1312342731 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07fc005260 con 0x7f0814071cc0 2024-02-13T03:13:18.874 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f08037fe700 1 -- 172.21.15.161:0/1312342731 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3194570781 0 0) 0x7f07fc005260 con 0x7f0814071cc0 2024-02-13T03:13:18.874 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f08037fe700 1 -- 172.21.15.161:0/1312342731 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f081410c9f0 con 0x7f0814071cc0 2024-02-13T03:13:18.874 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f08037fe700 1 -- 172.21.15.161:0/1312342731 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f0804000ed0 con 0x7f0814071cc0 2024-02-13T03:13:18.875 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f08037fe700 1 -- 172.21.15.161:0/1312342731 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 475548478 0 0) 0x7f081410c9f0 con 0x7f0814071cc0 2024-02-13T03:13:18.875 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f08037fe700 1 -- 172.21.15.161:0/1312342731 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081410cb80 con 0x7f0814071cc0 2024-02-13T03:13:18.875 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f081881d700 1 -- 172.21.15.161:0/1312342731 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081410ce30 con 0x7f0814071cc0 2024-02-13T03:13:18.875 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f081881d700 1 -- 172.21.15.161:0/1312342731 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0814110b20 con 0x7f0814071cc0 2024-02-13T03:13:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f08037fe700 1 -- 172.21.15.161:0/1312342731 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f0804004a80 con 0x7f0814071cc0 2024-02-13T03:13:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f08037fe700 1 -- 172.21.15.161:0/1312342731 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f08040060e0 con 0x7f0814071cc0 2024-02-13T03:13:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f08037fe700 1 -- 172.21.15.161:0/1312342731 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 1) v1 ==== 495+0+0 (unknown 2835230805 0 0) 0x7f0804006560 con 0x7f0814071cc0 2024-02-13T03:13:18.876 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f08037fe700 1 -- 172.21.15.161:0/1312342731 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 1315227326 0 0) 0x7f08040074c0 con 0x7f0814071cc0 2024-02-13T03:13:18.877 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.870+0000 7f081881d700 1 -- 172.21.15.161:0/1312342731 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0814046d80 con 0x7f0814071cc0 2024-02-13T03:13:18.879 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:18.874+0000 7f08037fe700 1 -- 172.21.15.161:0/1312342731 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (unknown 1092875540 0 259193764) 0x7f0804005920 con 0x7f0814071cc0 2024-02-13T03:13:19.014 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:19.010+0000 7f081881d700 1 -- 172.21.15.161:0/1312342731 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0814102bb0 con 0x7f0814071cc0 2024-02-13T03:13:19.017 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:19.010+0000 7f08037fe700 1 -- 172.21.15.161:0/1312342731 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (unknown 4201413639 0 415313620) 0x7f0804005b80 con 0x7f0814071cc0 2024-02-13T03:13:19.017 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:19.017 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: { 2024-02-13T03:13:19.017 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "fsid": "b9b3ef98-ca1d-11ee-95b9-87774f69a715", 2024-02-13T03:13:19.017 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "health": { 2024-02-13T03:13:19.017 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-13T03:13:19.017 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "checks": {}, 2024-02-13T03:13:19.017 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "mutes": [] 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "quorum": [ 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 0 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: ], 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "a" 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: ], 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "quorum_age": 6, 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "monmap": { 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-13T03:13:19.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:19.023 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "osdmap": { 2024-02-13T03:13:19.023 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-13T03:13:19.023 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-13T03:13:19.023 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-13T03:13:19.023 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-13T03:13:19.023 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-13T03:13:19.024 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-13T03:13:19.024 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-13T03:13:19.024 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:19.024 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "pgmap": { 2024-02-13T03:13:19.024 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-13T03:13:19.024 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-13T03:13:19.024 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-13T03:13:19.024 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-13T03:13:19.024 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-13T03:13:19.024 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-13T03:13:19.024 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-13T03:13:19.024 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "fsmap": { 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "available": false, 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "modules": [ 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "iostat", 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "nfs", 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "restful" 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: ], 2024-02-13T03:13:19.025 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "services": {} 2024-02-13T03:13:19.026 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:19.026 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "servicemap": { 2024-02-13T03:13:19.026 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-13T03:13:19.026 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "modified": "2024-02-13T03:13:05.823965+0000", 2024-02-13T03:13:19.026 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "services": {} 2024-02-13T03:13:19.026 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:19.026 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-13T03:13:19.026 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: } 2024-02-13T03:13:19.026 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:19.014+0000 7f081881d700 1 -- 172.21.15.161:0/1312342731 >> v1:172.21.15.161:6789/0 conn(0x7f0814071cc0 legacy=0x7f081410c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:19.026 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:19.014+0000 7f081881d700 1 -- 172.21.15.161:0/1312342731 shutdown_connections 2024-02-13T03:13:19.026 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:19.014+0000 7f081881d700 1 -- 172.21.15.161:0/1312342731 >> 172.21.15.161:0/1312342731 conn(0x7f08140f6f20 msgr2=0x7f08140fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:19.026 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:19.014+0000 7f081881d700 1 -- 172.21.15.161:0/1312342731 shutdown_connections 2024-02-13T03:13:19.026 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:19.014+0000 7f081881d700 1 -- 172.21.15.161:0/1312342731 wait complete. 2024-02-13T03:13:19.174 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:19 smithi161 bash[19769]: audit 2024-02-13T03:13:19.014899+0000 mon.a (mon.0) 8 : audit [DBG] from='client.? v1:172.21.15.161:0/1312342731' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-13T03:13:19.175 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:18 smithi161 bash[20075]: debug 2024-02-13T03:13:18.822+0000 7f2aeede9680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T03:13:19.175 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:18 smithi161 bash[20075]: debug 2024-02-13T03:13:18.938+0000 7f2aeede9680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T03:13:19.547 INFO:teuthology.orchestra.run.smithi161.stderr:mgr not available, waiting (2/15)... 2024-02-13T03:13:20.174 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:19 smithi161 bash[20075]: debug 2024-02-13T03:13:19.790+0000 7f2aeede9680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T03:13:20.175 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:19 smithi161 bash[20075]: debug 2024-02-13T03:13:19.894+0000 7f2aeede9680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T03:13:20.175 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:20 smithi161 bash[20075]: debug 2024-02-13T03:13:19.998+0000 7f2aeede9680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T03:13:20.498 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:20 smithi161 bash[20075]: debug 2024-02-13T03:13:20.206+0000 7f2aeede9680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T03:13:20.822 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:20 smithi161 bash[20075]: debug 2024-02-13T03:13:20.494+0000 7f2aeede9680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T03:13:20.822 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:20 smithi161 bash[20075]: debug 2024-02-13T03:13:20.650+0000 7f2aeede9680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T03:13:21.134 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:20 smithi161 bash[20075]: debug 2024-02-13T03:13:20.818+0000 7f2aeede9680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T03:13:21.134 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[20075]: debug 2024-02-13T03:13:21.026+0000 7f2aeede9680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T03:13:21.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[20075]: debug 2024-02-13T03:13:21.130+0000 7f2aeede9680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T03:13:21.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: cluster 2024-02-13T03:13:21.155307+0000 mon.a (mon.0) 9 : cluster [INF] Activating manager daemon y 2024-02-13T03:13:21.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: cluster 2024-02-13T03:13:21.159447+0000 mon.a (mon.0) 10 : cluster [DBG] mgrmap e2: y(active, starting, since 0.00432407s) 2024-02-13T03:13:21.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: audit 2024-02-13T03:13:21.162684+0000 mon.a (mon.0) 11 : audit [DBG] from='mgr.14100 v1:172.21.15.161:0/852488846' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T03:13:21.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: audit 2024-02-13T03:13:21.163147+0000 mon.a (mon.0) 12 : audit [DBG] from='mgr.14100 v1:172.21.15.161:0/852488846' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T03:13:21.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: audit 2024-02-13T03:13:21.163536+0000 mon.a (mon.0) 13 : audit [DBG] from='mgr.14100 v1:172.21.15.161:0/852488846' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T03:13:21.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: audit 2024-02-13T03:13:21.163902+0000 mon.a (mon.0) 14 : audit [DBG] from='mgr.14100 v1:172.21.15.161:0/852488846' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T03:13:21.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: audit 2024-02-13T03:13:21.164295+0000 mon.a (mon.0) 15 : audit [DBG] from='mgr.14100 v1:172.21.15.161:0/852488846' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T03:13:21.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: audit 2024-02-13T03:13:21.164677+0000 mon.a (mon.0) 16 : audit [DBG] from='mgr.14100 v1:172.21.15.161:0/852488846' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T03:13:21.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: audit 2024-02-13T03:13:21.165050+0000 mon.a (mon.0) 17 : audit [DBG] from='mgr.14100 v1:172.21.15.161:0/852488846' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T03:13:21.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: audit 2024-02-13T03:13:21.165457+0000 mon.a (mon.0) 18 : audit [DBG] from='mgr.14100 v1:172.21.15.161:0/852488846' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T03:13:21.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: cluster 2024-02-13T03:13:21.168666+0000 mon.a (mon.0) 19 : cluster [INF] Manager daemon y is now available 2024-02-13T03:13:21.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: audit 2024-02-13T03:13:21.188175+0000 mon.a (mon.0) 20 : audit [INF] from='mgr.14100 v1:172.21.15.161:0/852488846' entity='mgr.y' 2024-02-13T03:13:21.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: audit 2024-02-13T03:13:21.195570+0000 mon.a (mon.0) 21 : audit [INF] from='mgr.14100 v1:172.21.15.161:0/852488846' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T03:13:21.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:21 smithi161 bash[19769]: audit 2024-02-13T03:13:21.200657+0000 mon.a (mon.0) 22 : audit [INF] from='mgr.14100 v1:172.21.15.161:0/852488846' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T03:13:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.650+0000 7fc7701e2700 1 Processor -- start 2024-02-13T03:13:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.650+0000 7fc7701e2700 1 -- start start 2024-02-13T03:13:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.650+0000 7fc7701e2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc7681062d0 con 0x7fc768103410 2024-02-13T03:13:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.650+0000 7fc76df7e700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc768103410 0x7fc768103830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53654/0 (socket says 172.21.15.161:53654) 2024-02-13T03:13:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.650+0000 7fc76df7e700 1 -- 172.21.15.161:0/3341401049 learned_addr learned my addr 172.21.15.161:0/3341401049 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.650+0000 7fc76cf7c700 1 -- 172.21.15.161:0/3341401049 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043245324 0 0) 0x7fc7681062d0 con 0x7fc768103410 2024-02-13T03:13:22.655 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.650+0000 7fc76cf7c700 1 -- 172.21.15.161:0/3341401049 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7540033f0 con 0x7fc768103410 2024-02-13T03:13:22.656 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.650+0000 7fc76cf7c700 1 -- 172.21.15.161:0/3341401049 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3837998891 0 0) 0x7fc7540033f0 con 0x7fc768103410 2024-02-13T03:13:22.656 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.650+0000 7fc76cf7c700 1 -- 172.21.15.161:0/3341401049 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc768107470 con 0x7fc768103410 2024-02-13T03:13:22.656 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.650+0000 7fc76cf7c700 1 -- 172.21.15.161:0/3341401049 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fc764001590 con 0x7fc768103410 2024-02-13T03:13:22.656 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.650+0000 7fc76cf7c700 1 -- 172.21.15.161:0/3341401049 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7fc764001c30 con 0x7fc768103410 2024-02-13T03:13:22.656 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.650+0000 7fc76cf7c700 1 -- 172.21.15.161:0/3341401049 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fc764005630 con 0x7fc768103410 2024-02-13T03:13:22.657 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.650+0000 7fc7701e2700 1 -- 172.21.15.161:0/3341401049 >> v1:172.21.15.161:6789/0 conn(0x7fc768103410 legacy=0x7fc768103830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:22.657 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc7701e2700 1 -- 172.21.15.161:0/3341401049 shutdown_connections 2024-02-13T03:13:22.658 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc7701e2700 1 -- 172.21.15.161:0/3341401049 >> 172.21.15.161:0/3341401049 conn(0x7fc768071060 msgr2=0x7fc7680734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:22.658 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc7701e2700 1 -- 172.21.15.161:0/3341401049 shutdown_connections 2024-02-13T03:13:22.658 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc7701e2700 1 -- 172.21.15.161:0/3341401049 wait complete. 2024-02-13T03:13:22.658 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc7701e2700 1 Processor -- start 2024-02-13T03:13:22.659 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc7701e2700 1 -- start start 2024-02-13T03:13:22.659 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc7701e2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc768110f80 con 0x7fc768103410 2024-02-13T03:13:22.659 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc76df7e700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc768103410 0x7fc768110870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53656/0 (socket says 172.21.15.161:53656) 2024-02-13T03:13:22.659 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc76df7e700 1 -- 172.21.15.161:0/4033805757 learned_addr learned my addr 172.21.15.161:0/4033805757 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:22.660 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc75effd700 1 -- 172.21.15.161:0/4033805757 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644623600 0 0) 0x7fc768110f80 con 0x7fc768103410 2024-02-13T03:13:22.660 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc75effd700 1 -- 172.21.15.161:0/4033805757 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc754005260 con 0x7fc768103410 2024-02-13T03:13:22.660 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc75effd700 1 -- 172.21.15.161:0/4033805757 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3686971014 0 0) 0x7fc754005260 con 0x7fc768103410 2024-02-13T03:13:22.660 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc75effd700 1 -- 172.21.15.161:0/4033805757 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc768110f80 con 0x7fc768103410 2024-02-13T03:13:22.661 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc75effd700 1 -- 172.21.15.161:0/4033805757 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fc764000ed0 con 0x7fc768103410 2024-02-13T03:13:22.661 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc75effd700 1 -- 172.21.15.161:0/4033805757 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3197052229 0 0) 0x7fc768110f80 con 0x7fc768103410 2024-02-13T03:13:22.661 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc75effd700 1 -- 172.21.15.161:0/4033805757 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc768111110 con 0x7fc768103410 2024-02-13T03:13:22.661 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc7701e2700 1 -- 172.21.15.161:0/4033805757 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7681113c0 con 0x7fc768103410 2024-02-13T03:13:22.661 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.654+0000 7fc7701e2700 1 -- 172.21.15.161:0/4033805757 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7681150b0 con 0x7fc768103410 2024-02-13T03:13:22.661 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.658+0000 7fc75effd700 1 -- 172.21.15.161:0/4033805757 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7fc764004a20 con 0x7fc768103410 2024-02-13T03:13:22.661 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.658+0000 7fc75effd700 1 -- 172.21.15.161:0/4033805757 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fc764006100 con 0x7fc768103410 2024-02-13T03:13:22.662 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.658+0000 7fc75effd700 1 -- 172.21.15.161:0/4033805757 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 3) v1 ==== 38465+0+0 (unknown 1191238990 0 0) 0x7fc76400f970 con 0x7fc768103410 2024-02-13T03:13:22.662 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.658+0000 7fc75effd700 1 -- 172.21.15.161:0/4033805757 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 1315227326 0 0) 0x7fc76403f870 con 0x7fc768103410 2024-02-13T03:13:22.662 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.658+0000 7fc7701e2700 1 -- 172.21.15.161:0/4033805757 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74c004f50 con 0x7fc768103410 2024-02-13T03:13:22.668 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:22.662+0000 7fc75effd700 1 -- 172.21.15.161:0/4033805757 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fc764011050 con 0x7fc768103410 2024-02-13T03:13:22.674 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:22 smithi161 bash[19769]: audit 2024-02-13T03:13:21.209240+0000 mon.a (mon.0) 23 : audit [INF] from='mgr.14100 v1:172.21.15.161:0/852488846' entity='mgr.y' 2024-02-13T03:13:22.674 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:22 smithi161 bash[19769]: audit 2024-02-13T03:13:21.218548+0000 mon.a (mon.0) 24 : audit [INF] from='mgr.14100 v1:172.21.15.161:0/852488846' entity='mgr.y' 2024-02-13T03:13:22.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:22 smithi161 bash[19769]: cluster 2024-02-13T03:13:22.163528+0000 mon.a (mon.0) 25 : cluster [DBG] mgrmap e3: y(active, since 1.00841s) 2024-02-13T03:13:23.121 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:23.114+0000 7fc7701e2700 1 -- 172.21.15.161:0/4033805757 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc74c005200 con 0x7fc768103410 2024-02-13T03:13:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:23.118+0000 7fc75effd700 1 -- 172.21.15.161:0/4033805757 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (unknown 4201413639 0 971687290) 0x7fc7640147e0 con 0x7fc768103410 2024-02-13T03:13:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: { 2024-02-13T03:13:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "fsid": "b9b3ef98-ca1d-11ee-95b9-87774f69a715", 2024-02-13T03:13:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "health": { 2024-02-13T03:13:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-13T03:13:23.122 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "checks": {}, 2024-02-13T03:13:23.124 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "mutes": [] 2024-02-13T03:13:23.124 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:23.124 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-13T03:13:23.124 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "quorum": [ 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 0 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: ], 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "a" 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: ], 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "quorum_age": 11, 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "monmap": { 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "osdmap": { 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-13T03:13:23.125 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-13T03:13:23.127 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-13T03:13:23.128 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-13T03:13:23.128 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-13T03:13:23.128 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-13T03:13:23.128 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-13T03:13:23.128 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:23.128 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "pgmap": { 2024-02-13T03:13:23.128 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-13T03:13:23.128 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-13T03:13:23.128 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-13T03:13:23.128 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-13T03:13:23.128 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-13T03:13:23.128 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-13T03:13:23.129 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-13T03:13:23.129 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-13T03:13:23.129 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:23.129 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "fsmap": { 2024-02-13T03:13:23.129 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-13T03:13:23.129 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-13T03:13:23.129 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-13T03:13:23.131 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:23.131 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-13T03:13:23.131 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "available": true, 2024-02-13T03:13:23.131 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "modules": [ 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "iostat", 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "nfs", 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "restful" 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: ], 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "services": {} 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "servicemap": { 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "modified": "2024-02-13T03:13:05.823965+0000", 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "services": {} 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: }, 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-13T03:13:23.132 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: } 2024-02-13T03:13:23.133 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:23.122+0000 7fc7701e2700 1 -- 172.21.15.161:0/4033805757 >> v1:172.21.15.161:6800/1159674441 conn(0x7fc7540336f0 legacy=0x7fc754035bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:23.133 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:23.122+0000 7fc7701e2700 1 -- 172.21.15.161:0/4033805757 >> v1:172.21.15.161:6789/0 conn(0x7fc768103410 legacy=0x7fc768110870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:23.133 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:23.122+0000 7fc7701e2700 1 -- 172.21.15.161:0/4033805757 shutdown_connections 2024-02-13T03:13:23.133 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:23.122+0000 7fc7701e2700 1 -- 172.21.15.161:0/4033805757 >> 172.21.15.161:0/4033805757 conn(0x7fc768071060 msgr2=0x7fc768072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:23.133 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:23.122+0000 7fc7701e2700 1 -- 172.21.15.161:0/4033805757 shutdown_connections 2024-02-13T03:13:23.133 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:23.122+0000 7fc7701e2700 1 -- 172.21.15.161:0/4033805757 wait complete. 2024-02-13T03:13:23.643 INFO:teuthology.orchestra.run.smithi161.stderr:mgr is available 2024-02-13T03:13:23.674 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:23 smithi161 bash[19769]: audit 2024-02-13T03:13:23.121455+0000 mon.a (mon.0) 26 : audit [DBG] from='client.? v1:172.21.15.161:0/4033805757' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-13T03:13:24.606 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f76452b0700 1 Processor -- start 2024-02-13T03:13:24.606 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f76452b0700 1 -- start start 2024-02-13T03:13:24.606 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f76452b0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f76400fa630 con 0x7f76400f9b00 2024-02-13T03:13:24.606 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f763effd700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f76400f9b00 0x7f76400f9f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53660/0 (socket says 172.21.15.161:53660) 2024-02-13T03:13:24.606 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f763effd700 1 -- 172.21.15.161:0/1564840582 learned_addr learned my addr 172.21.15.161:0/1564840582 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:24.607 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f763dffb700 1 -- 172.21.15.161:0/1564840582 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947707412 0 0) 0x7f76400fa630 con 0x7f76400f9b00 2024-02-13T03:13:24.607 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f763dffb700 1 -- 172.21.15.161:0/1564840582 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76280033f0 con 0x7f76400f9b00 2024-02-13T03:13:24.607 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f763dffb700 1 -- 172.21.15.161:0/1564840582 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3205653735 0 0) 0x7f76280033f0 con 0x7f76400f9b00 2024-02-13T03:13:24.608 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f763dffb700 1 -- 172.21.15.161:0/1564840582 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76400fa7c0 con 0x7f76400f9b00 2024-02-13T03:13:24.608 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f763dffb700 1 -- 172.21.15.161:0/1564840582 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f7630001590 con 0x7f76400f9b00 2024-02-13T03:13:24.608 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f763dffb700 1 -- 172.21.15.161:0/1564840582 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f7630001d00 con 0x7f76400f9b00 2024-02-13T03:13:24.608 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f763dffb700 1 -- 172.21.15.161:0/1564840582 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f7630005700 con 0x7f76400f9b00 2024-02-13T03:13:24.609 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f76452b0700 1 -- 172.21.15.161:0/1564840582 >> v1:172.21.15.161:6789/0 conn(0x7f76400f9b00 legacy=0x7f76400f9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:24.609 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f76452b0700 1 -- 172.21.15.161:0/1564840582 shutdown_connections 2024-02-13T03:13:24.609 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f76452b0700 1 -- 172.21.15.161:0/1564840582 >> 172.21.15.161:0/1564840582 conn(0x7f76400f55c0 msgr2=0x7f76400f79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:24.609 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f76452b0700 1 -- 172.21.15.161:0/1564840582 shutdown_connections 2024-02-13T03:13:24.609 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f76452b0700 1 -- 172.21.15.161:0/1564840582 wait complete. 2024-02-13T03:13:24.610 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.602+0000 7f76452b0700 1 Processor -- start 2024-02-13T03:13:24.610 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f76452b0700 1 -- start start 2024-02-13T03:13:24.610 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f76452b0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f764010ca50 con 0x7f76400f9b00 2024-02-13T03:13:24.610 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f763effd700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f76400f9b00 0x7f764010c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53662/0 (socket says 172.21.15.161:53662) 2024-02-13T03:13:24.610 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f763effd700 1 -- 172.21.15.161:0/1218694407 learned_addr learned my addr 172.21.15.161:0/1218694407 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:24.610 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f7627fff700 1 -- 172.21.15.161:0/1218694407 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74939094 0 0) 0x7f764010ca50 con 0x7f76400f9b00 2024-02-13T03:13:24.610 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f7627fff700 1 -- 172.21.15.161:0/1218694407 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7628005210 con 0x7f76400f9b00 2024-02-13T03:13:24.611 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f7627fff700 1 -- 172.21.15.161:0/1218694407 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2894594111 0 0) 0x7f7628005210 con 0x7f76400f9b00 2024-02-13T03:13:24.612 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f7627fff700 1 -- 172.21.15.161:0/1218694407 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f764010ca50 con 0x7f76400f9b00 2024-02-13T03:13:24.612 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f7627fff700 1 -- 172.21.15.161:0/1218694407 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f7630004a70 con 0x7f76400f9b00 2024-02-13T03:13:24.612 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f7627fff700 1 -- 172.21.15.161:0/1218694407 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4149071976 0 0) 0x7f764010ca50 con 0x7f76400f9b00 2024-02-13T03:13:24.612 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f7627fff700 1 -- 172.21.15.161:0/1218694407 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764010cbe0 con 0x7f76400f9b00 2024-02-13T03:13:24.612 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f76452b0700 1 -- 172.21.15.161:0/1218694407 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764010ce90 con 0x7f76400f9b00 2024-02-13T03:13:24.612 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f76452b0700 1 -- 172.21.15.161:0/1218694407 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7640110b80 con 0x7f76400f9b00 2024-02-13T03:13:24.612 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f7627fff700 1 -- 172.21.15.161:0/1218694407 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f7630001100 con 0x7f76400f9b00 2024-02-13T03:13:24.612 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f7627fff700 1 -- 172.21.15.161:0/1218694407 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f76300060d0 con 0x7f76400f9b00 2024-02-13T03:13:24.613 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f7627fff700 1 -- 172.21.15.161:0/1218694407 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 4) v1 ==== 38571+0+0 (unknown 2252295444 0 0) 0x7f763000f9d0 con 0x7f76400f9b00 2024-02-13T03:13:24.613 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.606+0000 7f7627fff700 1 -- 172.21.15.161:0/1218694407 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 1315227326 0 0) 0x7f763003ee10 con 0x7f76400f9b00 2024-02-13T03:13:24.613 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.610+0000 7f76452b0700 1 -- 172.21.15.161:0/1218694407 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f762c004f50 con 0x7f76400f9b00 2024-02-13T03:13:24.619 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.614+0000 7f7627fff700 1 -- 172.21.15.161:0/1218694407 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f76300110b0 con 0x7f76400f9b00 2024-02-13T03:13:24.674 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:24 smithi161 bash[19769]: cluster 2024-02-13T03:13:23.220894+0000 mon.a (mon.0) 27 : cluster [DBG] mgrmap e4: y(active, since 2s) 2024-02-13T03:13:24.941 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.938+0000 7f76452b0700 1 -- 172.21.15.161:0/1218694407 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f762c005780 con 0x7f76400f9b00 2024-02-13T03:13:24.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.938+0000 7f7627fff700 1 -- 172.21.15.161:0/1218694407 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+307 (unknown 1187553405 0 2286747643) 0x7f76300143c0 con 0x7f76400f9b00 2024-02-13T03:13:24.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.944 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: [global] 2024-02-13T03:13:24.944 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: fsid = b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:13:24.944 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-13T03:13:24.944 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-13T03:13:24.945 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-13T03:13:24.945 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-13T03:13:24.945 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-13T03:13:24.945 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.945 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: [mgr] 2024-02-13T03:13:24.945 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-13T03:13:24.946 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.946 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: [osd] 2024-02-13T03:13:24.946 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-13T03:13:24.946 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-13T03:13:24.946 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.942+0000 7f76452b0700 1 -- 172.21.15.161:0/1218694407 >> v1:172.21.15.161:6800/1159674441 conn(0x7f7628033800 legacy=0x7f7628035cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:24.946 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.942+0000 7f76452b0700 1 -- 172.21.15.161:0/1218694407 >> v1:172.21.15.161:6789/0 conn(0x7f76400f9b00 legacy=0x7f764010c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:24.946 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.942+0000 7f76452b0700 1 -- 172.21.15.161:0/1218694407 shutdown_connections 2024-02-13T03:13:24.947 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.942+0000 7f76452b0700 1 -- 172.21.15.161:0/1218694407 >> 172.21.15.161:0/1218694407 conn(0x7f76400f55c0 msgr2=0x7f76400f79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:24.947 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.942+0000 7f76452b0700 1 -- 172.21.15.161:0/1218694407 shutdown_connections 2024-02-13T03:13:24.947 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:24.942+0000 7f76452b0700 1 -- 172.21.15.161:0/1218694407 wait complete. 2024-02-13T03:13:25.424 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:25 smithi161 bash[19769]: audit 2024-02-13T03:13:24.942142+0000 mon.a (mon.0) 28 : audit [INF] from='client.? v1:172.21.15.161:0/1218694407' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-13T03:13:25.542 INFO:teuthology.orchestra.run.smithi161.stderr:Enabling cephadm module... 2024-02-13T03:13:26.629 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.622+0000 7f0acb2a6700 1 Processor -- start 2024-02-13T03:13:26.629 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.622+0000 7f0acb2a6700 1 -- start start 2024-02-13T03:13:26.630 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0acb2a6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0ac40720e0 con 0x7f0ac4071cc0 2024-02-13T03:13:26.630 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0ac9042700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0ac4071cc0 0x7f0ac4070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53668/0 (socket says 172.21.15.161:53668) 2024-02-13T03:13:26.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0ac9042700 1 -- 172.21.15.161:0/187523606 learned_addr learned my addr 172.21.15.161:0/187523606 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:26.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0ac3fff700 1 -- 172.21.15.161:0/187523606 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384098684 0 0) 0x7f0ac40720e0 con 0x7f0ac4071cc0 2024-02-13T03:13:26.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0ac3fff700 1 -- 172.21.15.161:0/187523606 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aac0033f0 con 0x7f0ac4071cc0 2024-02-13T03:13:26.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0ac3fff700 1 -- 172.21.15.161:0/187523606 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 397398690 0 0) 0x7f0aac0033f0 con 0x7f0ac4071cc0 2024-02-13T03:13:26.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0ac3fff700 1 -- 172.21.15.161:0/187523606 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac4070970 con 0x7f0ac4071cc0 2024-02-13T03:13:26.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0ac3fff700 1 -- 172.21.15.161:0/187523606 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f0abc001910 con 0x7f0ac4071cc0 2024-02-13T03:13:26.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0ac3fff700 1 -- 172.21.15.161:0/187523606 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f0abc001fb0 con 0x7f0ac4071cc0 2024-02-13T03:13:26.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0ac3fff700 1 -- 172.21.15.161:0/187523606 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f0abc0059b0 con 0x7f0ac4071cc0 2024-02-13T03:13:26.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0acb2a6700 1 -- 172.21.15.161:0/187523606 >> v1:172.21.15.161:6789/0 conn(0x7f0ac4071cc0 legacy=0x7f0ac4070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:26.632 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0acb2a6700 1 -- 172.21.15.161:0/187523606 shutdown_connections 2024-02-13T03:13:26.632 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0acb2a6700 1 -- 172.21.15.161:0/187523606 >> 172.21.15.161:0/187523606 conn(0x7f0ac40f6f40 msgr2=0x7f0ac40f9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:26.633 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0acb2a6700 1 -- 172.21.15.161:0/187523606 shutdown_connections 2024-02-13T03:13:26.633 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0acb2a6700 1 -- 172.21.15.161:0/187523606 wait complete. 2024-02-13T03:13:26.633 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0acb2a6700 1 Processor -- start 2024-02-13T03:13:26.633 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.626+0000 7f0acb2a6700 1 -- start start 2024-02-13T03:13:26.633 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0acb2a6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0ac410ca40 con 0x7f0ac4071cc0 2024-02-13T03:13:26.634 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0ac9042700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0ac4071cc0 0x7f0ac410c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53680/0 (socket says 172.21.15.161:53680) 2024-02-13T03:13:26.634 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0ac9042700 1 -- 172.21.15.161:0/3095742026 learned_addr learned my addr 172.21.15.161:0/3095742026 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:26.634 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585103097 0 0) 0x7f0ac410ca40 con 0x7f0ac4071cc0 2024-02-13T03:13:26.634 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aac005260 con 0x7f0ac4071cc0 2024-02-13T03:13:26.636 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 543475307 0 0) 0x7f0aac005260 con 0x7f0ac4071cc0 2024-02-13T03:13:26.637 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0ac410ca40 con 0x7f0ac4071cc0 2024-02-13T03:13:26.637 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f0abc001250 con 0x7f0ac4071cc0 2024-02-13T03:13:26.637 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2803034376 0 0) 0x7f0ac410ca40 con 0x7f0ac4071cc0 2024-02-13T03:13:26.637 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac410cbd0 con 0x7f0ac4071cc0 2024-02-13T03:13:26.637 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0acb2a6700 1 -- 172.21.15.161:0/3095742026 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac410ce80 con 0x7f0ac4071cc0 2024-02-13T03:13:26.637 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f0abc004ce0 con 0x7f0ac4071cc0 2024-02-13T03:13:26.637 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f0abc006510 con 0x7f0ac4071cc0 2024-02-13T03:13:26.637 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0acb2a6700 1 -- 172.21.15.161:0/3095742026 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac4110b70 con 0x7f0ac4071cc0 2024-02-13T03:13:26.637 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 4) v1 ==== 38571+0+0 (unknown 2252295444 0 0) 0x7f0abc006670 con 0x7f0ac4071cc0 2024-02-13T03:13:26.637 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.630+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 1315227326 0 0) 0x7f0abc03ee50 con 0x7f0ac4071cc0 2024-02-13T03:13:26.638 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.634+0000 7f0acb2a6700 1 -- 172.21.15.161:0/3095742026 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac4046d80 con 0x7f0ac4071cc0 2024-02-13T03:13:26.644 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:26.638+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f0abc007d50 con 0x7f0ac4071cc0 2024-02-13T03:13:27.038 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:27.034+0000 7f0acb2a6700 1 -- 172.21.15.161:0/3095742026 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f0ac4110e20 con 0x7f0ac4071cc0 2024-02-13T03:13:27.093 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:27.086+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mgrmap(e 5) v1 ==== 38582+0+0 (unknown 3095178131 0 0) 0x7f0abc03e440 con 0x7f0ac4071cc0 2024-02-13T03:13:27.096 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:27.090+0000 7f0ac2777700 1 -- 172.21.15.161:0/3095742026 <== mon.0 v1:172.21.15.161:6789/0 11 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (unknown 2263024820 0 0) 0x7f0abc0082d0 con 0x7f0ac4071cc0 2024-02-13T03:13:27.099 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:27.094+0000 7f0acb2a6700 1 -- 172.21.15.161:0/3095742026 >> v1:172.21.15.161:6800/1159674441 conn(0x7f0aac0339a0 legacy=0x7f0aac035e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:27.100 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:27.094+0000 7f0acb2a6700 1 -- 172.21.15.161:0/3095742026 >> v1:172.21.15.161:6789/0 conn(0x7f0ac4071cc0 legacy=0x7f0ac410c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:27.100 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:27.094+0000 7f0acb2a6700 1 -- 172.21.15.161:0/3095742026 shutdown_connections 2024-02-13T03:13:27.100 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:27.094+0000 7f0acb2a6700 1 -- 172.21.15.161:0/3095742026 >> 172.21.15.161:0/3095742026 conn(0x7f0ac40f6f40 msgr2=0x7f0ac40fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:27.101 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:27.098+0000 7f0acb2a6700 1 -- 172.21.15.161:0/3095742026 shutdown_connections 2024-02-13T03:13:27.102 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:27.098+0000 7f0acb2a6700 1 -- 172.21.15.161:0/3095742026 wait complete. 2024-02-13T03:13:27.374 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:27 smithi161 bash[19769]: audit 2024-02-13T03:13:27.038618+0000 mon.a (mon.0) 29 : audit [INF] from='client.? v1:172.21.15.161:0/3095742026' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-13T03:13:27.375 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:27 smithi161 bash[20075]: ignoring --setuser ceph since I am not root 2024-02-13T03:13:27.375 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:27 smithi161 bash[20075]: ignoring --setgroup ceph since I am not root 2024-02-13T03:13:27.375 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:27 smithi161 bash[20075]: debug 2024-02-13T03:13:27.278+0000 7fa905364680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T03:13:27.674 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:27 smithi161 bash[20075]: debug 2024-02-13T03:13:27.370+0000 7fa905364680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T03:13:28.099 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:27 smithi161 bash[20075]: debug 2024-02-13T03:13:27.794+0000 7fa905364680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T03:13:28.424 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:28 smithi161 bash[19769]: audit 2024-02-13T03:13:27.096628+0000 mon.a (mon.0) 30 : audit [INF] from='client.? v1:172.21.15.161:0/3095742026' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-13T03:13:28.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:28 smithi161 bash[19769]: cluster 2024-02-13T03:13:27.096816+0000 mon.a (mon.0) 31 : cluster [DBG] mgrmap e5: y(active, since 5s) 2024-02-13T03:13:28.924 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:28 smithi161 bash[20075]: debug 2024-02-13T03:13:28.690+0000 7fa905364680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T03:13:28.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:28 smithi161 bash[20075]: debug 2024-02-13T03:13:28.794+0000 7fa905364680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T03:13:28.939 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa217e35700 1 Processor -- start 2024-02-13T03:13:28.940 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa217e35700 1 -- start start 2024-02-13T03:13:28.940 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa217e35700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa2100fc250 con 0x7fa2100fb720 2024-02-13T03:13:28.940 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa215bd1700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa2100fb720 0x7fa2100fbb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33018/0 (socket says 172.21.15.161:33018) 2024-02-13T03:13:28.940 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa215bd1700 1 -- 172.21.15.161:0/1967631157 learned_addr learned my addr 172.21.15.161:0/1967631157 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:28.940 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa214bcf700 1 -- 172.21.15.161:0/1967631157 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443684351 0 0) 0x7fa2100fc250 con 0x7fa2100fb720 2024-02-13T03:13:28.940 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa214bcf700 1 -- 172.21.15.161:0/1967631157 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1fc0033f0 con 0x7fa2100fb720 2024-02-13T03:13:28.940 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa214bcf700 1 -- 172.21.15.161:0/1967631157 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2928721133 0 0) 0x7fa1fc0033f0 con 0x7fa2100fb720 2024-02-13T03:13:28.940 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa214bcf700 1 -- 172.21.15.161:0/1967631157 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa210100ca0 con 0x7fa2100fb720 2024-02-13T03:13:28.940 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa214bcf700 1 -- 172.21.15.161:0/1967631157 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa20c001590 con 0x7fa2100fb720 2024-02-13T03:13:28.940 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa214bcf700 1 -- 172.21.15.161:0/1967631157 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7fa20c001c30 con 0x7fa2100fb720 2024-02-13T03:13:28.940 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa214bcf700 1 -- 172.21.15.161:0/1967631157 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa20c005630 con 0x7fa2100fb720 2024-02-13T03:13:28.942 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa217e35700 1 -- 172.21.15.161:0/1967631157 >> v1:172.21.15.161:6789/0 conn(0x7fa2100fb720 legacy=0x7fa2100fbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:28.942 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa217e35700 1 -- 172.21.15.161:0/1967631157 shutdown_connections 2024-02-13T03:13:28.942 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa217e35700 1 -- 172.21.15.161:0/1967631157 >> 172.21.15.161:0/1967631157 conn(0x7fa2100f6f20 msgr2=0x7fa2100f9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:28.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa217e35700 1 -- 172.21.15.161:0/1967631157 shutdown_connections 2024-02-13T03:13:28.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa217e35700 1 -- 172.21.15.161:0/1967631157 wait complete. 2024-02-13T03:13:28.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa217e35700 1 Processor -- start 2024-02-13T03:13:28.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.934+0000 7fa217e35700 1 -- start start 2024-02-13T03:13:28.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa217e35700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa21010a810 con 0x7fa2100fb720 2024-02-13T03:13:28.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa215bd1700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa2100fb720 0x7fa21010a100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33030/0 (socket says 172.21.15.161:33030) 2024-02-13T03:13:28.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa215bd1700 1 -- 172.21.15.161:0/2796579664 learned_addr learned my addr 172.21.15.161:0/2796579664 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:28.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa206ffd700 1 -- 172.21.15.161:0/2796579664 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309466926 0 0) 0x7fa21010a810 con 0x7fa2100fb720 2024-02-13T03:13:28.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa206ffd700 1 -- 172.21.15.161:0/2796579664 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1fc005260 con 0x7fa2100fb720 2024-02-13T03:13:28.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa206ffd700 1 -- 172.21.15.161:0/2796579664 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 167400140 0 0) 0x7fa1fc005260 con 0x7fa2100fb720 2024-02-13T03:13:28.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa206ffd700 1 -- 172.21.15.161:0/2796579664 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa21010a810 con 0x7fa2100fb720 2024-02-13T03:13:28.943 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa206ffd700 1 -- 172.21.15.161:0/2796579664 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa20c000ed0 con 0x7fa2100fb720 2024-02-13T03:13:28.944 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa206ffd700 1 -- 172.21.15.161:0/2796579664 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2130080487 0 0) 0x7fa21010a810 con 0x7fa2100fb720 2024-02-13T03:13:28.944 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa206ffd700 1 -- 172.21.15.161:0/2796579664 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21010a9a0 con 0x7fa2100fb720 2024-02-13T03:13:28.944 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa217e35700 1 -- 172.21.15.161:0/2796579664 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21010ac50 con 0x7fa2100fb720 2024-02-13T03:13:28.945 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa217e35700 1 -- 172.21.15.161:0/2796579664 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21010e940 con 0x7fa2100fb720 2024-02-13T03:13:28.945 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa206ffd700 1 -- 172.21.15.161:0/2796579664 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7fa20c0049e0 con 0x7fa2100fb720 2024-02-13T03:13:28.945 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa206ffd700 1 -- 172.21.15.161:0/2796579664 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa20c006070 con 0x7fa2100fb720 2024-02-13T03:13:28.945 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa217e35700 1 -- 172.21.15.161:0/2796579664 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f4004f50 con 0x7fa2100fb720 2024-02-13T03:13:28.945 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa206ffd700 1 -- 172.21.15.161:0/2796579664 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 5) v1 ==== 38582+0+0 (unknown 3095178131 0 0) 0x7fa20c00f950 con 0x7fa2100fb720 2024-02-13T03:13:28.945 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa206ffd700 1 -- 172.21.15.161:0/2796579664 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 1315227326 0 0) 0x7fa20c03eb90 con 0x7fa2100fb720 2024-02-13T03:13:28.945 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.938+0000 7fa2153d0700 1 -- 172.21.15.161:0/2796579664 >> v1:172.21.15.161:6800/1159674441 conn(0x7fa1fc0338d0 legacy=0x7fa1fc035d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/1159674441 2024-02-13T03:13:28.949 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:28.946+0000 7fa206ffd700 1 -- 172.21.15.161:0/2796579664 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fa20c011080 con 0x7fa2100fb720 2024-02-13T03:13:29.146 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:29.142+0000 7fa2153d0700 1 -- 172.21.15.161:0/2796579664 >> v1:172.21.15.161:6800/1159674441 conn(0x7fa1fc0338d0 legacy=0x7fa1fc035d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/1159674441 2024-02-13T03:13:29.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:29.326+0000 7fa217e35700 1 -- 172.21.15.161:0/2796579664 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fa1f4005a10 con 0x7fa2100fb720 2024-02-13T03:13:29.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:29.326+0000 7fa206ffd700 1 -- 172.21.15.161:0/2796579664 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (unknown 3768197548 0 15966916) 0x7fa20c014810 con 0x7fa2100fb720 2024-02-13T03:13:29.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: { 2024-02-13T03:13:29.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 5, 2024-02-13T03:13:29.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "available": true, 2024-02-13T03:13:29.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "active_name": "y", 2024-02-13T03:13:29.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-13T03:13:29.332 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: } 2024-02-13T03:13:29.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:29.330+0000 7fa217e35700 1 -- 172.21.15.161:0/2796579664 >> v1:172.21.15.161:6800/1159674441 conn(0x7fa1fc0338d0 legacy=0x7fa1fc035d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-13T03:13:29.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:29.330+0000 7fa217e35700 1 -- 172.21.15.161:0/2796579664 >> v1:172.21.15.161:6789/0 conn(0x7fa2100fb720 legacy=0x7fa21010a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:29.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:29.330+0000 7fa217e35700 1 -- 172.21.15.161:0/2796579664 shutdown_connections 2024-02-13T03:13:29.334 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:29.330+0000 7fa217e35700 1 -- 172.21.15.161:0/2796579664 >> 172.21.15.161:0/2796579664 conn(0x7fa2100f6f20 msgr2=0x7fa2100f7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:29.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:29.330+0000 7fa217e35700 1 -- 172.21.15.161:0/2796579664 shutdown_connections 2024-02-13T03:13:29.335 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:29.330+0000 7fa217e35700 1 -- 172.21.15.161:0/2796579664 wait complete. 2024-02-13T03:13:29.382 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:29 smithi161 bash[20075]: debug 2024-02-13T03:13:29.042+0000 7fa905364680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T03:13:29.674 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:29 smithi161 bash[19769]: audit 2024-02-13T03:13:29.331272+0000 mon.a (mon.0) 32 : audit [DBG] from='client.? v1:172.21.15.161:0/2796579664' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-13T03:13:29.885 INFO:teuthology.orchestra.run.smithi161.stderr:Waiting for the mgr to restart... 2024-02-13T03:13:29.885 INFO:teuthology.orchestra.run.smithi161.stderr:Waiting for mgr epoch 5... 2024-02-13T03:13:30.401 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:30 smithi161 bash[20075]: debug 2024-02-13T03:13:30.142+0000 7fa905364680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T03:13:30.402 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:30 smithi161 bash[20075]: debug 2024-02-13T03:13:30.302+0000 7fa905364680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T03:13:30.402 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:30 smithi161 bash[20075]: debug 2024-02-13T03:13:30.398+0000 7fa905364680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T03:13:30.674 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:30 smithi161 bash[20075]: debug 2024-02-13T03:13:30.506+0000 7fa905364680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T03:13:30.930 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42ebc23700 1 Processor -- start 2024-02-13T03:13:30.930 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42ebc23700 1 -- start start 2024-02-13T03:13:30.930 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42ebc23700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f42e40fc820 con 0x7f42e40fbcf0 2024-02-13T03:13:30.930 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42e99bf700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f42e40fbcf0 0x7f42e40fc110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33042/0 (socket says 172.21.15.161:33042) 2024-02-13T03:13:30.930 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42e99bf700 1 -- 172.21.15.161:0/715596150 learned_addr learned my addr 172.21.15.161:0/715596150 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42e89bd700 1 -- 172.21.15.161:0/715596150 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312906794 0 0) 0x7f42e40fc820 con 0x7f42e40fbcf0 2024-02-13T03:13:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42e89bd700 1 -- 172.21.15.161:0/715596150 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d00033f0 con 0x7f42e40fbcf0 2024-02-13T03:13:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42e89bd700 1 -- 172.21.15.161:0/715596150 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2166554845 0 0) 0x7f42d00033f0 con 0x7f42e40fbcf0 2024-02-13T03:13:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42e89bd700 1 -- 172.21.15.161:0/715596150 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e40ffc60 con 0x7f42e40fbcf0 2024-02-13T03:13:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42e89bd700 1 -- 172.21.15.161:0/715596150 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f42e0001590 con 0x7f42e40fbcf0 2024-02-13T03:13:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42e89bd700 1 -- 172.21.15.161:0/715596150 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f42e0001c30 con 0x7f42e40fbcf0 2024-02-13T03:13:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42e89bd700 1 -- 172.21.15.161:0/715596150 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f42e0005630 con 0x7f42e40fbcf0 2024-02-13T03:13:30.932 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42ebc23700 1 -- 172.21.15.161:0/715596150 >> v1:172.21.15.161:6789/0 conn(0x7f42e40fbcf0 legacy=0x7f42e40fc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:30.933 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42ebc23700 1 -- 172.21.15.161:0/715596150 shutdown_connections 2024-02-13T03:13:30.933 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42ebc23700 1 -- 172.21.15.161:0/715596150 >> 172.21.15.161:0/715596150 conn(0x7f42e40f55a0 msgr2=0x7f42e40f79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:30.933 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42ebc23700 1 -- 172.21.15.161:0/715596150 shutdown_connections 2024-02-13T03:13:30.933 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42ebc23700 1 -- 172.21.15.161:0/715596150 wait complete. 2024-02-13T03:13:30.934 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42ebc23700 1 Processor -- start 2024-02-13T03:13:30.934 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.926+0000 7f42ebc23700 1 -- start start 2024-02-13T03:13:30.934 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42ebc23700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f42e410a8d0 con 0x7f42e40fbcf0 2024-02-13T03:13:30.934 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42e99bf700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f42e40fbcf0 0x7f42e410a1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33050/0 (socket says 172.21.15.161:33050) 2024-02-13T03:13:30.934 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42e99bf700 1 -- 172.21.15.161:0/2037196176 learned_addr learned my addr 172.21.15.161:0/2037196176 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:30.934 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1013671568 0 0) 0x7f42e410a8d0 con 0x7f42e40fbcf0 2024-02-13T03:13:30.934 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d0005260 con 0x7f42e40fbcf0 2024-02-13T03:13:30.934 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4084758736 0 0) 0x7f42d0005260 con 0x7f42e40fbcf0 2024-02-13T03:13:30.934 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f42e410a8d0 con 0x7f42e40fbcf0 2024-02-13T03:13:30.935 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f42e0000ed0 con 0x7f42e40fbcf0 2024-02-13T03:13:30.935 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3605374007 0 0) 0x7f42e410a8d0 con 0x7f42e40fbcf0 2024-02-13T03:13:30.935 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e410aa60 con 0x7f42e40fbcf0 2024-02-13T03:13:30.935 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42ebc23700 1 -- 172.21.15.161:0/2037196176 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e410ad10 con 0x7f42e40fbcf0 2024-02-13T03:13:30.935 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42ebc23700 1 -- 172.21.15.161:0/2037196176 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e410e990 con 0x7f42e40fbcf0 2024-02-13T03:13:30.935 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f42e0004b00 con 0x7f42e40fbcf0 2024-02-13T03:13:30.936 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f42e0006110 con 0x7f42e40fbcf0 2024-02-13T03:13:30.936 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 5) v1 ==== 38582+0+0 (unknown 3095178131 0 0) 0x7f42e000fa10 con 0x7f42e40fbcf0 2024-02-13T03:13:30.936 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42ebc23700 1 -- 172.21.15.161:0/2037196176 --> v1:172.21.15.161:6800/1159674441 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42c8000d40 con 0x7f42d0033800 2024-02-13T03:13:30.936 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 1315227326 0 0) 0x7f42e00018d0 con 0x7f42e40fbcf0 2024-02-13T03:13:30.936 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:30.930+0000 7f42e91be700 1 -- 172.21.15.161:0/2037196176 >> v1:172.21.15.161:6800/1159674441 conn(0x7f42d0033800 legacy=0x7f42d0035cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/1159674441 2024-02-13T03:13:31.103 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:30 smithi161 bash[20075]: debug 2024-02-13T03:13:30.730+0000 7fa905364680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T03:13:31.137 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:31.134+0000 7f42e91be700 1 -- 172.21.15.161:0/2037196176 >> v1:172.21.15.161:6800/1159674441 conn(0x7f42d0033800 legacy=0x7f42d0035cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/1159674441 2024-02-13T03:13:31.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:31 smithi161 bash[20075]: debug 2024-02-13T03:13:31.098+0000 7fa905364680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T03:13:31.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:31 smithi161 bash[20075]: debug 2024-02-13T03:13:31.214+0000 7fa905364680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T03:13:31.542 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:31.538+0000 7f42e91be700 1 -- 172.21.15.161:0/2037196176 >> v1:172.21.15.161:6800/1159674441 conn(0x7f42d0033800 legacy=0x7f42d0035cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/1159674441 2024-02-13T03:13:32.347 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:32.342+0000 7f42e91be700 1 -- 172.21.15.161:0/2037196176 >> v1:172.21.15.161:6800/1159674441 conn(0x7f42d0033800 legacy=0x7f42d0035cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/1159674441 2024-02-13T03:13:32.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:32 smithi161 bash[20075]: debug 2024-02-13T03:13:32.054+0000 7fa905364680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T03:13:32.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:32 smithi161 bash[20075]: debug 2024-02-13T03:13:32.154+0000 7fa905364680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T03:13:32.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:32 smithi161 bash[20075]: debug 2024-02-13T03:13:32.266+0000 7fa905364680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T03:13:32.771 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:32 smithi161 bash[20075]: debug 2024-02-13T03:13:32.478+0000 7fa905364680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T03:13:33.093 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:32 smithi161 bash[20075]: debug 2024-02-13T03:13:32.766+0000 7fa905364680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T03:13:33.093 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:32 smithi161 bash[20075]: debug 2024-02-13T03:13:32.926+0000 7fa905364680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T03:13:33.402 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[20075]: debug 2024-02-13T03:13:33.086+0000 7fa905364680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T03:13:33.402 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[20075]: debug 2024-02-13T03:13:33.294+0000 7fa905364680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T03:13:33.428 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:33.422+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mgrmap(e 6) v1 ==== 38441+0+0 (unknown 3625954480 0 0) 0x7f42e0005910 con 0x7f42e40fbcf0 2024-02-13T03:13:33.428 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:33.422+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 >> v1:172.21.15.161:6800/1159674441 conn(0x7f42d0033800 legacy=0x7f42d0035cc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-13T03:13:33.675 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[20075]: debug 2024-02-13T03:13:33.398+0000 7fa905364680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T03:13:33.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[19769]: cluster 2024-02-13T03:13:33.423464+0000 mon.a (mon.0) 33 : cluster [INF] Active manager daemon y restarted 2024-02-13T03:13:33.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[19769]: cluster 2024-02-13T03:13:33.423937+0000 mon.a (mon.0) 34 : cluster [INF] Activating manager daemon y 2024-02-13T03:13:33.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[19769]: cluster 2024-02-13T03:13:33.429901+0000 mon.a (mon.0) 35 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2024-02-13T03:13:33.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[19769]: cluster 2024-02-13T03:13:33.430118+0000 mon.a (mon.0) 36 : cluster [DBG] mgrmap e6: y(active, starting, since 0.00633293s) 2024-02-13T03:13:33.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[19769]: audit 2024-02-13T03:13:33.431726+0000 mon.a (mon.0) 37 : audit [DBG] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T03:13:33.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[19769]: audit 2024-02-13T03:13:33.433191+0000 mon.a (mon.0) 38 : audit [DBG] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T03:13:33.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[19769]: audit 2024-02-13T03:13:33.434168+0000 mon.a (mon.0) 39 : audit [DBG] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T03:13:33.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[19769]: audit 2024-02-13T03:13:33.434524+0000 mon.a (mon.0) 40 : audit [DBG] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T03:13:33.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[19769]: audit 2024-02-13T03:13:33.434816+0000 mon.a (mon.0) 41 : audit [DBG] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T03:13:33.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[19769]: cluster 2024-02-13T03:13:33.438364+0000 mon.a (mon.0) 42 : cluster [INF] Manager daemon y is now available 2024-02-13T03:13:33.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:33 smithi161 bash[19769]: audit 2024-02-13T03:13:33.459480+0000 mon.a (mon.0) 43 : audit [INF] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' 2024-02-13T03:13:34.430 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:34.426+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mgrmap(e 7) v1 ==== 38476+0+0 (unknown 2831474933 0 0) 0x7f42e00116c0 con 0x7f42e40fbcf0 2024-02-13T03:13:34.500 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:34.426+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 --> v1:172.21.15.161:6800/2381271555 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42c8000d40 con 0x7f42d00380f0 2024-02-13T03:13:34.500 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:34.430+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 <== mgr.14116 v1:172.21.15.161:6800/2381271555 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (unknown 0 0 4198994610) 0x7f42c8000d40 con 0x7f42d00380f0 2024-02-13T03:13:34.500 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:34.446+0000 7f42ebc23700 1 -- 172.21.15.161:0/2037196176 --> v1:172.21.15.161:6800/2381271555 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f42c8002610 con 0x7f42d00380f0 2024-02-13T03:13:34.501 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:34.446+0000 7f42daffd700 1 -- 172.21.15.161:0/2037196176 <== mgr.14116 v1:172.21.15.161:6800/2381271555 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (unknown 0 0 96372106) 0x7f42c8002610 con 0x7f42d00380f0 2024-02-13T03:13:34.501 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: { 2024-02-13T03:13:34.501 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2024-02-13T03:13:34.501 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "initialized": true 2024-02-13T03:13:34.501 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: } 2024-02-13T03:13:34.501 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:34.446+0000 7f42ebc23700 1 -- 172.21.15.161:0/2037196176 >> v1:172.21.15.161:6800/2381271555 conn(0x7f42d00380f0 legacy=0x7f42d003a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:34.501 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:34.446+0000 7f42ebc23700 1 -- 172.21.15.161:0/2037196176 >> v1:172.21.15.161:6789/0 conn(0x7f42e40fbcf0 legacy=0x7f42e410a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:34.501 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:34.446+0000 7f42ebc23700 1 -- 172.21.15.161:0/2037196176 shutdown_connections 2024-02-13T03:13:34.501 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:34.446+0000 7f42ebc23700 1 -- 172.21.15.161:0/2037196176 >> 172.21.15.161:0/2037196176 conn(0x7f42e40f55a0 msgr2=0x7f42e40f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:34.501 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:34.450+0000 7f42ebc23700 1 -- 172.21.15.161:0/2037196176 shutdown_connections 2024-02-13T03:13:34.501 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:34.450+0000 7f42ebc23700 1 -- 172.21.15.161:0/2037196176 wait complete. 2024-02-13T03:13:34.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:34 smithi161 bash[19769]: cephadm 2024-02-13T03:13:33.455614+0000 mgr.y (mgr.14116) 1 : cephadm [INF] Found migration_current of "None". Setting to last migration. 2024-02-13T03:13:34.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:34 smithi161 bash[19769]: audit 2024-02-13T03:13:33.585831+0000 mon.a (mon.0) 44 : audit [INF] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' 2024-02-13T03:13:34.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:34 smithi161 bash[19769]: audit 2024-02-13T03:13:33.587684+0000 mon.a (mon.0) 45 : audit [DBG] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:13:34.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:34 smithi161 bash[19769]: audit 2024-02-13T03:13:33.590490+0000 mon.a (mon.0) 46 : audit [DBG] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:13:34.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:34 smithi161 bash[19769]: audit 2024-02-13T03:13:33.601937+0000 mon.a (mon.0) 47 : audit [DBG] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:13:34.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:34 smithi161 bash[19769]: audit 2024-02-13T03:13:33.618586+0000 mon.a (mon.0) 48 : audit [INF] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T03:13:34.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:34 smithi161 bash[19769]: audit 2024-02-13T03:13:33.623851+0000 mon.a (mon.0) 49 : audit [INF] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T03:13:34.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:34 smithi161 bash[19769]: cluster 2024-02-13T03:13:34.435079+0000 mon.a (mon.0) 50 : cluster [DBG] mgrmap e7: y(active, since 1.0113s) 2024-02-13T03:13:35.033 INFO:teuthology.orchestra.run.smithi161.stderr:mgr epoch 5 is available 2024-02-13T03:13:35.033 INFO:teuthology.orchestra.run.smithi161.stderr:Setting orchestrator backend to cephadm... 2024-02-13T03:13:35.924 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:35 smithi161 bash[19769]: audit 2024-02-13T03:13:34.433080+0000 mgr.y (mgr.14116) 2 : audit [DBG] from='client.14120 v1:172.21.15.161:0/2037196176' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-13T03:13:35.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:35 smithi161 bash[19769]: audit 2024-02-13T03:13:34.452205+0000 mgr.y (mgr.14116) 3 : audit [DBG] from='client.14120 v1:172.21.15.161:0/2037196176' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-13T03:13:36.060 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.054+0000 7f444a41e700 1 Processor -- start 2024-02-13T03:13:36.060 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.054+0000 7f444a41e700 1 -- start start 2024-02-13T03:13:36.060 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.054+0000 7f444a41e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f44440fc8a0 con 0x7f44440fbd70 2024-02-13T03:13:36.060 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.054+0000 7f4443fff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f44440fbd70 0x7f44440fc190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33080/0 (socket says 172.21.15.161:33080) 2024-02-13T03:13:36.060 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.054+0000 7f4443fff700 1 -- 172.21.15.161:0/3053956049 learned_addr learned my addr 172.21.15.161:0/3053956049 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:36.061 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.054+0000 7f44437fe700 1 -- 172.21.15.161:0/3053956049 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298605952 0 0) 0x7f44440fc8a0 con 0x7f44440fbd70 2024-02-13T03:13:36.061 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.054+0000 7f44437fe700 1 -- 172.21.15.161:0/3053956049 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44300033f0 con 0x7f44440fbd70 2024-02-13T03:13:36.061 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.054+0000 7f44437fe700 1 -- 172.21.15.161:0/3053956049 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1598089246 0 0) 0x7f44300033f0 con 0x7f44440fbd70 2024-02-13T03:13:36.062 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.054+0000 7f44437fe700 1 -- 172.21.15.161:0/3053956049 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44440fca30 con 0x7f44440fbd70 2024-02-13T03:13:36.062 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.054+0000 7f44437fe700 1 -- 172.21.15.161:0/3053956049 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f4434001590 con 0x7f44440fbd70 2024-02-13T03:13:36.062 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.054+0000 7f44437fe700 1 -- 172.21.15.161:0/3053956049 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f4434001c30 con 0x7f44440fbd70 2024-02-13T03:13:36.062 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.054+0000 7f44437fe700 1 -- 172.21.15.161:0/3053956049 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f4434005630 con 0x7f44440fbd70 2024-02-13T03:13:36.062 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f444a41e700 1 -- 172.21.15.161:0/3053956049 >> v1:172.21.15.161:6789/0 conn(0x7f44440fbd70 legacy=0x7f44440fc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:36.063 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f444a41e700 1 -- 172.21.15.161:0/3053956049 shutdown_connections 2024-02-13T03:13:36.063 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f444a41e700 1 -- 172.21.15.161:0/3053956049 >> 172.21.15.161:0/3053956049 conn(0x7f44440f55c0 msgr2=0x7f44440f79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:36.063 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f444a41e700 1 -- 172.21.15.161:0/3053956049 shutdown_connections 2024-02-13T03:13:36.063 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f444a41e700 1 -- 172.21.15.161:0/3053956049 wait complete. 2024-02-13T03:13:36.064 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f444a41e700 1 Processor -- start 2024-02-13T03:13:36.064 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f444a41e700 1 -- start start 2024-02-13T03:13:36.064 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f444a41e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f444410c9c0 con 0x7f44440fbd70 2024-02-13T03:13:36.064 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f4443fff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f44440fbd70 0x7f444410c2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33092/0 (socket says 172.21.15.161:33092) 2024-02-13T03:13:36.064 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f4443fff700 1 -- 172.21.15.161:0/2854592257 learned_addr learned my addr 172.21.15.161:0/2854592257 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:36.064 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f4441ffb700 1 -- 172.21.15.161:0/2854592257 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537389836 0 0) 0x7f444410c9c0 con 0x7f44440fbd70 2024-02-13T03:13:36.065 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f4441ffb700 1 -- 172.21.15.161:0/2854592257 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4430005260 con 0x7f44440fbd70 2024-02-13T03:13:36.065 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f4441ffb700 1 -- 172.21.15.161:0/2854592257 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4123166493 0 0) 0x7f4430005260 con 0x7f44440fbd70 2024-02-13T03:13:36.065 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f4441ffb700 1 -- 172.21.15.161:0/2854592257 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f444410c9c0 con 0x7f44440fbd70 2024-02-13T03:13:36.065 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f4441ffb700 1 -- 172.21.15.161:0/2854592257 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f4434000ed0 con 0x7f44440fbd70 2024-02-13T03:13:36.065 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f4441ffb700 1 -- 172.21.15.161:0/2854592257 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 370454997 0 0) 0x7f444410c9c0 con 0x7f44440fbd70 2024-02-13T03:13:36.066 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f4441ffb700 1 -- 172.21.15.161:0/2854592257 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444410cb50 con 0x7f44440fbd70 2024-02-13T03:13:36.066 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f444a41e700 1 -- 172.21.15.161:0/2854592257 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444410ce00 con 0x7f44440fbd70 2024-02-13T03:13:36.066 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f444a41e700 1 -- 172.21.15.161:0/2854592257 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4444110af0 con 0x7f44440fbd70 2024-02-13T03:13:36.066 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f4441ffb700 1 -- 172.21.15.161:0/2854592257 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f4434004a00 con 0x7f44440fbd70 2024-02-13T03:13:36.066 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.058+0000 7f4441ffb700 1 -- 172.21.15.161:0/2854592257 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f44340060a0 con 0x7f44440fbd70 2024-02-13T03:13:36.067 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.062+0000 7f4441ffb700 1 -- 172.21.15.161:0/2854592257 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 3044270204 0 0) 0x7f443400f980 con 0x7f44440fbd70 2024-02-13T03:13:36.067 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.062+0000 7f444a41e700 1 -- 172.21.15.161:0/2854592257 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4424004f50 con 0x7f44440fbd70 2024-02-13T03:13:36.067 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.062+0000 7f4441ffb700 1 -- 172.21.15.161:0/2854592257 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 2320478970 0 0) 0x7f443403ec50 con 0x7f44440fbd70 2024-02-13T03:13:36.072 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.066+0000 7f4441ffb700 1 -- 172.21.15.161:0/2854592257 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f44340110b0 con 0x7f44440fbd70 2024-02-13T03:13:36.424 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.418+0000 7f444a41e700 1 -- 172.21.15.161:0/2854592257 --> v1:172.21.15.161:6800/2381271555 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f4424000bc0 con 0x7f4430033840 2024-02-13T03:13:36.437 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.430+0000 7f4441ffb700 1 -- 172.21.15.161:0/2854592257 <== mgr.14116 v1:172.21.15.161:6800/2381271555 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f4424000bc0 con 0x7f4430033840 2024-02-13T03:13:36.443 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.434+0000 7f444a41e700 1 -- 172.21.15.161:0/2854592257 >> v1:172.21.15.161:6800/2381271555 conn(0x7f4430033840 legacy=0x7f4430035d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.434+0000 7f444a41e700 1 -- 172.21.15.161:0/2854592257 >> v1:172.21.15.161:6789/0 conn(0x7f44440fbd70 legacy=0x7f444410c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.434+0000 7f444a41e700 1 -- 172.21.15.161:0/2854592257 shutdown_connections 2024-02-13T03:13:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.434+0000 7f444a41e700 1 -- 172.21.15.161:0/2854592257 >> 172.21.15.161:0/2854592257 conn(0x7f44440f55c0 msgr2=0x7f44440f61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.434+0000 7f444a41e700 1 -- 172.21.15.161:0/2854592257 shutdown_connections 2024-02-13T03:13:36.444 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:36.434+0000 7f444a41e700 1 -- 172.21.15.161:0/2854592257 wait complete. 2024-02-13T03:13:36.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:36 smithi161 bash[19769]: cluster 2024-02-13T03:13:35.590416+0000 mon.a (mon.0) 51 : cluster [DBG] mgrmap e8: y(active, since 2s) 2024-02-13T03:13:36.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:36 smithi161 bash[19769]: audit 2024-02-13T03:13:36.436468+0000 mon.a (mon.0) 52 : audit [INF] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' 2024-02-13T03:13:36.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:36 smithi161 bash[19769]: audit 2024-02-13T03:13:36.477348+0000 mon.a (mon.0) 53 : audit [DBG] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:13:38.010 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21fb684700 1 Processor -- start 2024-02-13T03:13:38.011 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21fb684700 1 -- start start 2024-02-13T03:13:38.012 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21fb684700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f21f4072100 con 0x7f21f4071ce0 2024-02-13T03:13:38.012 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21f9420700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f21f4071ce0 0x7f21f4070280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39022/0 (socket says 172.21.15.161:39022) 2024-02-13T03:13:38.012 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21f9420700 1 -- 172.21.15.161:0/2556456840 learned_addr learned my addr 172.21.15.161:0/2556456840 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:38.012 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21ebfff700 1 -- 172.21.15.161:0/2556456840 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518456204 0 0) 0x7f21f4072100 con 0x7f21f4071ce0 2024-02-13T03:13:38.012 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21ebfff700 1 -- 172.21.15.161:0/2556456840 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e00033f0 con 0x7f21f4071ce0 2024-02-13T03:13:38.014 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21ebfff700 1 -- 172.21.15.161:0/2556456840 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3128059106 0 0) 0x7f21e00033f0 con 0x7f21f4071ce0 2024-02-13T03:13:38.014 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21ebfff700 1 -- 172.21.15.161:0/2556456840 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f4070990 con 0x7f21f4071ce0 2024-02-13T03:13:38.014 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21ebfff700 1 -- 172.21.15.161:0/2556456840 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f21f0001910 con 0x7f21f4071ce0 2024-02-13T03:13:38.014 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21ebfff700 1 -- 172.21.15.161:0/2556456840 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f21f0002080 con 0x7f21f4071ce0 2024-02-13T03:13:38.014 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21ebfff700 1 -- 172.21.15.161:0/2556456840 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f21f0005a80 con 0x7f21f4071ce0 2024-02-13T03:13:38.014 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21fb684700 1 -- 172.21.15.161:0/2556456840 >> v1:172.21.15.161:6789/0 conn(0x7f21f4071ce0 legacy=0x7f21f4070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:38.014 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21fb684700 1 -- 172.21.15.161:0/2556456840 shutdown_connections 2024-02-13T03:13:38.014 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21fb684700 1 -- 172.21.15.161:0/2556456840 >> 172.21.15.161:0/2556456840 conn(0x7f21f40f6f40 msgr2=0x7f21f40f9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:38.014 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21fb684700 1 -- 172.21.15.161:0/2556456840 shutdown_connections 2024-02-13T03:13:38.014 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.005+0000 7f21fb684700 1 -- 172.21.15.161:0/2556456840 wait complete. 2024-02-13T03:13:38.015 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21fb684700 1 Processor -- start 2024-02-13T03:13:38.015 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21fb684700 1 -- start start 2024-02-13T03:13:38.016 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21fb684700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f21f410ca00 con 0x7f21f4071ce0 2024-02-13T03:13:38.016 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21f9420700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f21f4071ce0 0x7f21f410c2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39024/0 (socket says 172.21.15.161:39024) 2024-02-13T03:13:38.016 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21f9420700 1 -- 172.21.15.161:0/4244843431 learned_addr learned my addr 172.21.15.161:0/4244843431 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:38.016 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21ea7fc700 1 -- 172.21.15.161:0/4244843431 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2808716161 0 0) 0x7f21f410ca00 con 0x7f21f4071ce0 2024-02-13T03:13:38.016 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21ea7fc700 1 -- 172.21.15.161:0/4244843431 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e0005210 con 0x7f21f4071ce0 2024-02-13T03:13:38.017 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21ea7fc700 1 -- 172.21.15.161:0/4244843431 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1411294354 0 0) 0x7f21e0005210 con 0x7f21f4071ce0 2024-02-13T03:13:38.017 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21ea7fc700 1 -- 172.21.15.161:0/4244843431 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21f410ca00 con 0x7f21f4071ce0 2024-02-13T03:13:38.017 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21ea7fc700 1 -- 172.21.15.161:0/4244843431 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f21f0001360 con 0x7f21f4071ce0 2024-02-13T03:13:38.017 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21ea7fc700 1 -- 172.21.15.161:0/4244843431 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2013802240 0 0) 0x7f21f410ca00 con 0x7f21f4071ce0 2024-02-13T03:13:38.017 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21ea7fc700 1 -- 172.21.15.161:0/4244843431 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f410cb90 con 0x7f21f4071ce0 2024-02-13T03:13:38.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21fb684700 1 -- 172.21.15.161:0/4244843431 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f410ce40 con 0x7f21f4071ce0 2024-02-13T03:13:38.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21ea7fc700 1 -- 172.21.15.161:0/4244843431 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f21f00058c0 con 0x7f21f4071ce0 2024-02-13T03:13:38.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21ea7fc700 1 -- 172.21.15.161:0/4244843431 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f21f0006500 con 0x7f21f4071ce0 2024-02-13T03:13:38.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.009+0000 7f21fb684700 1 -- 172.21.15.161:0/4244843431 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f4110b30 con 0x7f21f4071ce0 2024-02-13T03:13:38.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.013+0000 7f21ea7fc700 1 -- 172.21.15.161:0/4244843431 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 3044270204 0 0) 0x7f21f000fe00 con 0x7f21f4071ce0 2024-02-13T03:13:38.018 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.013+0000 7f21fb684700 1 -- 172.21.15.161:0/4244843431 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f4046d80 con 0x7f21f4071ce0 2024-02-13T03:13:38.019 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.013+0000 7f21ea7fc700 1 -- 172.21.15.161:0/4244843431 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 2320478970 0 0) 0x7f21f003f290 con 0x7f21f4071ce0 2024-02-13T03:13:38.023 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.017+0000 7f21ea7fc700 1 -- 172.21.15.161:0/4244843431 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f21f0011440 con 0x7f21f4071ce0 2024-02-13T03:13:38.346 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.341+0000 7f21fb684700 1 -- 172.21.15.161:0/4244843431 --> v1:172.21.15.161:6800/2381271555 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f21f4101bc0 con 0x7f21e0033850 2024-02-13T03:13:38.347 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.341+0000 7f21ea7fc700 1 -- 172.21.15.161:0/4244843431 <== mgr.14116 v1:172.21.15.161:6800/2381271555 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (unknown 0 0 2070689548) 0x7f21f4101bc0 con 0x7f21e0033850 2024-02-13T03:13:38.347 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: value unchanged 2024-02-13T03:13:38.349 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.341+0000 7f21fb684700 1 -- 172.21.15.161:0/4244843431 >> v1:172.21.15.161:6800/2381271555 conn(0x7f21e0033850 legacy=0x7f21e0035d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:38.349 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.341+0000 7f21fb684700 1 -- 172.21.15.161:0/4244843431 >> v1:172.21.15.161:6789/0 conn(0x7f21f4071ce0 legacy=0x7f21f410c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:38.350 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.341+0000 7f21fb684700 1 -- 172.21.15.161:0/4244843431 shutdown_connections 2024-02-13T03:13:38.350 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.341+0000 7f21fb684700 1 -- 172.21.15.161:0/4244843431 >> 172.21.15.161:0/4244843431 conn(0x7f21f40f6f40 msgr2=0x7f21f40fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:38.350 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.345+0000 7f21fb684700 1 -- 172.21.15.161:0/4244843431 shutdown_connections 2024-02-13T03:13:38.350 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:38.345+0000 7f21fb684700 1 -- 172.21.15.161:0/4244843431 wait complete. 2024-02-13T03:13:38.411 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:38 smithi161 bash[19769]: audit 2024-02-13T03:13:36.426516+0000 mgr.y (mgr.14116) 4 : audit [DBG] from='client.14126 v1:172.21.15.161:0/2854592257' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:13:38.920 INFO:teuthology.orchestra.run.smithi161.stderr:Generating ssh key... 2024-02-13T03:13:39.674 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:39 smithi161 bash[19769]: audit 2024-02-13T03:13:38.346755+0000 mgr.y (mgr.14116) 5 : audit [DBG] from='client.14128 v1:172.21.15.161:0/4244843431' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:13:39.986 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f10107c0700 1 Processor -- start 2024-02-13T03:13:39.986 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f10107c0700 1 -- start start 2024-02-13T03:13:39.986 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f10107c0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f10080fc290 con 0x7f10080fb760 2024-02-13T03:13:39.986 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f100e55c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f10080fb760 0x7f10080fbb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39028/0 (socket says 172.21.15.161:39028) 2024-02-13T03:13:39.987 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f100e55c700 1 -- 172.21.15.161:0/3074046975 learned_addr learned my addr 172.21.15.161:0/3074046975 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:39.987 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f100d55a700 1 -- 172.21.15.161:0/3074046975 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663344628 0 0) 0x7f10080fc290 con 0x7f10080fb760 2024-02-13T03:13:39.987 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f100d55a700 1 -- 172.21.15.161:0/3074046975 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff40033f0 con 0x7f10080fb760 2024-02-13T03:13:39.987 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f100d55a700 1 -- 172.21.15.161:0/3074046975 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2240336115 0 0) 0x7f0ff40033f0 con 0x7f10080fb760 2024-02-13T03:13:39.987 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f100d55a700 1 -- 172.21.15.161:0/3074046975 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1008100ce0 con 0x7f10080fb760 2024-02-13T03:13:39.987 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f100d55a700 1 -- 172.21.15.161:0/3074046975 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f1000001910 con 0x7f10080fb760 2024-02-13T03:13:39.988 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f100d55a700 1 -- 172.21.15.161:0/3074046975 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f1000002080 con 0x7f10080fb760 2024-02-13T03:13:39.988 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f100d55a700 1 -- 172.21.15.161:0/3074046975 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f1000005a80 con 0x7f10080fb760 2024-02-13T03:13:39.988 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f10107c0700 1 -- 172.21.15.161:0/3074046975 >> v1:172.21.15.161:6789/0 conn(0x7f10080fb760 legacy=0x7f10080fbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:39.988 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f10107c0700 1 -- 172.21.15.161:0/3074046975 shutdown_connections 2024-02-13T03:13:39.988 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f10107c0700 1 -- 172.21.15.161:0/3074046975 >> 172.21.15.161:0/3074046975 conn(0x7f10080f6f40 msgr2=0x7f10080f9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:39.988 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f10107c0700 1 -- 172.21.15.161:0/3074046975 shutdown_connections 2024-02-13T03:13:39.989 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.981+0000 7f10107c0700 1 -- 172.21.15.161:0/3074046975 wait complete. 2024-02-13T03:13:39.989 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f10107c0700 1 Processor -- start 2024-02-13T03:13:39.989 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f10107c0700 1 -- start start 2024-02-13T03:13:39.990 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f10107c0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f100810a850 con 0x7f10080fb760 2024-02-13T03:13:39.990 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f100e55c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f10080fb760 0x7f100810a140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39040/0 (socket says 172.21.15.161:39040) 2024-02-13T03:13:39.990 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f100e55c700 1 -- 172.21.15.161:0/3598932865 learned_addr learned my addr 172.21.15.161:0/3598932865 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:39.991 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f0fff7fe700 1 -- 172.21.15.161:0/3598932865 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912072297 0 0) 0x7f100810a850 con 0x7f10080fb760 2024-02-13T03:13:39.991 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f0fff7fe700 1 -- 172.21.15.161:0/3598932865 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff4005210 con 0x7f10080fb760 2024-02-13T03:13:39.991 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f0fff7fe700 1 -- 172.21.15.161:0/3598932865 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4191191998 0 0) 0x7f0ff4005210 con 0x7f10080fb760 2024-02-13T03:13:39.991 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f0fff7fe700 1 -- 172.21.15.161:0/3598932865 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f100810a850 con 0x7f10080fb760 2024-02-13T03:13:39.991 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f0fff7fe700 1 -- 172.21.15.161:0/3598932865 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f1000004f10 con 0x7f10080fb760 2024-02-13T03:13:39.992 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f0fff7fe700 1 -- 172.21.15.161:0/3598932865 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 794191659 0 0) 0x7f100810a850 con 0x7f10080fb760 2024-02-13T03:13:39.992 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f0fff7fe700 1 -- 172.21.15.161:0/3598932865 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100810a9e0 con 0x7f10080fb760 2024-02-13T03:13:39.992 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f10107c0700 1 -- 172.21.15.161:0/3598932865 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100810ac90 con 0x7f10080fb760 2024-02-13T03:13:39.992 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f10107c0700 1 -- 172.21.15.161:0/3598932865 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f100810e980 con 0x7f10080fb760 2024-02-13T03:13:39.992 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f0fff7fe700 1 -- 172.21.15.161:0/3598932865 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f1000001310 con 0x7f10080fb760 2024-02-13T03:13:39.993 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f0fff7fe700 1 -- 172.21.15.161:0/3598932865 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f1000006490 con 0x7f10080fb760 2024-02-13T03:13:39.993 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.985+0000 7f0fff7fe700 1 -- 172.21.15.161:0/3598932865 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 3044270204 0 0) 0x7f100000fd70 con 0x7f10080fb760 2024-02-13T03:13:39.993 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.989+0000 7f0fff7fe700 1 -- 172.21.15.161:0/3598932865 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 2320478970 0 0) 0x7f100003fec0 con 0x7f10080fb760 2024-02-13T03:13:39.994 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.989+0000 7f10107c0700 1 -- 172.21.15.161:0/3598932865 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec004f50 con 0x7f10080fb760 2024-02-13T03:13:40.002 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:39.997+0000 7f0fff7fe700 1 -- 172.21.15.161:0/3598932865 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f10000114a0 con 0x7f10080fb760 2024-02-13T03:13:40.327 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:40.321+0000 7f10107c0700 1 -- 172.21.15.161:0/3598932865 --> v1:172.21.15.161:6800/2381271555 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fec000bc0 con 0x7f0ff4033840 2024-02-13T03:13:40.440 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:40.433+0000 7f0fff7fe700 1 -- 172.21.15.161:0/3598932865 <== mgr.14116 v1:172.21.15.161:6800/2381271555 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f0fec000bc0 con 0x7f0ff4033840 2024-02-13T03:13:40.444 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:40.437+0000 7f10107c0700 1 -- 172.21.15.161:0/3598932865 >> v1:172.21.15.161:6800/2381271555 conn(0x7f0ff4033840 legacy=0x7f0ff4035ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:40.444 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:40.437+0000 7f10107c0700 1 -- 172.21.15.161:0/3598932865 >> v1:172.21.15.161:6789/0 conn(0x7f10080fb760 legacy=0x7f100810a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:40.445 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:40.437+0000 7f10107c0700 1 -- 172.21.15.161:0/3598932865 shutdown_connections 2024-02-13T03:13:40.445 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:40.437+0000 7f10107c0700 1 -- 172.21.15.161:0/3598932865 >> 172.21.15.161:0/3598932865 conn(0x7f10080f6f40 msgr2=0x7f10080f7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:40.445 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:40.437+0000 7f10107c0700 1 -- 172.21.15.161:0/3598932865 shutdown_connections 2024-02-13T03:13:40.445 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:40.437+0000 7f10107c0700 1 -- 172.21.15.161:0/3598932865 wait complete. 2024-02-13T03:13:40.924 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: Generating public/private rsa key pair. 2024-02-13T03:13:40.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: Your identification has been saved in /tmp/tmp9ue0xib0/key. 2024-02-13T03:13:40.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: Your public key has been saved in /tmp/tmp9ue0xib0/key.pub. 2024-02-13T03:13:40.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: The key fingerprint is: 2024-02-13T03:13:40.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: SHA256:WWPJw9eXre1gh1SOWql88n5csHPjyOMaymDXN6udanM ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:13:40.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: The key's randomart image is: 2024-02-13T03:13:40.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: +---[RSA 3072]----+ 2024-02-13T03:13:40.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: | .| 2024-02-13T03:13:40.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: | o . . =o| 2024-02-13T03:13:40.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: | O . *.+| 2024-02-13T03:13:40.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: | + = =.= | 2024-02-13T03:13:40.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: | S = *oo| 2024-02-13T03:13:40.926 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: | . =o=o| 2024-02-13T03:13:40.926 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: | o . o.o+++| 2024-02-13T03:13:40.926 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: | . + .o+E+.o| 2024-02-13T03:13:40.926 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: | o.+B=o. | 2024-02-13T03:13:40.926 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:40 smithi161 bash[20075]: +----[SHA256]-----+ 2024-02-13T03:13:41.713 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:41 smithi161 bash[19769]: audit 2024-02-13T03:13:40.327625+0000 mgr.y (mgr.14116) 6 : audit [DBG] from='client.14130 v1:172.21.15.161:0/3598932865' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:13:41.714 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:41 smithi161 bash[19769]: cephadm 2024-02-13T03:13:40.328062+0000 mgr.y (mgr.14116) 7 : cephadm [INF] Generating ssh key... 2024-02-13T03:13:41.714 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:41 smithi161 bash[19769]: audit 2024-02-13T03:13:40.434959+0000 mon.a (mon.0) 54 : audit [INF] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' 2024-02-13T03:13:41.714 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:41 smithi161 bash[19769]: audit 2024-02-13T03:13:40.439135+0000 mon.a (mon.0) 55 : audit [INF] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' 2024-02-13T03:13:41.948 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.941+0000 7f789c734700 1 Processor -- start 2024-02-13T03:13:41.948 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.941+0000 7f789c734700 1 -- start start 2024-02-13T03:13:41.948 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.941+0000 7f789c734700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7894101e90 con 0x7f78940fefd0 2024-02-13T03:13:41.948 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.941+0000 7f789a4d0700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f78940fefd0 0x7f78940ff410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39050/0 (socket says 172.21.15.161:39050) 2024-02-13T03:13:41.948 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.941+0000 7f789a4d0700 1 -- 172.21.15.161:0/1764308230 learned_addr learned my addr 172.21.15.161:0/1764308230 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:41.948 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.941+0000 7f78994ce700 1 -- 172.21.15.161:0/1764308230 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867760484 0 0) 0x7f7894101e90 con 0x7f78940fefd0 2024-02-13T03:13:41.948 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.941+0000 7f78994ce700 1 -- 172.21.15.161:0/1764308230 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f787c0033f0 con 0x7f78940fefd0 2024-02-13T03:13:41.950 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.941+0000 7f78994ce700 1 -- 172.21.15.161:0/1764308230 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2630555163 0 0) 0x7f787c0033f0 con 0x7f78940fefd0 2024-02-13T03:13:41.950 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.941+0000 7f78994ce700 1 -- 172.21.15.161:0/1764308230 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7894103030 con 0x7f78940fefd0 2024-02-13T03:13:41.950 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.941+0000 7f78994ce700 1 -- 172.21.15.161:0/1764308230 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f7884001590 con 0x7f78940fefd0 2024-02-13T03:13:41.950 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.941+0000 7f78994ce700 1 -- 172.21.15.161:0/1764308230 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f7884001c30 con 0x7f78940fefd0 2024-02-13T03:13:41.950 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.941+0000 7f78994ce700 1 -- 172.21.15.161:0/1764308230 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f7884005630 con 0x7f78940fefd0 2024-02-13T03:13:41.950 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f789c734700 1 -- 172.21.15.161:0/1764308230 >> v1:172.21.15.161:6789/0 conn(0x7f78940fefd0 legacy=0x7f78940ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:41.950 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f789c734700 1 -- 172.21.15.161:0/1764308230 shutdown_connections 2024-02-13T03:13:41.950 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f789c734700 1 -- 172.21.15.161:0/1764308230 >> 172.21.15.161:0/1764308230 conn(0x7f7894071330 msgr2=0x7f7894071760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:41.951 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f789c734700 1 -- 172.21.15.161:0/1764308230 shutdown_connections 2024-02-13T03:13:41.951 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f789c734700 1 -- 172.21.15.161:0/1764308230 wait complete. 2024-02-13T03:13:41.951 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f789c734700 1 Processor -- start 2024-02-13T03:13:41.951 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f789c734700 1 -- start start 2024-02-13T03:13:41.951 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f789c734700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f789410cb30 con 0x7f78940fefd0 2024-02-13T03:13:41.953 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f789a4d0700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f78940fefd0 0x7f789410c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39058/0 (socket says 172.21.15.161:39058) 2024-02-13T03:13:41.953 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f789a4d0700 1 -- 172.21.15.161:0/625479383 learned_addr learned my addr 172.21.15.161:0/625479383 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:41.953 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f788b7fe700 1 -- 172.21.15.161:0/625479383 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895589607 0 0) 0x7f789410cb30 con 0x7f78940fefd0 2024-02-13T03:13:41.953 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f788b7fe700 1 -- 172.21.15.161:0/625479383 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f787c005260 con 0x7f78940fefd0 2024-02-13T03:13:41.953 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f788b7fe700 1 -- 172.21.15.161:0/625479383 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3170834073 0 0) 0x7f787c005260 con 0x7f78940fefd0 2024-02-13T03:13:41.953 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f788b7fe700 1 -- 172.21.15.161:0/625479383 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f789410cb30 con 0x7f78940fefd0 2024-02-13T03:13:41.953 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f788b7fe700 1 -- 172.21.15.161:0/625479383 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f7884000ed0 con 0x7f78940fefd0 2024-02-13T03:13:41.954 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f788b7fe700 1 -- 172.21.15.161:0/625479383 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3369935339 0 0) 0x7f789410cb30 con 0x7f78940fefd0 2024-02-13T03:13:41.954 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f788b7fe700 1 -- 172.21.15.161:0/625479383 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789410ccc0 con 0x7f78940fefd0 2024-02-13T03:13:41.954 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f789c734700 1 -- 172.21.15.161:0/625479383 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789410cf70 con 0x7f78940fefd0 2024-02-13T03:13:41.954 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f789c734700 1 -- 172.21.15.161:0/625479383 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7894110c60 con 0x7f78940fefd0 2024-02-13T03:13:41.954 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f788b7fe700 1 -- 172.21.15.161:0/625479383 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7f7884004a20 con 0x7f78940fefd0 2024-02-13T03:13:41.954 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.945+0000 7f788b7fe700 1 -- 172.21.15.161:0/625479383 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f7884006150 con 0x7f78940fefd0 2024-02-13T03:13:41.955 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.949+0000 7f788b7fe700 1 -- 172.21.15.161:0/625479383 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 3044270204 0 0) 0x7f78840013b0 con 0x7f78940fefd0 2024-02-13T03:13:41.955 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.949+0000 7f788b7fe700 1 -- 172.21.15.161:0/625479383 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 2320478970 0 0) 0x7f788403ec20 con 0x7f78940fefd0 2024-02-13T03:13:41.955 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.949+0000 7f789c734700 1 -- 172.21.15.161:0/625479383 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7880004f50 con 0x7f78940fefd0 2024-02-13T03:13:41.959 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:41.953+0000 7f788b7fe700 1 -- 172.21.15.161:0/625479383 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f7884005920 con 0x7f78940fefd0 2024-02-13T03:13:42.284 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:42.277+0000 7f789c734700 1 -- 172.21.15.161:0/625479383 --> v1:172.21.15.161:6800/2381271555 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f7880000bc0 con 0x7f787c033900 2024-02-13T03:13:42.285 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:42.277+0000 7f788b7fe700 1 -- 172.21.15.161:0/625479383 <== mgr.14116 v1:172.21.15.161:6800/2381271555 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (unknown 0 0 1072056736) 0x7f7880000bc0 con 0x7f787c033900 2024-02-13T03:13:42.285 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC17oVVUIIh7asXf9BVbeHf8GEaNZ0Zty5Wrxx328tMSZ1L/xstI/rjeoHZxdEVT1j94rXkyWGqp3W8tMSTRdOiuT2CHp+9mjwgTg5GijqLYTi0ZmWJGDI2ALmUGe6lq/n9Yu0bU0wD4Bbh0kCio/Ue6cBBmYRKM0tEKMP7zus6sIy6oqIbnEjqgCR2a1jrIDEiZTChgNkPGTSy4grLWrRCoYFeJsEw4y6z8SbUTGkytH4Z1JJwZKcTBK33i+0BI0+nRgPGIhziB67Oaq7JP6z1M0sRVBsR+2gB6hN5vvYRT8074J7vv+WU9MUeeCh62xE3BWIbN+MLRzzK/QsCGTatV3gLzl94v3UsfPKGXdsqhgB6W4CYos512HwEM0/s+E+BKpGULlwTcJfSydM/gvOYwpIT6F78P/XL6XEo5rcxdcaV8GXNP50vLPVmyrAyWzdCp2s/bbhEJGhFvALBz8Ldaz+aNpWSl/QN50Iwgjv19dSy/02BuhLrf6zcV5fBh9k= ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:13:42.287 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:42.281+0000 7f789c734700 1 -- 172.21.15.161:0/625479383 >> v1:172.21.15.161:6800/2381271555 conn(0x7f787c033900 legacy=0x7f787c035dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:42.287 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:42.281+0000 7f789c734700 1 -- 172.21.15.161:0/625479383 >> v1:172.21.15.161:6789/0 conn(0x7f78940fefd0 legacy=0x7f789410c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:42.288 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:42.281+0000 7f789c734700 1 -- 172.21.15.161:0/625479383 shutdown_connections 2024-02-13T03:13:42.288 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:42.281+0000 7f789c734700 1 -- 172.21.15.161:0/625479383 >> 172.21.15.161:0/625479383 conn(0x7f7894071330 msgr2=0x7f78940face0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:42.288 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:42.281+0000 7f789c734700 1 -- 172.21.15.161:0/625479383 shutdown_connections 2024-02-13T03:13:42.288 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:42.281+0000 7f789c734700 1 -- 172.21.15.161:0/625479383 wait complete. 2024-02-13T03:13:42.798 INFO:teuthology.orchestra.run.smithi161.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-13T03:13:42.799 INFO:teuthology.orchestra.run.smithi161.stderr:Adding key to root@localhost authorized_keys... 2024-02-13T03:13:42.800 INFO:teuthology.orchestra.run.smithi161.stderr:Adding host smithi161... 2024-02-13T03:13:43.494 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:43 smithi161 bash[19769]: audit 2024-02-13T03:13:42.284938+0000 mgr.y (mgr.14116) 8 : audit [DBG] from='client.14132 v1:172.21.15.161:0/625479383' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:13:43.809 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.801+0000 7fead7d6d700 1 Processor -- start 2024-02-13T03:13:43.809 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.801+0000 7fead7d6d700 1 -- start start 2024-02-13T03:13:43.810 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead7d6d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fead00ffae0 con 0x7fead006fc20 2024-02-13T03:13:43.810 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead5b09700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fead006fc20 0x7fead00fe3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39070/0 (socket says 172.21.15.161:39070) 2024-02-13T03:13:43.811 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead5b09700 1 -- 172.21.15.161:0/2456556340 learned_addr learned my addr 172.21.15.161:0/2456556340 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:43.811 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead4b07700 1 -- 172.21.15.161:0/2456556340 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763854927 0 0) 0x7fead00ffae0 con 0x7fead006fc20 2024-02-13T03:13:43.811 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead4b07700 1 -- 172.21.15.161:0/2456556340 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feab80033f0 con 0x7fead006fc20 2024-02-13T03:13:43.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead4b07700 1 -- 172.21.15.161:0/2456556340 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 290983078 0 0) 0x7feab80033f0 con 0x7fead006fc20 2024-02-13T03:13:43.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead4b07700 1 -- 172.21.15.161:0/2456556340 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead0100c80 con 0x7fead006fc20 2024-02-13T03:13:43.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead4b07700 1 -- 172.21.15.161:0/2456556340 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7feac0001910 con 0x7fead006fc20 2024-02-13T03:13:43.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead4b07700 1 -- 172.21.15.161:0/2456556340 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7feac0002080 con 0x7fead006fc20 2024-02-13T03:13:43.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead4b07700 1 -- 172.21.15.161:0/2456556340 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7feac0005a80 con 0x7fead006fc20 2024-02-13T03:13:43.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead7d6d700 1 -- 172.21.15.161:0/2456556340 >> v1:172.21.15.161:6789/0 conn(0x7fead006fc20 legacy=0x7fead00fe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:43.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead7d6d700 1 -- 172.21.15.161:0/2456556340 shutdown_connections 2024-02-13T03:13:43.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead7d6d700 1 -- 172.21.15.161:0/2456556340 >> 172.21.15.161:0/2456556340 conn(0x7fead00f5e40 msgr2=0x7fead00f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:43.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead7d6d700 1 -- 172.21.15.161:0/2456556340 shutdown_connections 2024-02-13T03:13:43.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead7d6d700 1 -- 172.21.15.161:0/2456556340 wait complete. 2024-02-13T03:13:43.813 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead7d6d700 1 Processor -- start 2024-02-13T03:13:43.814 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead7d6d700 1 -- start start 2024-02-13T03:13:43.814 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead7d6d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fead010a810 con 0x7fead006fc20 2024-02-13T03:13:43.814 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead5b09700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fead006fc20 0x7fead010a100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39084/0 (socket says 172.21.15.161:39084) 2024-02-13T03:13:43.814 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7fead5b09700 1 -- 172.21.15.161:0/997061739 learned_addr learned my addr 172.21.15.161:0/997061739 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:43.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7feaceffd700 1 -- 172.21.15.161:0/997061739 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582723519 0 0) 0x7fead010a810 con 0x7fead006fc20 2024-02-13T03:13:43.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.805+0000 7feaceffd700 1 -- 172.21.15.161:0/997061739 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feab8005210 con 0x7fead006fc20 2024-02-13T03:13:43.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.809+0000 7feaceffd700 1 -- 172.21.15.161:0/997061739 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 894932472 0 0) 0x7feab8005210 con 0x7fead006fc20 2024-02-13T03:13:43.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.809+0000 7feaceffd700 1 -- 172.21.15.161:0/997061739 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fead010a810 con 0x7fead006fc20 2024-02-13T03:13:43.815 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.809+0000 7feaceffd700 1 -- 172.21.15.161:0/997061739 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7feac0001360 con 0x7fead006fc20 2024-02-13T03:13:43.816 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.809+0000 7feaceffd700 1 -- 172.21.15.161:0/997061739 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 996577618 0 0) 0x7fead010a810 con 0x7fead006fc20 2024-02-13T03:13:43.816 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.809+0000 7feaceffd700 1 -- 172.21.15.161:0/997061739 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead010a9a0 con 0x7fead006fc20 2024-02-13T03:13:43.816 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.809+0000 7fead7d6d700 1 -- 172.21.15.161:0/997061739 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead010ac50 con 0x7fead006fc20 2024-02-13T03:13:43.816 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.809+0000 7fead7d6d700 1 -- 172.21.15.161:0/997061739 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fead010e940 con 0x7fead006fc20 2024-02-13T03:13:43.816 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.809+0000 7feaceffd700 1 -- 172.21.15.161:0/997061739 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7feac00058c0 con 0x7fead006fc20 2024-02-13T03:13:43.817 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.809+0000 7feaceffd700 1 -- 172.21.15.161:0/997061739 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7feac0006500 con 0x7fead006fc20 2024-02-13T03:13:43.817 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.809+0000 7fead7d6d700 1 -- 172.21.15.161:0/997061739 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead0046d80 con 0x7fead006fc20 2024-02-13T03:13:43.817 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.809+0000 7feaceffd700 1 -- 172.21.15.161:0/997061739 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 3044270204 0 0) 0x7feac0006660 con 0x7fead006fc20 2024-02-13T03:13:43.817 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.809+0000 7feaceffd700 1 -- 172.21.15.161:0/997061739 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 2320478970 0 0) 0x7feac003f0a0 con 0x7fead006fc20 2024-02-13T03:13:43.821 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:43.817+0000 7feaceffd700 1 -- 172.21.15.161:0/997061739 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7feac00070a0 con 0x7fead006fc20 2024-02-13T03:13:44.160 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:44.153+0000 7fead7d6d700 1 -- 172.21.15.161:0/997061739 --> v1:172.21.15.161:6800/2381271555 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "addr": "172.21.15.161", "target": ["mon-mgr", ""]}) v1 -- 0x7fead00ff0d0 con 0x7feab8033850 2024-02-13T03:13:44.675 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:44 smithi161 bash[20075]: Warning: Permanently added '172.21.15.161' (ECDSA) to the list of known hosts. 2024-02-13T03:13:44.996 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:44.989+0000 7feaceffd700 1 -- 172.21.15.161:0/997061739 <== mgr.14116 v1:172.21.15.161:6800/2381271555 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (unknown 0 0 1846269252) 0x7fead00ff0d0 con 0x7feab8033850 2024-02-13T03:13:44.996 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: Added host 'smithi161' with addr '172.21.15.161' 2024-02-13T03:13:45.000 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:44.993+0000 7fead7d6d700 1 -- 172.21.15.161:0/997061739 >> v1:172.21.15.161:6800/2381271555 conn(0x7feab8033850 legacy=0x7feab8035d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:45.001 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:44.993+0000 7fead7d6d700 1 -- 172.21.15.161:0/997061739 >> v1:172.21.15.161:6789/0 conn(0x7fead006fc20 legacy=0x7fead010a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:45.001 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:44.993+0000 7fead7d6d700 1 -- 172.21.15.161:0/997061739 shutdown_connections 2024-02-13T03:13:45.001 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:44.993+0000 7fead7d6d700 1 -- 172.21.15.161:0/997061739 >> 172.21.15.161:0/997061739 conn(0x7fead00f5e40 msgr2=0x7fead00fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:45.001 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:44.993+0000 7fead7d6d700 1 -- 172.21.15.161:0/997061739 shutdown_connections 2024-02-13T03:13:45.001 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:44.993+0000 7fead7d6d700 1 -- 172.21.15.161:0/997061739 wait complete. 2024-02-13T03:13:45.788 INFO:teuthology.orchestra.run.smithi161.stderr:Deploying unmanaged mon service... 2024-02-13T03:13:46.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:45 smithi161 bash[19769]: audit 2024-02-13T03:13:44.162355+0000 mgr.y (mgr.14116) 9 : audit [DBG] from='client.14134 v1:172.21.15.161:0/997061739' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "addr": "172.21.15.161", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:13:46.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:45 smithi161 bash[19769]: audit 2024-02-13T03:13:44.994582+0000 mon.a (mon.0) 56 : audit [INF] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' 2024-02-13T03:13:46.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:45 smithi161 bash[19769]: cephadm 2024-02-13T03:13:44.995324+0000 mgr.y (mgr.14116) 10 : cephadm [INF] Added host smithi161 2024-02-13T03:13:46.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:45 smithi161 bash[19769]: audit 2024-02-13T03:13:45.019923+0000 mon.a (mon.0) 57 : audit [DBG] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:13:47.263 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.253+0000 7fa116549700 1 Processor -- start 2024-02-13T03:13:47.263 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.253+0000 7fa116549700 1 -- start start 2024-02-13T03:13:47.263 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.253+0000 7fa116549700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa1100fe490 con 0x7fa1100fd960 2024-02-13T03:13:47.263 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.253+0000 7fa10ffff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa1100fd960 0x7fa1100fdd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59520/0 (socket says 172.21.15.161:59520) 2024-02-13T03:13:47.263 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.253+0000 7fa10ffff700 1 -- 172.21.15.161:0/4078899510 learned_addr learned my addr 172.21.15.161:0/4078899510 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:47.263 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.253+0000 7fa10f7fe700 1 -- 172.21.15.161:0/4078899510 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062071458 0 0) 0x7fa1100fe490 con 0x7fa1100fd960 2024-02-13T03:13:47.263 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.253+0000 7fa10f7fe700 1 -- 172.21.15.161:0/4078899510 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0fc0033f0 con 0x7fa1100fd960 2024-02-13T03:13:47.263 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.253+0000 7fa10f7fe700 1 -- 172.21.15.161:0/4078899510 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3079550261 0 0) 0x7fa0fc0033f0 con 0x7fa1100fd960 2024-02-13T03:13:47.263 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.253+0000 7fa10f7fe700 1 -- 172.21.15.161:0/4078899510 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa110102ee0 con 0x7fa1100fd960 2024-02-13T03:13:47.263 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.253+0000 7fa10f7fe700 1 -- 172.21.15.161:0/4078899510 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa100001910 con 0x7fa1100fd960 2024-02-13T03:13:47.263 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa10f7fe700 1 -- 172.21.15.161:0/4078899510 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7fa100002080 con 0x7fa1100fd960 2024-02-13T03:13:47.263 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa10f7fe700 1 -- 172.21.15.161:0/4078899510 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa100005a80 con 0x7fa1100fd960 2024-02-13T03:13:47.264 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa116549700 1 -- 172.21.15.161:0/4078899510 >> v1:172.21.15.161:6789/0 conn(0x7fa1100fd960 legacy=0x7fa1100fdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:47.264 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa116549700 1 -- 172.21.15.161:0/4078899510 shutdown_connections 2024-02-13T03:13:47.264 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa116549700 1 -- 172.21.15.161:0/4078899510 >> 172.21.15.161:0/4078899510 conn(0x7fa1100710c0 msgr2=0x7fa1100714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:47.264 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa116549700 1 -- 172.21.15.161:0/4078899510 shutdown_connections 2024-02-13T03:13:47.264 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa116549700 1 -- 172.21.15.161:0/4078899510 wait complete. 2024-02-13T03:13:47.264 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa116549700 1 Processor -- start 2024-02-13T03:13:47.264 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa116549700 1 -- start start 2024-02-13T03:13:47.265 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa116549700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa11010c9e0 con 0x7fa1100fd960 2024-02-13T03:13:47.266 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa10ffff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa1100fd960 0x7fa11010c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59536/0 (socket says 172.21.15.161:59536) 2024-02-13T03:13:47.266 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa10ffff700 1 -- 172.21.15.161:0/2935139611 learned_addr learned my addr 172.21.15.161:0/2935139611 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:47.266 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa10dffb700 1 -- 172.21.15.161:0/2935139611 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924567781 0 0) 0x7fa11010c9e0 con 0x7fa1100fd960 2024-02-13T03:13:47.266 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa10dffb700 1 -- 172.21.15.161:0/2935139611 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0fc005210 con 0x7fa1100fd960 2024-02-13T03:13:47.266 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa10dffb700 1 -- 172.21.15.161:0/2935139611 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3516499788 0 0) 0x7fa0fc005210 con 0x7fa1100fd960 2024-02-13T03:13:47.266 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa10dffb700 1 -- 172.21.15.161:0/2935139611 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa11010c9e0 con 0x7fa1100fd960 2024-02-13T03:13:47.266 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa10dffb700 1 -- 172.21.15.161:0/2935139611 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa100004f10 con 0x7fa1100fd960 2024-02-13T03:13:47.266 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa10dffb700 1 -- 172.21.15.161:0/2935139611 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3094067482 0 0) 0x7fa11010c9e0 con 0x7fa1100fd960 2024-02-13T03:13:47.266 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa10dffb700 1 -- 172.21.15.161:0/2935139611 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11010cb70 con 0x7fa1100fd960 2024-02-13T03:13:47.266 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa116549700 1 -- 172.21.15.161:0/2935139611 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11010ce20 con 0x7fa1100fd960 2024-02-13T03:13:47.266 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa116549700 1 -- 172.21.15.161:0/2935139611 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa110110b10 con 0x7fa1100fd960 2024-02-13T03:13:47.267 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa10dffb700 1 -- 172.21.15.161:0/2935139611 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 2682489661 0 0) 0x7fa100001500 con 0x7fa1100fd960 2024-02-13T03:13:47.267 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.257+0000 7fa10dffb700 1 -- 172.21.15.161:0/2935139611 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa1000063b0 con 0x7fa1100fd960 2024-02-13T03:13:47.267 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.261+0000 7fa10dffb700 1 -- 172.21.15.161:0/2935139611 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 3044270204 0 0) 0x7fa10000fbd0 con 0x7fa1100fd960 2024-02-13T03:13:47.268 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.261+0000 7fa10dffb700 1 -- 172.21.15.161:0/2935139611 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 2320478970 0 0) 0x7fa10003eef0 con 0x7fa1100fd960 2024-02-13T03:13:47.268 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.261+0000 7fa116549700 1 -- 172.21.15.161:0/2935139611 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f0004f50 con 0x7fa1100fd960 2024-02-13T03:13:47.273 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.269+0000 7fa10dffb700 1 -- 172.21.15.161:0/2935139611 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fa100011480 con 0x7fa1100fd960 2024-02-13T03:13:47.618 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.613+0000 7fa116549700 1 -- 172.21.15.161:0/2935139611 --> v1:172.21.15.161:6800/2381271555 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa0f0000bc0 con 0x7fa0fc033960 2024-02-13T03:13:47.628 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.621+0000 7fa10dffb700 1 -- 172.21.15.161:0/2935139611 <== mgr.14116 v1:172.21.15.161:6800/2381271555 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7fa0f0000bc0 con 0x7fa0fc033960 2024-02-13T03:13:47.628 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: Scheduled mon update... 2024-02-13T03:13:47.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.625+0000 7fa116549700 1 -- 172.21.15.161:0/2935139611 >> v1:172.21.15.161:6800/2381271555 conn(0x7fa0fc033960 legacy=0x7fa0fc035e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:47.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.625+0000 7fa116549700 1 -- 172.21.15.161:0/2935139611 >> v1:172.21.15.161:6789/0 conn(0x7fa1100fd960 legacy=0x7fa11010c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:47.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.625+0000 7fa116549700 1 -- 172.21.15.161:0/2935139611 shutdown_connections 2024-02-13T03:13:47.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.625+0000 7fa116549700 1 -- 172.21.15.161:0/2935139611 >> 172.21.15.161:0/2935139611 conn(0x7fa1100710c0 msgr2=0x7fa1100fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:47.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.625+0000 7fa116549700 1 -- 172.21.15.161:0/2935139611 shutdown_connections 2024-02-13T03:13:47.631 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:47.625+0000 7fa116549700 1 -- 172.21.15.161:0/2935139611 wait complete. 2024-02-13T03:13:48.255 INFO:teuthology.orchestra.run.smithi161.stderr:Deploying unmanaged mgr service... 2024-02-13T03:13:48.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:48 smithi161 bash[19769]: audit 2024-02-13T03:13:47.619318+0000 mgr.y (mgr.14116) 11 : audit [DBG] from='client.14136 v1:172.21.15.161:0/2935139611' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:13:48.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:48 smithi161 bash[19769]: cephadm 2024-02-13T03:13:47.622560+0000 mgr.y (mgr.14116) 12 : cephadm [INF] Saving service mon spec with placement count:5 2024-02-13T03:13:48.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:48 smithi161 bash[19769]: audit 2024-02-13T03:13:47.627207+0000 mon.a (mon.0) 58 : audit [INF] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' 2024-02-13T03:13:48.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:48 smithi161 bash[19769]: audit 2024-02-13T03:13:47.928823+0000 mon.a (mon.0) 59 : audit [INF] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' 2024-02-13T03:13:48.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:48 smithi161 bash[19769]: audit 2024-02-13T03:13:48.219919+0000 mon.a (mon.0) 60 : audit [INF] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' 2024-02-13T03:13:49.280 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.273+0000 7fe19c09d700 1 Processor -- start 2024-02-13T03:13:49.280 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.273+0000 7fe19c09d700 1 -- start start 2024-02-13T03:13:49.281 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.273+0000 7fe19c09d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe1940fc290 con 0x7fe1940fb760 2024-02-13T03:13:49.281 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.273+0000 7fe199e39700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe1940fb760 0x7fe1940fbb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59542/0 (socket says 172.21.15.161:59542) 2024-02-13T03:13:49.281 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.273+0000 7fe199e39700 1 -- 172.21.15.161:0/1049876408 learned_addr learned my addr 172.21.15.161:0/1049876408 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:49.282 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.273+0000 7fe198e37700 1 -- 172.21.15.161:0/1049876408 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743529992 0 0) 0x7fe1940fc290 con 0x7fe1940fb760 2024-02-13T03:13:49.282 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.273+0000 7fe198e37700 1 -- 172.21.15.161:0/1049876408 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1800033f0 con 0x7fe1940fb760 2024-02-13T03:13:49.282 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe198e37700 1 -- 172.21.15.161:0/1049876408 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1613451540 0 0) 0x7fe1800033f0 con 0x7fe1940fb760 2024-02-13T03:13:49.282 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe198e37700 1 -- 172.21.15.161:0/1049876408 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe194100ce0 con 0x7fe1940fb760 2024-02-13T03:13:49.282 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe198e37700 1 -- 172.21.15.161:0/1049876408 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fe184001590 con 0x7fe1940fb760 2024-02-13T03:13:49.282 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe198e37700 1 -- 172.21.15.161:0/1049876408 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fe184001c50 con 0x7fe1940fb760 2024-02-13T03:13:49.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe198e37700 1 -- 172.21.15.161:0/1049876408 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fe184005670 con 0x7fe1940fb760 2024-02-13T03:13:49.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe19c09d700 1 -- 172.21.15.161:0/1049876408 >> v1:172.21.15.161:6789/0 conn(0x7fe1940fb760 legacy=0x7fe1940fbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:49.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe19c09d700 1 -- 172.21.15.161:0/1049876408 shutdown_connections 2024-02-13T03:13:49.283 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe19c09d700 1 -- 172.21.15.161:0/1049876408 >> 172.21.15.161:0/1049876408 conn(0x7fe1940f6f40 msgr2=0x7fe1940f9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:49.284 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe19c09d700 1 -- 172.21.15.161:0/1049876408 shutdown_connections 2024-02-13T03:13:49.284 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe19c09d700 1 -- 172.21.15.161:0/1049876408 wait complete. 2024-02-13T03:13:49.284 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe19c09d700 1 Processor -- start 2024-02-13T03:13:49.284 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe19c09d700 1 -- start start 2024-02-13T03:13:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe19c09d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe194101be0 con 0x7fe1940fb760 2024-02-13T03:13:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe199e39700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe1940fb760 0x7fe1941014d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59550/0 (socket says 172.21.15.161:59550) 2024-02-13T03:13:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe199e39700 1 -- 172.21.15.161:0/887737471 learned_addr learned my addr 172.21.15.161:0/887737471 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe18affd700 1 -- 172.21.15.161:0/887737471 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068719259 0 0) 0x7fe194101be0 con 0x7fe1940fb760 2024-02-13T03:13:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe18affd700 1 -- 172.21.15.161:0/887737471 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe180005260 con 0x7fe1940fb760 2024-02-13T03:13:49.286 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe18affd700 1 -- 172.21.15.161:0/887737471 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1448325618 0 0) 0x7fe180005260 con 0x7fe1940fb760 2024-02-13T03:13:49.287 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe18affd700 1 -- 172.21.15.161:0/887737471 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe194101be0 con 0x7fe1940fb760 2024-02-13T03:13:49.287 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe18affd700 1 -- 172.21.15.161:0/887737471 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fe184000ff0 con 0x7fe1940fb760 2024-02-13T03:13:49.287 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe18affd700 1 -- 172.21.15.161:0/887737471 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2377267039 0 0) 0x7fe194101be0 con 0x7fe1940fb760 2024-02-13T03:13:49.287 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe18affd700 1 -- 172.21.15.161:0/887737471 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe194101d70 con 0x7fe1940fb760 2024-02-13T03:13:49.287 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.277+0000 7fe19c09d700 1 -- 172.21.15.161:0/887737471 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe194101fc0 con 0x7fe1940fb760 2024-02-13T03:13:49.287 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.281+0000 7fe19c09d700 1 -- 172.21.15.161:0/887737471 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe194103d70 con 0x7fe1940fb760 2024-02-13T03:13:49.287 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.281+0000 7fe18affd700 1 -- 172.21.15.161:0/887737471 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fe184004d30 con 0x7fe1940fb760 2024-02-13T03:13:49.287 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.281+0000 7fe18affd700 1 -- 172.21.15.161:0/887737471 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fe184006060 con 0x7fe1940fb760 2024-02-13T03:13:49.288 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.281+0000 7fe18affd700 1 -- 172.21.15.161:0/887737471 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 3044270204 0 0) 0x7fe184005950 con 0x7fe1940fb760 2024-02-13T03:13:49.288 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.281+0000 7fe19c09d700 1 -- 172.21.15.161:0/887737471 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe178004f50 con 0x7fe1940fb760 2024-02-13T03:13:49.288 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.281+0000 7fe18affd700 1 -- 172.21.15.161:0/887737471 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 2320478970 0 0) 0x7fe18403ed30 con 0x7fe1940fb760 2024-02-13T03:13:49.294 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.289+0000 7fe18affd700 1 -- 172.21.15.161:0/887737471 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fe184013eb0 con 0x7fe1940fb760 2024-02-13T03:13:49.637 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.629+0000 7fe19c09d700 1 -- 172.21.15.161:0/887737471 --> v1:172.21.15.161:6800/2381271555 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe178000bc0 con 0x7fe180033930 2024-02-13T03:13:49.643 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.637+0000 7fe18affd700 1 -- 172.21.15.161:0/887737471 <== mgr.14116 v1:172.21.15.161:6800/2381271555 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7fe178000bc0 con 0x7fe180033930 2024-02-13T03:13:49.643 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: Scheduled mgr update... 2024-02-13T03:13:49.646 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.641+0000 7fe19c09d700 1 -- 172.21.15.161:0/887737471 >> v1:172.21.15.161:6800/2381271555 conn(0x7fe180033930 legacy=0x7fe180035df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:49.646 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.641+0000 7fe19c09d700 1 -- 172.21.15.161:0/887737471 >> v1:172.21.15.161:6789/0 conn(0x7fe1940fb760 legacy=0x7fe1941014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:49.646 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.641+0000 7fe19c09d700 1 -- 172.21.15.161:0/887737471 shutdown_connections 2024-02-13T03:13:49.646 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.641+0000 7fe19c09d700 1 -- 172.21.15.161:0/887737471 >> 172.21.15.161:0/887737471 conn(0x7fe1940f6f40 msgr2=0x7fe1940f7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:49.646 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.641+0000 7fe19c09d700 1 -- 172.21.15.161:0/887737471 shutdown_connections 2024-02-13T03:13:49.646 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:49.641+0000 7fe19c09d700 1 -- 172.21.15.161:0/887737471 wait complete. 2024-02-13T03:13:50.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:50 smithi161 bash[19769]: audit 2024-02-13T03:13:49.638666+0000 mgr.y (mgr.14116) 13 : audit [DBG] from='client.14138 v1:172.21.15.161:0/887737471' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:13:50.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:50 smithi161 bash[19769]: cephadm 2024-02-13T03:13:49.639733+0000 mgr.y (mgr.14116) 14 : cephadm [INF] Saving service mgr spec with placement count:2 2024-02-13T03:13:50.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:50 smithi161 bash[19769]: audit 2024-02-13T03:13:49.642897+0000 mon.a (mon.0) 61 : audit [INF] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' 2024-02-13T03:13:51.229 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.221+0000 7fa24aac3700 1 Processor -- start 2024-02-13T03:13:51.229 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.221+0000 7fa24aac3700 1 -- start start 2024-02-13T03:13:51.229 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.221+0000 7fa24aac3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa2440febc0 con 0x7fa2440fbd00 2024-02-13T03:13:51.229 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.221+0000 7fa24885f700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa2440fbd00 0x7fa2440fc120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59552/0 (socket says 172.21.15.161:59552) 2024-02-13T03:13:51.229 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.221+0000 7fa24885f700 1 -- 172.21.15.161:0/2362677356 learned_addr learned my addr 172.21.15.161:0/2362677356 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:51.230 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.225+0000 7fa2437fe700 1 -- 172.21.15.161:0/2362677356 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060841641 0 0) 0x7fa2440febc0 con 0x7fa2440fbd00 2024-02-13T03:13:51.230 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.225+0000 7fa2437fe700 1 -- 172.21.15.161:0/2362677356 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2300033f0 con 0x7fa2440fbd00 2024-02-13T03:13:51.231 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.225+0000 7fa2437fe700 1 -- 172.21.15.161:0/2362677356 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1295335495 0 0) 0x7fa2300033f0 con 0x7fa2440fbd00 2024-02-13T03:13:51.231 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.225+0000 7fa2437fe700 1 -- 172.21.15.161:0/2362677356 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2440ffd60 con 0x7fa2440fbd00 2024-02-13T03:13:51.231 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.225+0000 7fa2437fe700 1 -- 172.21.15.161:0/2362677356 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa234001590 con 0x7fa2440fbd00 2024-02-13T03:13:51.231 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.225+0000 7fa2437fe700 1 -- 172.21.15.161:0/2362677356 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fa234001c50 con 0x7fa2440fbd00 2024-02-13T03:13:51.231 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.225+0000 7fa2437fe700 1 -- 172.21.15.161:0/2362677356 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa234005670 con 0x7fa2440fbd00 2024-02-13T03:13:51.231 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.225+0000 7fa24aac3700 1 -- 172.21.15.161:0/2362677356 >> v1:172.21.15.161:6789/0 conn(0x7fa2440fbd00 legacy=0x7fa2440fc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:51.232 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.225+0000 7fa24aac3700 1 -- 172.21.15.161:0/2362677356 shutdown_connections 2024-02-13T03:13:51.232 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.225+0000 7fa24aac3700 1 -- 172.21.15.161:0/2362677356 >> 172.21.15.161:0/2362677356 conn(0x7fa2440f5e30 msgr2=0x7fa2440f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:51.232 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.225+0000 7fa24aac3700 1 -- 172.21.15.161:0/2362677356 shutdown_connections 2024-02-13T03:13:51.232 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.225+0000 7fa24aac3700 1 -- 172.21.15.161:0/2362677356 wait complete. 2024-02-13T03:13:51.233 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa24aac3700 1 Processor -- start 2024-02-13T03:13:51.233 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa24aac3700 1 -- start start 2024-02-13T03:13:51.234 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa24aac3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa24410cb10 con 0x7fa2440fbd00 2024-02-13T03:13:51.234 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa24885f700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa2440fbd00 0x7fa24410c400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59564/0 (socket says 172.21.15.161:59564) 2024-02-13T03:13:51.234 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa24885f700 1 -- 172.21.15.161:0/3837282240 learned_addr learned my addr 172.21.15.161:0/3837282240 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:51.235 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa241f76700 1 -- 172.21.15.161:0/3837282240 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585058936 0 0) 0x7fa24410cb10 con 0x7fa2440fbd00 2024-02-13T03:13:51.235 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa241f76700 1 -- 172.21.15.161:0/3837282240 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa230005260 con 0x7fa2440fbd00 2024-02-13T03:13:51.235 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa241f76700 1 -- 172.21.15.161:0/3837282240 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 578516437 0 0) 0x7fa230005260 con 0x7fa2440fbd00 2024-02-13T03:13:51.237 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa241f76700 1 -- 172.21.15.161:0/3837282240 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa24410cb10 con 0x7fa2440fbd00 2024-02-13T03:13:51.237 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa241f76700 1 -- 172.21.15.161:0/3837282240 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa234000ff0 con 0x7fa2440fbd00 2024-02-13T03:13:51.237 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa241f76700 1 -- 172.21.15.161:0/3837282240 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1631616025 0 0) 0x7fa24410cb10 con 0x7fa2440fbd00 2024-02-13T03:13:51.237 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa241f76700 1 -- 172.21.15.161:0/3837282240 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24410cca0 con 0x7fa2440fbd00 2024-02-13T03:13:51.237 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa24aac3700 1 -- 172.21.15.161:0/3837282240 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24410cf50 con 0x7fa2440fbd00 2024-02-13T03:13:51.237 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa241f76700 1 -- 172.21.15.161:0/3837282240 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fa234004730 con 0x7fa2440fbd00 2024-02-13T03:13:51.237 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa24aac3700 1 -- 172.21.15.161:0/3837282240 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa244110c40 con 0x7fa2440fbd00 2024-02-13T03:13:51.237 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.229+0000 7fa241f76700 1 -- 172.21.15.161:0/3837282240 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa234006290 con 0x7fa2440fbd00 2024-02-13T03:13:51.238 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.233+0000 7fa241f76700 1 -- 172.21.15.161:0/3837282240 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 3044270204 0 0) 0x7fa234005950 con 0x7fa2440fbd00 2024-02-13T03:13:51.238 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.233+0000 7fa241f76700 1 -- 172.21.15.161:0/3837282240 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 2320478970 0 0) 0x7fa23403edc0 con 0x7fa2440fbd00 2024-02-13T03:13:51.238 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.233+0000 7fa24aac3700 1 -- 172.21.15.161:0/3837282240 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa244046d80 con 0x7fa2440fbd00 2024-02-13T03:13:51.247 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.241+0000 7fa241f76700 1 -- 172.21.15.161:0/3837282240 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fa234013b70 con 0x7fa2440fbd00 2024-02-13T03:13:51.573 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.565+0000 7fa24aac3700 1 -- 172.21.15.161:0/3837282240 --> v1:172.21.15.161:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fa2440ff920 con 0x7fa2440fbd00 2024-02-13T03:13:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.573+0000 7fa241f76700 1 -- 172.21.15.161:0/3837282240 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (unknown 2684808024 0 0) 0x7fa234013b70 con 0x7fa2440fbd00 2024-02-13T03:13:51.588 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.577+0000 7fa24aac3700 1 -- 172.21.15.161:0/3837282240 >> v1:172.21.15.161:6800/2381271555 conn(0x7fa230033800 legacy=0x7fa230035cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:51.588 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.577+0000 7fa24aac3700 1 -- 172.21.15.161:0/3837282240 >> v1:172.21.15.161:6789/0 conn(0x7fa2440fbd00 legacy=0x7fa24410c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:51.588 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.577+0000 7fa24aac3700 1 -- 172.21.15.161:0/3837282240 shutdown_connections 2024-02-13T03:13:51.588 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.577+0000 7fa24aac3700 1 -- 172.21.15.161:0/3837282240 >> 172.21.15.161:0/3837282240 conn(0x7fa2440f5e30 msgr2=0x7fa2440f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:51.589 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.581+0000 7fa24aac3700 1 -- 172.21.15.161:0/3837282240 shutdown_connections 2024-02-13T03:13:51.589 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:51.581+0000 7fa24aac3700 1 -- 172.21.15.161:0/3837282240 wait complete. 2024-02-13T03:13:52.909 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:52 smithi161 bash[19769]: audit 2024-02-13T03:13:51.579448+0000 mon.a (mon.0) 62 : audit [INF] from='client.? v1:172.21.15.161:0/3837282240' entity='client.admin' 2024-02-13T03:13:53.153 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.145+0000 7f81c3be2700 1 Processor -- start 2024-02-13T03:13:53.153 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.145+0000 7f81c3be2700 1 -- start start 2024-02-13T03:13:53.153 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.145+0000 7f81c3be2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f81bc0fb190 con 0x7f81bc0fa660 2024-02-13T03:13:53.153 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.145+0000 7f81c197e700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f81bc0fa660 0x7f81bc0faa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59570/0 (socket says 172.21.15.161:59570) 2024-02-13T03:13:53.153 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.145+0000 7f81c197e700 1 -- 172.21.15.161:0/206057127 learned_addr learned my addr 172.21.15.161:0/206057127 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:53.155 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.145+0000 7f81c097c700 1 -- 172.21.15.161:0/206057127 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282708354 0 0) 0x7f81bc0fb190 con 0x7f81bc0fa660 2024-02-13T03:13:53.155 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.145+0000 7f81c097c700 1 -- 172.21.15.161:0/206057127 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a80033f0 con 0x7f81bc0fa660 2024-02-13T03:13:53.155 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c097c700 1 -- 172.21.15.161:0/206057127 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 295842038 0 0) 0x7f81a80033f0 con 0x7f81bc0fa660 2024-02-13T03:13:53.155 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c097c700 1 -- 172.21.15.161:0/206057127 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc0ffbe0 con 0x7f81bc0fa660 2024-02-13T03:13:53.155 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c097c700 1 -- 172.21.15.161:0/206057127 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f81b8001590 con 0x7f81bc0fa660 2024-02-13T03:13:53.155 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c097c700 1 -- 172.21.15.161:0/206057127 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f81b8001c50 con 0x7f81bc0fa660 2024-02-13T03:13:53.155 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c097c700 1 -- 172.21.15.161:0/206057127 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f81b8005670 con 0x7f81bc0fa660 2024-02-13T03:13:53.155 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c3be2700 1 -- 172.21.15.161:0/206057127 >> v1:172.21.15.161:6789/0 conn(0x7f81bc0fa660 legacy=0x7f81bc0faa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:53.155 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c3be2700 1 -- 172.21.15.161:0/206057127 shutdown_connections 2024-02-13T03:13:53.155 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c3be2700 1 -- 172.21.15.161:0/206057127 >> 172.21.15.161:0/206057127 conn(0x7f81bc0f5e40 msgr2=0x7f81bc0f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:53.155 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c3be2700 1 -- 172.21.15.161:0/206057127 shutdown_connections 2024-02-13T03:13:53.156 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c3be2700 1 -- 172.21.15.161:0/206057127 wait complete. 2024-02-13T03:13:53.156 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c3be2700 1 Processor -- start 2024-02-13T03:13:53.157 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c3be2700 1 -- start start 2024-02-13T03:13:53.157 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c3be2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f81bc10acd0 con 0x7f81bc0fa660 2024-02-13T03:13:53.157 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c197e700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f81bc0fa660 0x7f81bc10f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59578/0 (socket says 172.21.15.161:59578) 2024-02-13T03:13:53.157 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81c197e700 1 -- 172.21.15.161:0/127541790 learned_addr learned my addr 172.21.15.161:0/127541790 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81b2ffd700 1 -- 172.21.15.161:0/127541790 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367672876 0 0) 0x7f81bc10acd0 con 0x7f81bc0fa660 2024-02-13T03:13:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.149+0000 7f81b2ffd700 1 -- 172.21.15.161:0/127541790 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a8005260 con 0x7f81bc0fa660 2024-02-13T03:13:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.153+0000 7f81b2ffd700 1 -- 172.21.15.161:0/127541790 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3334089593 0 0) 0x7f81a8005260 con 0x7f81bc0fa660 2024-02-13T03:13:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.153+0000 7f81b2ffd700 1 -- 172.21.15.161:0/127541790 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81bc10acd0 con 0x7f81bc0fa660 2024-02-13T03:13:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.153+0000 7f81b2ffd700 1 -- 172.21.15.161:0/127541790 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f81b80046b0 con 0x7f81bc0fa660 2024-02-13T03:13:53.159 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.153+0000 7f81b2ffd700 1 -- 172.21.15.161:0/127541790 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 736151145 0 0) 0x7f81bc10acd0 con 0x7f81bc0fa660 2024-02-13T03:13:53.159 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.153+0000 7f81b2ffd700 1 -- 172.21.15.161:0/127541790 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc06fbd0 con 0x7f81bc0fa660 2024-02-13T03:13:53.159 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.153+0000 7f81c3be2700 1 -- 172.21.15.161:0/127541790 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc06fe80 con 0x7f81bc0fa660 2024-02-13T03:13:53.159 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.153+0000 7f81c3be2700 1 -- 172.21.15.161:0/127541790 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc110c40 con 0x7f81bc0fa660 2024-02-13T03:13:53.159 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.153+0000 7f81b2ffd700 1 -- 172.21.15.161:0/127541790 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f81b8004d30 con 0x7f81bc0fa660 2024-02-13T03:13:53.159 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.153+0000 7f81b2ffd700 1 -- 172.21.15.161:0/127541790 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f81b80060b0 con 0x7f81bc0fa660 2024-02-13T03:13:53.159 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.153+0000 7f81b2ffd700 1 -- 172.21.15.161:0/127541790 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 3044270204 0 0) 0x7f81b800f990 con 0x7f81bc0fa660 2024-02-13T03:13:53.159 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.153+0000 7f81b2ffd700 1 -- 172.21.15.161:0/127541790 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 2320478970 0 0) 0x7f81b803fbb0 con 0x7f81bc0fa660 2024-02-13T03:13:53.160 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.153+0000 7f81c3be2700 1 -- 172.21.15.161:0/127541790 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a0004f50 con 0x7f81bc0fa660 2024-02-13T03:13:53.170 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.165+0000 7f81b2ffd700 1 -- 172.21.15.161:0/127541790 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f81b80110c0 con 0x7f81bc0fa660 2024-02-13T03:13:53.503 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.497+0000 7f81c3be2700 1 -- 172.21.15.161:0/127541790 --> v1:172.21.15.161:6789/0 -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f81a0005780 con 0x7f81bc0fa660 2024-02-13T03:13:53.510 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.505+0000 7f81b2ffd700 1 -- 172.21.15.161:0/127541790 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (unknown 1496729260 0 0) 0x7f81b80131a0 con 0x7f81bc0fa660 2024-02-13T03:13:53.515 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.509+0000 7f81c3be2700 1 -- 172.21.15.161:0/127541790 >> v1:172.21.15.161:6800/2381271555 conn(0x7f81a8033890 legacy=0x7f81a8035d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:53.516 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.509+0000 7f81c3be2700 1 -- 172.21.15.161:0/127541790 >> v1:172.21.15.161:6789/0 conn(0x7f81bc0fa660 legacy=0x7f81bc10f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:53.516 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.509+0000 7f81c3be2700 1 -- 172.21.15.161:0/127541790 shutdown_connections 2024-02-13T03:13:53.516 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.509+0000 7f81c3be2700 1 -- 172.21.15.161:0/127541790 >> 172.21.15.161:0/127541790 conn(0x7f81bc0f5e40 msgr2=0x7f81bc0f6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:53.516 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.509+0000 7f81c3be2700 1 -- 172.21.15.161:0/127541790 shutdown_connections 2024-02-13T03:13:53.517 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:53.509+0000 7f81c3be2700 1 -- 172.21.15.161:0/127541790 wait complete. 2024-02-13T03:13:54.043 INFO:teuthology.orchestra.run.smithi161.stderr:Enabling the dashboard module... 2024-02-13T03:13:54.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:54 smithi161 bash[19769]: audit 2024-02-13T03:13:53.349151+0000 mon.a (mon.0) 63 : audit [INF] from='mgr.14116 v1:172.21.15.161:0/3618531692' entity='mgr.y' 2024-02-13T03:13:54.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:54 smithi161 bash[19769]: cluster 2024-02-13T03:13:53.436027+0000 mgr.y (mgr.14116) 15 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:13:54.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:54 smithi161 bash[19769]: audit 2024-02-13T03:13:53.509961+0000 mon.a (mon.0) 64 : audit [INF] from='client.? v1:172.21.15.161:0/127541790' entity='client.admin' 2024-02-13T03:13:55.069 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.065+0000 7fa7926b2700 1 Processor -- start 2024-02-13T03:13:55.069 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.065+0000 7fa7926b2700 1 -- start start 2024-02-13T03:13:55.069 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.065+0000 7fa7926b2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa78c072080 con 0x7fa78c071c60 2024-02-13T03:13:55.069 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.065+0000 7fa78bfff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa78c071c60 0x7fa78c070200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59586/0 (socket says 172.21.15.161:59586) 2024-02-13T03:13:55.069 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.065+0000 7fa78bfff700 1 -- 172.21.15.161:0/2748402198 learned_addr learned my addr 172.21.15.161:0/2748402198 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:55.069 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.065+0000 7fa78affd700 1 -- 172.21.15.161:0/2748402198 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260888592 0 0) 0x7fa78c072080 con 0x7fa78c071c60 2024-02-13T03:13:55.069 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.065+0000 7fa78affd700 1 -- 172.21.15.161:0/2748402198 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7780033f0 con 0x7fa78c071c60 2024-02-13T03:13:55.069 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.065+0000 7fa78affd700 1 -- 172.21.15.161:0/2748402198 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3054150234 0 0) 0x7fa7780033f0 con 0x7fa78c071c60 2024-02-13T03:13:55.070 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.065+0000 7fa78affd700 1 -- 172.21.15.161:0/2748402198 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c070910 con 0x7fa78c071c60 2024-02-13T03:13:55.070 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.065+0000 7fa78affd700 1 -- 172.21.15.161:0/2748402198 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa774001590 con 0x7fa78c071c60 2024-02-13T03:13:55.070 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.065+0000 7fa78affd700 1 -- 172.21.15.161:0/2748402198 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fa774001d20 con 0x7fa78c071c60 2024-02-13T03:13:55.070 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.065+0000 7fa78affd700 1 -- 172.21.15.161:0/2748402198 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa774005740 con 0x7fa78c071c60 2024-02-13T03:13:55.071 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7926b2700 1 -- 172.21.15.161:0/2748402198 >> v1:172.21.15.161:6789/0 conn(0x7fa78c071c60 legacy=0x7fa78c070200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:55.071 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7926b2700 1 -- 172.21.15.161:0/2748402198 shutdown_connections 2024-02-13T03:13:55.071 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7926b2700 1 -- 172.21.15.161:0/2748402198 >> 172.21.15.161:0/2748402198 conn(0x7fa78c0f6f40 msgr2=0x7fa78c0f9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:55.071 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7926b2700 1 -- 172.21.15.161:0/2748402198 shutdown_connections 2024-02-13T03:13:55.071 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7926b2700 1 -- 172.21.15.161:0/2748402198 wait complete. 2024-02-13T03:13:55.071 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7926b2700 1 Processor -- start 2024-02-13T03:13:55.071 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7926b2700 1 -- start start 2024-02-13T03:13:55.071 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7926b2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa78c10ca60 con 0x7fa78c071c60 2024-02-13T03:13:55.072 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa78bfff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa78c071c60 0x7fa78c10c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59600/0 (socket says 172.21.15.161:59600) 2024-02-13T03:13:55.072 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa78bfff700 1 -- 172.21.15.161:0/160012315 learned_addr learned my addr 172.21.15.161:0/160012315 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:55.072 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383200146 0 0) 0x7fa78c10ca60 con 0x7fa78c071c60 2024-02-13T03:13:55.073 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa778005210 con 0x7fa78c071c60 2024-02-13T03:13:55.073 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2011032267 0 0) 0x7fa778005210 con 0x7fa78c071c60 2024-02-13T03:13:55.073 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa78c10ca60 con 0x7fa78c071c60 2024-02-13T03:13:55.073 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa774004870 con 0x7fa78c071c60 2024-02-13T03:13:55.074 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2431111468 0 0) 0x7fa78c10ca60 con 0x7fa78c071c60 2024-02-13T03:13:55.074 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c10cbf0 con 0x7fa78c071c60 2024-02-13T03:13:55.074 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7926b2700 1 -- 172.21.15.161:0/160012315 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78c10cea0 con 0x7fa78c071c60 2024-02-13T03:13:55.074 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.069+0000 7fa7926b2700 1 -- 172.21.15.161:0/160012315 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78c110b90 con 0x7fa78c071c60 2024-02-13T03:13:55.074 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.073+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fa774001160 con 0x7fa78c071c60 2024-02-13T03:13:55.074 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.073+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fa774006100 con 0x7fa78c071c60 2024-02-13T03:13:55.075 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.073+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 3044270204 0 0) 0x7fa77400f940 con 0x7fa78c071c60 2024-02-13T03:13:55.075 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.073+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 2320478970 0 0) 0x7fa77403fb30 con 0x7fa78c071c60 2024-02-13T03:13:55.077 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.073+0000 7fa7926b2700 1 -- 172.21.15.161:0/160012315 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76c004f50 con 0x7fa78c071c60 2024-02-13T03:13:55.082 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.081+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fa7740111f0 con 0x7fa78c071c60 2024-02-13T03:13:55.491 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:55.489+0000 7fa7926b2700 1 -- 172.21.15.161:0/160012315 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fa76c005200 con 0x7fa78c071c60 2024-02-13T03:13:56.498 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:56.497+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mgrmap(e 9) v1 ==== 38595+0+0 (unknown 747460767 0 0) 0x7fa77403f0a0 con 0x7fa78c071c60 2024-02-13T03:13:56.500 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:56.497+0000 7fa7897fa700 1 -- 172.21.15.161:0/160012315 <== mon.0 v1:172.21.15.161:6789/0 11 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (unknown 1498667528 0 0) 0x7fa774014ee0 con 0x7fa78c071c60 2024-02-13T03:13:56.506 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:56.505+0000 7fa7926b2700 1 -- 172.21.15.161:0/160012315 >> v1:172.21.15.161:6800/2381271555 conn(0x7fa778033890 legacy=0x7fa778035d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:56.507 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:56.505+0000 7fa7926b2700 1 -- 172.21.15.161:0/160012315 >> v1:172.21.15.161:6789/0 conn(0x7fa78c071c60 legacy=0x7fa78c10c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:56.507 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:56.505+0000 7fa7926b2700 1 -- 172.21.15.161:0/160012315 shutdown_connections 2024-02-13T03:13:56.507 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:56.505+0000 7fa7926b2700 1 -- 172.21.15.161:0/160012315 >> 172.21.15.161:0/160012315 conn(0x7fa78c0f6f40 msgr2=0x7fa78c0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:56.507 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:56.505+0000 7fa7926b2700 1 -- 172.21.15.161:0/160012315 shutdown_connections 2024-02-13T03:13:56.507 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:56.505+0000 7fa7926b2700 1 -- 172.21.15.161:0/160012315 wait complete. 2024-02-13T03:13:56.808 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:56 smithi161 bash[19769]: cluster 2024-02-13T03:13:55.436321+0000 mgr.y (mgr.14116) 16 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:13:56.808 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:56 smithi161 bash[19769]: audit 2024-02-13T03:13:55.491600+0000 mon.a (mon.0) 65 : audit [INF] from='client.? v1:172.21.15.161:0/160012315' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-13T03:13:56.808 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:56 smithi161 bash[20075]: ignoring --setuser ceph since I am not root 2024-02-13T03:13:56.808 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:56 smithi161 bash[20075]: ignoring --setgroup ceph since I am not root 2024-02-13T03:13:56.808 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:56 smithi161 bash[20075]: debug 2024-02-13T03:13:56.705+0000 7f2e6fe0f680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T03:13:57.175 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:56 smithi161 bash[20075]: debug 2024-02-13T03:13:56.805+0000 7f2e6fe0f680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T03:13:57.496 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:57 smithi161 bash[20075]: debug 2024-02-13T03:13:57.237+0000 7f2e6fe0f680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T03:13:57.924 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:57 smithi161 bash[19769]: audit 2024-02-13T03:13:56.500270+0000 mon.a (mon.0) 66 : audit [INF] from='client.? v1:172.21.15.161:0/160012315' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-13T03:13:57.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:57 smithi161 bash[19769]: cluster 2024-02-13T03:13:56.500406+0000 mon.a (mon.0) 67 : cluster [DBG] mgrmap e9: y(active, since 23s) 2024-02-13T03:13:58.248 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:58 smithi161 bash[20075]: debug 2024-02-13T03:13:58.149+0000 7f2e6fe0f680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T03:13:58.289 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.285+0000 7f322151d700 1 Processor -- start 2024-02-13T03:13:58.289 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.285+0000 7f322151d700 1 -- start start 2024-02-13T03:13:58.290 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.285+0000 7f322151d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f321c0720e0 con 0x7f321c071cc0 2024-02-13T03:13:58.290 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.285+0000 7f321affd700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f321c071cc0 0x7f321c070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54726/0 (socket says 172.21.15.161:54726) 2024-02-13T03:13:58.290 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.285+0000 7f321affd700 1 -- 172.21.15.161:0/1596594077 learned_addr learned my addr 172.21.15.161:0/1596594077 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:58.290 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.289+0000 7f3219ffb700 1 -- 172.21.15.161:0/1596594077 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698580929 0 0) 0x7f321c0720e0 con 0x7f321c071cc0 2024-02-13T03:13:58.291 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.289+0000 7f3219ffb700 1 -- 172.21.15.161:0/1596594077 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32040033f0 con 0x7f321c071cc0 2024-02-13T03:13:58.291 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.289+0000 7f3219ffb700 1 -- 172.21.15.161:0/1596594077 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3051766594 0 0) 0x7f32040033f0 con 0x7f321c071cc0 2024-02-13T03:13:58.291 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.289+0000 7f3219ffb700 1 -- 172.21.15.161:0/1596594077 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c070970 con 0x7f321c071cc0 2024-02-13T03:13:58.291 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.289+0000 7f3219ffb700 1 -- 172.21.15.161:0/1596594077 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f320c001590 con 0x7f321c071cc0 2024-02-13T03:13:58.291 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.289+0000 7f3219ffb700 1 -- 172.21.15.161:0/1596594077 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f320c001c50 con 0x7f321c071cc0 2024-02-13T03:13:58.291 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.289+0000 7f3219ffb700 1 -- 172.21.15.161:0/1596594077 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f320c005670 con 0x7f321c071cc0 2024-02-13T03:13:58.292 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.289+0000 7f322151d700 1 -- 172.21.15.161:0/1596594077 >> v1:172.21.15.161:6789/0 conn(0x7f321c071cc0 legacy=0x7f321c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:58.292 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.289+0000 7f322151d700 1 -- 172.21.15.161:0/1596594077 shutdown_connections 2024-02-13T03:13:58.292 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.289+0000 7f322151d700 1 -- 172.21.15.161:0/1596594077 >> 172.21.15.161:0/1596594077 conn(0x7f321c0f6f20 msgr2=0x7f321c0f9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:58.292 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.289+0000 7f322151d700 1 -- 172.21.15.161:0/1596594077 shutdown_connections 2024-02-13T03:13:58.293 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.289+0000 7f322151d700 1 -- 172.21.15.161:0/1596594077 wait complete. 2024-02-13T03:13:58.293 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f322151d700 1 Processor -- start 2024-02-13T03:13:58.294 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f322151d700 1 -- start start 2024-02-13T03:13:58.294 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f322151d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f321c0fbda0 con 0x7f321c071cc0 2024-02-13T03:13:58.294 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f321affd700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f321c071cc0 0x7f321c0fb690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54742/0 (socket says 172.21.15.161:54742) 2024-02-13T03:13:58.294 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f321affd700 1 -- 172.21.15.161:0/1742645567 learned_addr learned my addr 172.21.15.161:0/1742645567 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:13:58.295 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f3203fff700 1 -- 172.21.15.161:0/1742645567 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566154963 0 0) 0x7f321c0fbda0 con 0x7f321c071cc0 2024-02-13T03:13:58.295 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f3203fff700 1 -- 172.21.15.161:0/1742645567 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3204005260 con 0x7f321c071cc0 2024-02-13T03:13:58.295 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f3203fff700 1 -- 172.21.15.161:0/1742645567 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1163375496 0 0) 0x7f3204005260 con 0x7f321c071cc0 2024-02-13T03:13:58.295 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f3203fff700 1 -- 172.21.15.161:0/1742645567 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f321c0fbda0 con 0x7f321c071cc0 2024-02-13T03:13:58.295 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f3203fff700 1 -- 172.21.15.161:0/1742645567 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f320c000ff0 con 0x7f321c071cc0 2024-02-13T03:13:58.296 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f3203fff700 1 -- 172.21.15.161:0/1742645567 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1432999009 0 0) 0x7f321c0fbda0 con 0x7f321c071cc0 2024-02-13T03:13:58.296 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f3203fff700 1 -- 172.21.15.161:0/1742645567 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c0fbf30 con 0x7f321c071cc0 2024-02-13T03:13:58.296 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f322151d700 1 -- 172.21.15.161:0/1742645567 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321c0fc180 con 0x7f321c071cc0 2024-02-13T03:13:58.296 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f322151d700 1 -- 172.21.15.161:0/1742645567 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f321c0fdf30 con 0x7f321c071cc0 2024-02-13T03:13:58.296 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f3203fff700 1 -- 172.21.15.161:0/1742645567 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f320c004730 con 0x7f321c071cc0 2024-02-13T03:13:58.297 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f3203fff700 1 -- 172.21.15.161:0/1742645567 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f320c0061a0 con 0x7f321c071cc0 2024-02-13T03:13:58.297 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f3203fff700 1 -- 172.21.15.161:0/1742645567 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 9) v1 ==== 38595+0+0 (unknown 747460767 0 0) 0x7f320c00fab0 con 0x7f321c071cc0 2024-02-13T03:13:58.297 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.293+0000 7f3201ffb700 1 -- 172.21.15.161:0/1742645567 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31fc004f20 con 0x7f321c071cc0 2024-02-13T03:13:58.297 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.297+0000 7f3203fff700 1 -- 172.21.15.161:0/1742645567 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 2320478970 0 0) 0x7f320c03fce0 con 0x7f321c071cc0 2024-02-13T03:13:58.298 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.297+0000 7f321a7fc700 1 -- 172.21.15.161:0/1742645567 >> v1:172.21.15.161:6800/2381271555 conn(0x7f32040338e0 legacy=0x7f3204035da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/2381271555 2024-02-13T03:13:58.305 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.305+0000 7f3203fff700 1 -- 172.21.15.161:0/1742645567 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f320c011270 con 0x7f321c071cc0 2024-02-13T03:13:58.499 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:58 smithi161 bash[20075]: debug 2024-02-13T03:13:58.245+0000 7f2e6fe0f680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T03:13:58.501 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.501+0000 7f321a7fc700 1 -- 172.21.15.161:0/1742645567 >> v1:172.21.15.161:6800/2381271555 conn(0x7f32040338e0 legacy=0x7f3204035da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/2381271555 2024-02-13T03:13:58.713 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.709+0000 7f3201ffb700 1 -- 172.21.15.161:0/1742645567 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f31fc0059e0 con 0x7f321c071cc0 2024-02-13T03:13:58.714 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.713+0000 7f3203fff700 1 -- 172.21.15.161:0/1742645567 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (unknown 2005831594 0 1748205903) 0x7f320c013350 con 0x7f321c071cc0 2024-02-13T03:13:58.714 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: { 2024-02-13T03:13:58.714 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "epoch": 9, 2024-02-13T03:13:58.714 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "available": true, 2024-02-13T03:13:58.715 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "active_name": "y", 2024-02-13T03:13:58.715 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-13T03:13:58.715 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: } 2024-02-13T03:13:58.720 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.717+0000 7f3201ffb700 1 -- 172.21.15.161:0/1742645567 >> v1:172.21.15.161:6800/2381271555 conn(0x7f32040338e0 legacy=0x7f3204035da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-13T03:13:58.720 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.717+0000 7f3201ffb700 1 -- 172.21.15.161:0/1742645567 >> v1:172.21.15.161:6789/0 conn(0x7f321c071cc0 legacy=0x7f321c0fb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:13:58.720 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.717+0000 7f3201ffb700 1 -- 172.21.15.161:0/1742645567 shutdown_connections 2024-02-13T03:13:58.720 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.717+0000 7f3201ffb700 1 -- 172.21.15.161:0/1742645567 >> 172.21.15.161:0/1742645567 conn(0x7f321c0f6f20 msgr2=0x7f321c0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:13:58.720 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.717+0000 7f3201ffb700 1 -- 172.21.15.161:0/1742645567 shutdown_connections 2024-02-13T03:13:58.720 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:13:58.717+0000 7f3201ffb700 1 -- 172.21.15.161:0/1742645567 wait complete. 2024-02-13T03:13:58.765 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:58 smithi161 bash[20075]: debug 2024-02-13T03:13:58.497+0000 7f2e6fe0f680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T03:13:59.174 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:13:58 smithi161 bash[19769]: audit 2024-02-13T03:13:58.713665+0000 mon.a (mon.0) 68 : audit [DBG] from='client.? v1:172.21.15.161:0/1742645567' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-13T03:13:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:Waiting for the mgr to restart... 2024-02-13T03:13:59.262 INFO:teuthology.orchestra.run.smithi161.stderr:Waiting for mgr epoch 9... 2024-02-13T03:13:59.860 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:59 smithi161 bash[20075]: debug 2024-02-13T03:13:59.605+0000 7f2e6fe0f680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T03:13:59.860 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:59 smithi161 bash[20075]: debug 2024-02-13T03:13:59.765+0000 7f2e6fe0f680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T03:14:00.176 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:59 smithi161 bash[20075]: debug 2024-02-13T03:13:59.857+0000 7f2e6fe0f680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T03:14:00.176 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:13:59 smithi161 bash[20075]: debug 2024-02-13T03:13:59.965+0000 7f2e6fe0f680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T03:14:00.529 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:00 smithi161 bash[20075]: debug 2024-02-13T03:14:00.185+0000 7f2e6fe0f680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T03:14:00.757 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.753+0000 7f9158c9d700 1 Processor -- start 2024-02-13T03:14:00.758 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.753+0000 7f9158c9d700 1 -- start start 2024-02-13T03:14:00.758 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.753+0000 7f9158c9d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f91540720c0 con 0x7f9154071ca0 2024-02-13T03:14:00.758 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.753+0000 7f915259c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f9154071ca0 0x7f9154070240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54758/0 (socket says 172.21.15.161:54758) 2024-02-13T03:14:00.758 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.753+0000 7f915259c700 1 -- 172.21.15.161:0/1039365401 learned_addr learned my addr 172.21.15.161:0/1039365401 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:00.758 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9151d9b700 1 -- 172.21.15.161:0/1039365401 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167210888 0 0) 0x7f91540720c0 con 0x7f9154071ca0 2024-02-13T03:14:00.758 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9151d9b700 1 -- 172.21.15.161:0/1039365401 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91400033f0 con 0x7f9154071ca0 2024-02-13T03:14:00.759 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9151d9b700 1 -- 172.21.15.161:0/1039365401 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 875387868 0 0) 0x7f91400033f0 con 0x7f9154071ca0 2024-02-13T03:14:00.759 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9151d9b700 1 -- 172.21.15.161:0/1039365401 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9154070950 con 0x7f9154071ca0 2024-02-13T03:14:00.759 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9151d9b700 1 -- 172.21.15.161:0/1039365401 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f9144001590 con 0x7f9154071ca0 2024-02-13T03:14:00.759 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9151d9b700 1 -- 172.21.15.161:0/1039365401 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f9144001c50 con 0x7f9154071ca0 2024-02-13T03:14:00.760 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9151d9b700 1 -- 172.21.15.161:0/1039365401 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f9144005670 con 0x7f9154071ca0 2024-02-13T03:14:00.760 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9158c9d700 1 -- 172.21.15.161:0/1039365401 >> v1:172.21.15.161:6789/0 conn(0x7f9154071ca0 legacy=0x7f9154070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:00.760 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9158c9d700 1 -- 172.21.15.161:0/1039365401 shutdown_connections 2024-02-13T03:14:00.760 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9158c9d700 1 -- 172.21.15.161:0/1039365401 >> 172.21.15.161:0/1039365401 conn(0x7f91540f6f20 msgr2=0x7f91540f9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:00.760 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9158c9d700 1 -- 172.21.15.161:0/1039365401 shutdown_connections 2024-02-13T03:14:00.760 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9158c9d700 1 -- 172.21.15.161:0/1039365401 wait complete. 2024-02-13T03:14:00.761 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9158c9d700 1 Processor -- start 2024-02-13T03:14:00.761 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9158c9d700 1 -- start start 2024-02-13T03:14:00.761 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f9158c9d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f915410c9e0 con 0x7f9154071ca0 2024-02-13T03:14:00.761 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f915259c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f9154071ca0 0x7f915410c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54766/0 (socket says 172.21.15.161:54766) 2024-02-13T03:14:00.761 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f915259c700 1 -- 172.21.15.161:0/407515663 learned_addr learned my addr 172.21.15.161:0/407515663 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:00.762 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461774228 0 0) 0x7f915410c9e0 con 0x7f9154071ca0 2024-02-13T03:14:00.762 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.757+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9140005260 con 0x7f9154071ca0 2024-02-13T03:14:00.762 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.761+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2789212111 0 0) 0x7f9140005260 con 0x7f9154071ca0 2024-02-13T03:14:00.762 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.761+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f915410c9e0 con 0x7f9154071ca0 2024-02-13T03:14:00.763 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.761+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f9144000ff0 con 0x7f9154071ca0 2024-02-13T03:14:00.764 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.761+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3429900062 0 0) 0x7f915410c9e0 con 0x7f9154071ca0 2024-02-13T03:14:00.764 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.761+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915410cb70 con 0x7f9154071ca0 2024-02-13T03:14:00.764 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.761+0000 7f9158c9d700 1 -- 172.21.15.161:0/407515663 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915410ce20 con 0x7f9154071ca0 2024-02-13T03:14:00.764 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.761+0000 7f9158c9d700 1 -- 172.21.15.161:0/407515663 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9154110aa0 con 0x7f9154071ca0 2024-02-13T03:14:00.766 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.765+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f9144001850 con 0x7f9154071ca0 2024-02-13T03:14:00.766 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.765+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f9144006150 con 0x7f9154071ca0 2024-02-13T03:14:00.766 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.765+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 9) v1 ==== 38595+0+0 (unknown 747460767 0 0) 0x7f9144006390 con 0x7f9154071ca0 2024-02-13T03:14:00.767 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.765+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 --> v1:172.21.15.161:6800/2381271555 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9140037730 con 0x7f9140033a60 2024-02-13T03:14:00.767 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.765+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 2320478970 0 0) 0x7f914403fc80 con 0x7f9154071ca0 2024-02-13T03:14:00.767 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.765+0000 7f914bfff700 1 -- 172.21.15.161:0/407515663 >> v1:172.21.15.161:6800/2381271555 conn(0x7f9140033a60 legacy=0x7f9140035f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/2381271555 2024-02-13T03:14:00.845 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:00 smithi161 bash[20075]: debug 2024-02-13T03:14:00.545+0000 7f2e6fe0f680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T03:14:00.845 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:00 smithi161 bash[20075]: debug 2024-02-13T03:14:00.665+0000 7f2e6fe0f680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T03:14:00.970 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:00.969+0000 7f914bfff700 1 -- 172.21.15.161:0/407515663 >> v1:172.21.15.161:6800/2381271555 conn(0x7f9140033a60 legacy=0x7f9140035f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/2381271555 2024-02-13T03:14:01.374 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:01.373+0000 7f914bfff700 1 -- 172.21.15.161:0/407515663 >> v1:172.21.15.161:6800/2381271555 conn(0x7f9140033a60 legacy=0x7f9140035f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/2381271555 2024-02-13T03:14:01.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:01 smithi161 bash[20075]: debug 2024-02-13T03:14:01.529+0000 7f2e6fe0f680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T03:14:01.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:01 smithi161 bash[20075]: debug 2024-02-13T03:14:01.633+0000 7f2e6fe0f680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T03:14:01.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:01 smithi161 bash[20075]: debug 2024-02-13T03:14:01.745+0000 7f2e6fe0f680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T03:14:02.178 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:02.177+0000 7f914bfff700 1 -- 172.21.15.161:0/407515663 >> v1:172.21.15.161:6800/2381271555 conn(0x7f9140033a60 legacy=0x7f9140035f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/2381271555 2024-02-13T03:14:02.247 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:01 smithi161 bash[20075]: debug 2024-02-13T03:14:01.961+0000 7f2e6fe0f680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T03:14:02.572 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:02 smithi161 bash[20075]: debug 2024-02-13T03:14:02.245+0000 7f2e6fe0f680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T03:14:02.572 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:02 smithi161 bash[20075]: debug 2024-02-13T03:14:02.401+0000 7f2e6fe0f680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T03:14:02.889 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:02 smithi161 bash[20075]: debug 2024-02-13T03:14:02.569+0000 7f2e6fe0f680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T03:14:02.889 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:02 smithi161 bash[20075]: debug 2024-02-13T03:14:02.781+0000 7f2e6fe0f680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T03:14:02.916 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:02.913+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mgrmap(e 10) v1 ==== 38454+0+0 (unknown 1565972335 0 0) 0x7f914403f270 con 0x7f9154071ca0 2024-02-13T03:14:02.916 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:02.913+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 >> v1:172.21.15.161:6800/2381271555 conn(0x7f9140033a60 legacy=0x7f9140035f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-13T03:14:03.174 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:02 smithi161 bash[20075]: debug 2024-02-13T03:14:02.885+0000 7f2e6fe0f680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T03:14:03.174 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:03 smithi161 bash[20075]: Warning: Permanently added '172.21.15.161' (ECDSA) to the list of known hosts. 2024-02-13T03:14:03.174 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:03 smithi161 bash[19769]: cluster 2024-02-13T03:14:02.911207+0000 mon.a (mon.0) 69 : cluster [INF] Active manager daemon y restarted 2024-02-13T03:14:03.174 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:03 smithi161 bash[19769]: cluster 2024-02-13T03:14:02.911662+0000 mon.a (mon.0) 70 : cluster [INF] Activating manager daemon y 2024-02-13T03:14:03.174 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:03 smithi161 bash[19769]: cluster 2024-02-13T03:14:02.918682+0000 mon.a (mon.0) 71 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2024-02-13T03:14:03.174 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:03 smithi161 bash[19769]: cluster 2024-02-13T03:14:02.918932+0000 mon.a (mon.0) 72 : cluster [DBG] mgrmap e10: y(active, starting, since 0.00743579s) 2024-02-13T03:14:03.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:03 smithi161 bash[19769]: audit 2024-02-13T03:14:02.921124+0000 mon.a (mon.0) 73 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T03:14:03.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:03 smithi161 bash[19769]: audit 2024-02-13T03:14:02.922590+0000 mon.a (mon.0) 74 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T03:14:03.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:03 smithi161 bash[19769]: audit 2024-02-13T03:14:02.923756+0000 mon.a (mon.0) 75 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T03:14:03.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:03 smithi161 bash[19769]: audit 2024-02-13T03:14:02.924111+0000 mon.a (mon.0) 76 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T03:14:03.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:03 smithi161 bash[19769]: audit 2024-02-13T03:14:02.924433+0000 mon.a (mon.0) 77 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T03:14:03.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:03 smithi161 bash[19769]: cluster 2024-02-13T03:14:02.929058+0000 mon.a (mon.0) 78 : cluster [INF] Manager daemon y is now available 2024-02-13T03:14:03.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:03 smithi161 bash[19769]: audit 2024-02-13T03:14:03.062255+0000 mon.a (mon.0) 79 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:14:03.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:03 smithi161 bash[19769]: audit 2024-02-13T03:14:03.065195+0000 mon.a (mon.0) 80 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:14:03.918 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:03.913+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mgrmap(e 11) v1 ==== 38489+0+0 (unknown 3878769867 0 0) 0x7f9144006870 con 0x7f9154071ca0 2024-02-13T03:14:03.919 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:03.917+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 --> v1:172.21.15.161:6800/1187535987 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9140037730 con 0x7f914403f880 2024-02-13T03:14:03.921 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:03.917+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (unknown 0 0 4198994610) 0x7f9140037730 con 0x7f914403f880 2024-02-13T03:14:03.937 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:03.933+0000 7f9158c9d700 1 -- 172.21.15.161:0/407515663 --> v1:172.21.15.161:6800/1187535987 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f9154046d80 con 0x7f914403f880 2024-02-13T03:14:03.937 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:03.933+0000 7f914b7fe700 1 -- 172.21.15.161:0/407515663 <== mgr.14146 v1:172.21.15.161:6800/1187535987 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (unknown 0 0 3086460295) 0x7f9154046d80 con 0x7f914403f880 2024-02-13T03:14:03.937 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: { 2024-02-13T03:14:03.938 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "mgrmap_epoch": 11, 2024-02-13T03:14:03.938 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: "initialized": true 2024-02-13T03:14:03.939 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: } 2024-02-13T03:14:03.939 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:03.937+0000 7f9158c9d700 1 -- 172.21.15.161:0/407515663 >> v1:172.21.15.161:6800/1187535987 conn(0x7f914403f880 legacy=0x7f914003a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:03.939 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:03.937+0000 7f9158c9d700 1 -- 172.21.15.161:0/407515663 >> v1:172.21.15.161:6789/0 conn(0x7f9154071ca0 legacy=0x7f915410c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:03.939 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:03.937+0000 7f9158c9d700 1 -- 172.21.15.161:0/407515663 shutdown_connections 2024-02-13T03:14:03.939 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:03.937+0000 7f9158c9d700 1 -- 172.21.15.161:0/407515663 >> 172.21.15.161:0/407515663 conn(0x7f91540f6f20 msgr2=0x7f91540fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:03.939 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:03.937+0000 7f9158c9d700 1 -- 172.21.15.161:0/407515663 shutdown_connections 2024-02-13T03:14:03.939 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:03.937+0000 7f9158c9d700 1 -- 172.21.15.161:0/407515663 wait complete. 2024-02-13T03:14:04.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:04 smithi161 bash[19769]: audit 2024-02-13T03:14:03.115302+0000 mon.a (mon.0) 81 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T03:14:04.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:04 smithi161 bash[19769]: audit 2024-02-13T03:14:03.121623+0000 mon.a (mon.0) 82 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T03:14:04.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:04 smithi161 bash[19769]: cluster 2024-02-13T03:14:03.921858+0000 mon.a (mon.0) 83 : cluster [DBG] mgrmap e11: y(active, since 1.01035s) 2024-02-13T03:14:04.647 INFO:teuthology.orchestra.run.smithi161.stderr:mgr epoch 9 is available 2024-02-13T03:14:04.647 INFO:teuthology.orchestra.run.smithi161.stderr:Generating a dashboard self-signed certificate... 2024-02-13T03:14:05.379 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:05 smithi161 bash[19769]: audit 2024-02-13T03:14:03.920321+0000 mgr.y (mgr.14146) 1 : audit [DBG] from='client.14150 v1:172.21.15.161:0/407515663' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-13T03:14:05.380 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:05 smithi161 bash[19769]: audit 2024-02-13T03:14:03.937385+0000 mgr.y (mgr.14146) 2 : audit [DBG] from='client.14150 v1:172.21.15.161:0/407515663' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-13T03:14:05.688 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.685+0000 7f6544236700 1 Processor -- start 2024-02-13T03:14:05.688 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.685+0000 7f6544236700 1 -- start start 2024-02-13T03:14:05.688 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.685+0000 7f6544236700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f653c0fc220 con 0x7f653c06fc70 2024-02-13T03:14:05.689 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.685+0000 7f6541fd2700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f653c06fc70 0x7f653c100830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54824/0 (socket says 172.21.15.161:54824) 2024-02-13T03:14:05.689 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.685+0000 7f6541fd2700 1 -- 172.21.15.161:0/1145551500 learned_addr learned my addr 172.21.15.161:0/1145551500 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:05.689 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.685+0000 7f6540fd0700 1 -- 172.21.15.161:0/1145551500 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094910050 0 0) 0x7f653c0fc220 con 0x7f653c06fc70 2024-02-13T03:14:05.689 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.685+0000 7f6540fd0700 1 -- 172.21.15.161:0/1145551500 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65280033f0 con 0x7f653c06fc70 2024-02-13T03:14:05.690 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.685+0000 7f6540fd0700 1 -- 172.21.15.161:0/1145551500 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 781263829 0 0) 0x7f65280033f0 con 0x7f653c06fc70 2024-02-13T03:14:05.690 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.685+0000 7f6540fd0700 1 -- 172.21.15.161:0/1145551500 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c102f60 con 0x7f653c06fc70 2024-02-13T03:14:05.690 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.685+0000 7f6540fd0700 1 -- 172.21.15.161:0/1145551500 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f652c001590 con 0x7f653c06fc70 2024-02-13T03:14:05.690 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.685+0000 7f6540fd0700 1 -- 172.21.15.161:0/1145551500 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f652c001d20 con 0x7f653c06fc70 2024-02-13T03:14:05.690 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.685+0000 7f6540fd0700 1 -- 172.21.15.161:0/1145551500 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f652c0056d0 con 0x7f653c06fc70 2024-02-13T03:14:05.691 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.689+0000 7f6544236700 1 -- 172.21.15.161:0/1145551500 >> v1:172.21.15.161:6789/0 conn(0x7f653c06fc70 legacy=0x7f653c100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:05.691 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.689+0000 7f6544236700 1 -- 172.21.15.161:0/1145551500 shutdown_connections 2024-02-13T03:14:05.691 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.689+0000 7f6544236700 1 -- 172.21.15.161:0/1145551500 >> 172.21.15.161:0/1145551500 conn(0x7f653c0f5e50 msgr2=0x7f653c0f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:05.691 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.689+0000 7f6544236700 1 -- 172.21.15.161:0/1145551500 shutdown_connections 2024-02-13T03:14:05.691 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.689+0000 7f6544236700 1 -- 172.21.15.161:0/1145551500 wait complete. 2024-02-13T03:14:05.692 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.689+0000 7f6544236700 1 Processor -- start 2024-02-13T03:14:05.692 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.689+0000 7f6544236700 1 -- start start 2024-02-13T03:14:05.692 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.689+0000 7f6544236700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f653c10cad0 con 0x7f653c06fc70 2024-02-13T03:14:05.693 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.689+0000 7f6541fd2700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f653c06fc70 0x7f653c10c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:54838/0 (socket says 172.21.15.161:54838) 2024-02-13T03:14:05.693 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.689+0000 7f6541fd2700 1 -- 172.21.15.161:0/3586360179 learned_addr learned my addr 172.21.15.161:0/3586360179 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:05.694 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6532ffd700 1 -- 172.21.15.161:0/3586360179 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748238625 0 0) 0x7f653c10cad0 con 0x7f653c06fc70 2024-02-13T03:14:05.694 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6532ffd700 1 -- 172.21.15.161:0/3586360179 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6528005210 con 0x7f653c06fc70 2024-02-13T03:14:05.694 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6532ffd700 1 -- 172.21.15.161:0/3586360179 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 654689083 0 0) 0x7f6528005210 con 0x7f653c06fc70 2024-02-13T03:14:05.694 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6532ffd700 1 -- 172.21.15.161:0/3586360179 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f653c10cad0 con 0x7f653c06fc70 2024-02-13T03:14:05.695 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6532ffd700 1 -- 172.21.15.161:0/3586360179 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f652c000fe0 con 0x7f653c06fc70 2024-02-13T03:14:05.695 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6532ffd700 1 -- 172.21.15.161:0/3586360179 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 51604121 0 0) 0x7f653c10cad0 con 0x7f653c06fc70 2024-02-13T03:14:05.695 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6532ffd700 1 -- 172.21.15.161:0/3586360179 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c10cc60 con 0x7f653c06fc70 2024-02-13T03:14:05.695 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6544236700 1 -- 172.21.15.161:0/3586360179 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653c10cf10 con 0x7f653c06fc70 2024-02-13T03:14:05.696 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6544236700 1 -- 172.21.15.161:0/3586360179 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c110c00 con 0x7f653c06fc70 2024-02-13T03:14:05.697 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6532ffd700 1 -- 172.21.15.161:0/3586360179 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f652c004a60 con 0x7f653c06fc70 2024-02-13T03:14:05.697 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6532ffd700 1 -- 172.21.15.161:0/3586360179 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f652c0060c0 con 0x7f653c06fc70 2024-02-13T03:14:05.697 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6532ffd700 1 -- 172.21.15.161:0/3586360179 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 12) v1 ==== 38595+0+0 (unknown 3038144815 0 0) 0x7f652c00f9d0 con 0x7f653c06fc70 2024-02-13T03:14:05.697 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6532ffd700 1 -- 172.21.15.161:0/3586360179 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 544748005 0 0) 0x7f652c03ed90 con 0x7f653c06fc70 2024-02-13T03:14:05.697 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.693+0000 7f6544236700 1 -- 172.21.15.161:0/3586360179 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6520004f50 con 0x7f653c06fc70 2024-02-13T03:14:05.703 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:05.701+0000 7f6532ffd700 1 -- 172.21.15.161:0/3586360179 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f652c011320 con 0x7f653c06fc70 2024-02-13T03:14:06.032 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:06.029+0000 7f6544236700 1 -- 172.21.15.161:0/3586360179 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f6520000bc0 con 0x7f65280338e0 2024-02-13T03:14:06.115 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:06.113+0000 7f6532ffd700 1 -- 172.21.15.161:0/3586360179 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (unknown 0 0 3317752739) 0x7f6520000bc0 con 0x7f65280338e0 2024-02-13T03:14:06.115 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: Self-signed certificate created 2024-02-13T03:14:06.117 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:06.117+0000 7f6544236700 1 -- 172.21.15.161:0/3586360179 >> v1:172.21.15.161:6800/1187535987 conn(0x7f65280338e0 legacy=0x7f6528035da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:06.118 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:06.117+0000 7f6544236700 1 -- 172.21.15.161:0/3586360179 >> v1:172.21.15.161:6789/0 conn(0x7f653c06fc70 legacy=0x7f653c10c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:06.119 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:06.117+0000 7f6544236700 1 -- 172.21.15.161:0/3586360179 shutdown_connections 2024-02-13T03:14:06.119 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:06.117+0000 7f6544236700 1 -- 172.21.15.161:0/3586360179 >> 172.21.15.161:0/3586360179 conn(0x7f653c0f5e50 msgr2=0x7f653c0f6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:06.119 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:06.117+0000 7f6544236700 1 -- 172.21.15.161:0/3586360179 shutdown_connections 2024-02-13T03:14:06.119 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:06.117+0000 7f6544236700 1 -- 172.21.15.161:0/3586360179 wait complete. 2024-02-13T03:14:06.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:06 smithi161 bash[19769]: cluster 2024-02-13T03:14:05.129245+0000 mon.a (mon.0) 84 : cluster [DBG] mgrmap e12: y(active, since 2s) 2024-02-13T03:14:06.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:06 smithi161 bash[19769]: audit 2024-02-13T03:14:06.110255+0000 mon.a (mon.0) 85 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:06.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:06 smithi161 bash[19769]: audit 2024-02-13T03:14:06.114715+0000 mon.a (mon.0) 86 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:06.704 INFO:teuthology.orchestra.run.smithi161.stderr:Creating initial admin user... 2024-02-13T03:14:07.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:07 smithi161 bash[19769]: audit 2024-02-13T03:14:06.033583+0000 mgr.y (mgr.14146) 3 : audit [DBG] from='client.14156 v1:172.21.15.161:0/3586360179' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:14:07.761 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.757+0000 7f37f089f700 1 Processor -- start 2024-02-13T03:14:07.761 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.757+0000 7f37f089f700 1 -- start start 2024-02-13T03:14:07.761 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.757+0000 7f37f089f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f37ec101ed0 con 0x7f37ec0ff010 2024-02-13T03:14:07.761 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.757+0000 7f37ea59c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f37ec0ff010 0x7f37ec0ff430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:55686/0 (socket says 172.21.15.161:55686) 2024-02-13T03:14:07.762 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.757+0000 7f37ea59c700 1 -- 172.21.15.161:0/3577419417 learned_addr learned my addr 172.21.15.161:0/3577419417 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:07.762 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.761+0000 7f37e959a700 1 -- 172.21.15.161:0/3577419417 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560763685 0 0) 0x7f37ec101ed0 con 0x7f37ec0ff010 2024-02-13T03:14:07.762 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.761+0000 7f37e959a700 1 -- 172.21.15.161:0/3577419417 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37d80033f0 con 0x7f37ec0ff010 2024-02-13T03:14:07.763 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.761+0000 7f37e959a700 1 -- 172.21.15.161:0/3577419417 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2730947894 0 0) 0x7f37d80033f0 con 0x7f37ec0ff010 2024-02-13T03:14:07.763 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.761+0000 7f37e959a700 1 -- 172.21.15.161:0/3577419417 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec103070 con 0x7f37ec0ff010 2024-02-13T03:14:07.763 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.761+0000 7f37e959a700 1 -- 172.21.15.161:0/3577419417 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f37e4001910 con 0x7f37ec0ff010 2024-02-13T03:14:07.763 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.761+0000 7f37e959a700 1 -- 172.21.15.161:0/3577419417 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f37e4001fd0 con 0x7f37ec0ff010 2024-02-13T03:14:07.763 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.761+0000 7f37e959a700 1 -- 172.21.15.161:0/3577419417 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f37e40059f0 con 0x7f37ec0ff010 2024-02-13T03:14:07.763 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.761+0000 7f37f089f700 1 -- 172.21.15.161:0/3577419417 >> v1:172.21.15.161:6789/0 conn(0x7f37ec0ff010 legacy=0x7f37ec0ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:07.764 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.761+0000 7f37f089f700 1 -- 172.21.15.161:0/3577419417 shutdown_connections 2024-02-13T03:14:07.764 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.761+0000 7f37f089f700 1 -- 172.21.15.161:0/3577419417 >> 172.21.15.161:0/3577419417 conn(0x7f37ec071060 msgr2=0x7f37ec071490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:07.765 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.761+0000 7f37f089f700 1 -- 172.21.15.161:0/3577419417 shutdown_connections 2024-02-13T03:14:07.765 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.761+0000 7f37f089f700 1 -- 172.21.15.161:0/3577419417 wait complete. 2024-02-13T03:14:07.766 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37f089f700 1 Processor -- start 2024-02-13T03:14:07.766 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37f089f700 1 -- start start 2024-02-13T03:14:07.766 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37f089f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f37ec10cba0 con 0x7f37ec0ff010 2024-02-13T03:14:07.767 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37ea59c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f37ec0ff010 0x7f37ec10c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:55698/0 (socket says 172.21.15.161:55698) 2024-02-13T03:14:07.767 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37ea59c700 1 -- 172.21.15.161:0/3149362418 learned_addr learned my addr 172.21.15.161:0/3149362418 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:07.767 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37e37fe700 1 -- 172.21.15.161:0/3149362418 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306060416 0 0) 0x7f37ec10cba0 con 0x7f37ec0ff010 2024-02-13T03:14:07.767 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37e37fe700 1 -- 172.21.15.161:0/3149362418 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37d8005260 con 0x7f37ec0ff010 2024-02-13T03:14:07.768 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37e37fe700 1 -- 172.21.15.161:0/3149362418 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2324382911 0 0) 0x7f37d8005260 con 0x7f37ec0ff010 2024-02-13T03:14:07.768 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37e37fe700 1 -- 172.21.15.161:0/3149362418 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f37ec10cba0 con 0x7f37ec0ff010 2024-02-13T03:14:07.768 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37e37fe700 1 -- 172.21.15.161:0/3149362418 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f37e4004a30 con 0x7f37ec0ff010 2024-02-13T03:14:07.769 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37e37fe700 1 -- 172.21.15.161:0/3149362418 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3459856803 0 0) 0x7f37ec10cba0 con 0x7f37ec0ff010 2024-02-13T03:14:07.769 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37e37fe700 1 -- 172.21.15.161:0/3149362418 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec10cd30 con 0x7f37ec0ff010 2024-02-13T03:14:07.769 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37f089f700 1 -- 172.21.15.161:0/3149362418 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec10cfe0 con 0x7f37ec0ff010 2024-02-13T03:14:07.769 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37f089f700 1 -- 172.21.15.161:0/3149362418 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec110cd0 con 0x7f37ec0ff010 2024-02-13T03:14:07.769 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37e37fe700 1 -- 172.21.15.161:0/3149362418 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f37e4001bd0 con 0x7f37ec0ff010 2024-02-13T03:14:07.770 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.765+0000 7f37e37fe700 1 -- 172.21.15.161:0/3149362418 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f37e40064a0 con 0x7f37ec0ff010 2024-02-13T03:14:07.770 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.769+0000 7f37e37fe700 1 -- 172.21.15.161:0/3149362418 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 12) v1 ==== 38595+0+0 (unknown 3038144815 0 0) 0x7f37e400fd50 con 0x7f37ec0ff010 2024-02-13T03:14:07.770 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.769+0000 7f37e37fe700 1 -- 172.21.15.161:0/3149362418 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 544748005 0 0) 0x7f37e403f190 con 0x7f37ec0ff010 2024-02-13T03:14:07.771 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.769+0000 7f37f089f700 1 -- 172.21.15.161:0/3149362418 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37cc004f50 con 0x7f37ec0ff010 2024-02-13T03:14:07.777 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:07.773+0000 7f37e37fe700 1 -- 172.21.15.161:0/3149362418 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f37e40114c0 con 0x7f37ec0ff010 2024-02-13T03:14:08.134 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:08.133+0000 7f37f089f700 1 -- 172.21.15.161:0/3149362418 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f37cc002430 con 0x7f37d80338d0 2024-02-13T03:14:08.383 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:08.377+0000 7f37e37fe700 1 -- 172.21.15.161:0/3149362418 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (unknown 0 0 3520150408) 0x7f37cc002430 con 0x7f37d80338d0 2024-02-13T03:14:08.383 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$U/38zMz2FMoN75rBm55Mxu/Uyri29jStnVO.yOjXoQU/biOa7nAZa", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707794048, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-13T03:14:08.384 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:08.381+0000 7f37f089f700 1 -- 172.21.15.161:0/3149362418 >> v1:172.21.15.161:6800/1187535987 conn(0x7f37d80338d0 legacy=0x7f37d8035d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:08.385 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:08.381+0000 7f37f089f700 1 -- 172.21.15.161:0/3149362418 >> v1:172.21.15.161:6789/0 conn(0x7f37ec0ff010 legacy=0x7f37ec10c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:08.385 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:08.381+0000 7f37f089f700 1 -- 172.21.15.161:0/3149362418 shutdown_connections 2024-02-13T03:14:08.385 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:08.381+0000 7f37f089f700 1 -- 172.21.15.161:0/3149362418 >> 172.21.15.161:0/3149362418 conn(0x7f37ec071060 msgr2=0x7f37ec0faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:08.385 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:08.381+0000 7f37f089f700 1 -- 172.21.15.161:0/3149362418 shutdown_connections 2024-02-13T03:14:08.386 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:08.381+0000 7f37f089f700 1 -- 172.21.15.161:0/3149362418 wait complete. 2024-02-13T03:14:08.970 INFO:teuthology.orchestra.run.smithi161.stderr:Fetching dashboard port number... 2024-02-13T03:14:09.674 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:09 smithi161 bash[19769]: audit 2024-02-13T03:14:08.135416+0000 mgr.y (mgr.14146) 4 : audit [DBG] from='client.14158 v1:172.21.15.161:0/3149362418' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:14:09.674 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:09 smithi161 bash[19769]: audit 2024-02-13T03:14:08.379850+0000 mon.a (mon.0) 87 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:09.992 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.989+0000 7f804eb2c700 1 Processor -- start 2024-02-13T03:14:09.992 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.989+0000 7f804eb2c700 1 -- start start 2024-02-13T03:14:09.992 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.989+0000 7f804eb2c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f80480fa990 con 0x7f80480f9e60 2024-02-13T03:14:09.993 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.989+0000 7f804c8c8700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f80480f9e60 0x7f80480fa280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:55706/0 (socket says 172.21.15.161:55706) 2024-02-13T03:14:09.993 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.989+0000 7f804c8c8700 1 -- 172.21.15.161:0/1518122849 learned_addr learned my addr 172.21.15.161:0/1518122849 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:09.993 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.989+0000 7f80477fe700 1 -- 172.21.15.161:0/1518122849 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833018749 0 0) 0x7f803c001110 con 0x7f80480f9e60 2024-02-13T03:14:09.994 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.989+0000 7f80477fe700 1 -- 172.21.15.161:0/1518122849 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8038004500 con 0x7f80480f9e60 2024-02-13T03:14:09.994 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.989+0000 7f80477fe700 1 -- 172.21.15.161:0/1518122849 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2068610974 0 0) 0x7f8038004500 con 0x7f80480f9e60 2024-02-13T03:14:09.994 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.989+0000 7f80477fe700 1 -- 172.21.15.161:0/1518122849 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80480ff2a0 con 0x7f80480f9e60 2024-02-13T03:14:09.994 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.989+0000 7f80477fe700 1 -- 172.21.15.161:0/1518122849 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f803c001720 con 0x7f80480f9e60 2024-02-13T03:14:09.994 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.989+0000 7f80477fe700 1 -- 172.21.15.161:0/1518122849 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f803c004770 con 0x7f80480f9e60 2024-02-13T03:14:09.994 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.989+0000 7f80477fe700 1 -- 172.21.15.161:0/1518122849 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f803c0058e0 con 0x7f80480f9e60 2024-02-13T03:14:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f804eb2c700 1 -- 172.21.15.161:0/1518122849 >> v1:172.21.15.161:6789/0 conn(0x7f80480f9e60 legacy=0x7f80480fa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f804eb2c700 1 -- 172.21.15.161:0/1518122849 shutdown_connections 2024-02-13T03:14:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f804eb2c700 1 -- 172.21.15.161:0/1518122849 >> 172.21.15.161:0/1518122849 conn(0x7f80480f55a0 msgr2=0x7f80480f79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f804eb2c700 1 -- 172.21.15.161:0/1518122849 shutdown_connections 2024-02-13T03:14:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f804eb2c700 1 -- 172.21.15.161:0/1518122849 wait complete. 2024-02-13T03:14:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f804eb2c700 1 Processor -- start 2024-02-13T03:14:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f804eb2c700 1 -- start start 2024-02-13T03:14:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f804eb2c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f80480f7600 con 0x7f80480f9e60 2024-02-13T03:14:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f804c8c8700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f80480f9e60 0x7f80480f5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:55708/0 (socket says 172.21.15.161:55708) 2024-02-13T03:14:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f804c8c8700 1 -- 172.21.15.161:0/2445185517 learned_addr learned my addr 172.21.15.161:0/2445185517 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:09.997 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f8045ffb700 1 -- 172.21.15.161:0/2445185517 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648281789 0 0) 0x7f80480f7600 con 0x7f80480f9e60 2024-02-13T03:14:09.998 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f8045ffb700 1 -- 172.21.15.161:0/2445185517 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8038006320 con 0x7f80480f9e60 2024-02-13T03:14:09.998 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f8045ffb700 1 -- 172.21.15.161:0/2445185517 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1513634402 0 0) 0x7f8038006320 con 0x7f80480f9e60 2024-02-13T03:14:09.998 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f8045ffb700 1 -- 172.21.15.161:0/2445185517 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f80480f7600 con 0x7f80480f9e60 2024-02-13T03:14:09.998 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.993+0000 7f8045ffb700 1 -- 172.21.15.161:0/2445185517 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f803c000fe0 con 0x7f80480f9e60 2024-02-13T03:14:09.998 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.997+0000 7f8045ffb700 1 -- 172.21.15.161:0/2445185517 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3279979988 0 0) 0x7f80480f7600 con 0x7f80480f9e60 2024-02-13T03:14:09.998 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.997+0000 7f8045ffb700 1 -- 172.21.15.161:0/2445185517 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80480f7790 con 0x7f80480f9e60 2024-02-13T03:14:09.998 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.997+0000 7f804eb2c700 1 -- 172.21.15.161:0/2445185517 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80480f7a40 con 0x7f80480f9e60 2024-02-13T03:14:09.999 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.997+0000 7f804eb2c700 1 -- 172.21.15.161:0/2445185517 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f80480f6500 con 0x7f80480f9e60 2024-02-13T03:14:09.999 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.997+0000 7f8045ffb700 1 -- 172.21.15.161:0/2445185517 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f803c0012d0 con 0x7f80480f9e60 2024-02-13T03:14:09.999 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.997+0000 7f8045ffb700 1 -- 172.21.15.161:0/2445185517 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f803c006210 con 0x7f80480f9e60 2024-02-13T03:14:09.999 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.997+0000 7f8045ffb700 1 -- 172.21.15.161:0/2445185517 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7f803c00fa70 con 0x7f80480f9e60 2024-02-13T03:14:10.000 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.997+0000 7f804eb2c700 1 -- 172.21.15.161:0/2445185517 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c004f50 con 0x7f80480f9e60 2024-02-13T03:14:10.000 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:09.997+0000 7f8045ffb700 1 -- 172.21.15.161:0/2445185517 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 544748005 0 0) 0x7f803c03ec00 con 0x7f80480f9e60 2024-02-13T03:14:10.008 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:10.005+0000 7f8045ffb700 1 -- 172.21.15.161:0/2445185517 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f803c010f20 con 0x7f80480f9e60 2024-02-13T03:14:10.349 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:10.345+0000 7f804eb2c700 1 -- 172.21.15.161:0/2445185517 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f802c005780 con 0x7f80480f9e60 2024-02-13T03:14:10.350 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:10.349+0000 7f8045ffb700 1 -- 172.21.15.161:0/2445185517 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (unknown 3713421687 0 83753974) 0x7f803c013020 con 0x7f80480f9e60 2024-02-13T03:14:10.350 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 8443 2024-02-13T03:14:10.352 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:10.349+0000 7f804eb2c700 1 -- 172.21.15.161:0/2445185517 >> v1:172.21.15.161:6800/1187535987 conn(0x7f8038033940 legacy=0x7f8038035e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:10.352 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:10.349+0000 7f804eb2c700 1 -- 172.21.15.161:0/2445185517 >> v1:172.21.15.161:6789/0 conn(0x7f80480f9e60 legacy=0x7f80480f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:10.352 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:10.349+0000 7f804eb2c700 1 -- 172.21.15.161:0/2445185517 shutdown_connections 2024-02-13T03:14:10.352 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:10.349+0000 7f804eb2c700 1 -- 172.21.15.161:0/2445185517 >> 172.21.15.161:0/2445185517 conn(0x7f80480f55a0 msgr2=0x7f8048101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:10.353 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:10.349+0000 7f804eb2c700 1 -- 172.21.15.161:0/2445185517 shutdown_connections 2024-02-13T03:14:10.353 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:10.349+0000 7f804eb2c700 1 -- 172.21.15.161:0/2445185517 wait complete. 2024-02-13T03:14:10.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:10 smithi161 bash[19769]: cluster 2024-02-13T03:14:09.388481+0000 mon.a (mon.0) 88 : cluster [DBG] mgrmap e13: y(active, since 6s) 2024-02-13T03:14:10.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:10 smithi161 bash[19769]: audit 2024-02-13T03:14:10.026899+0000 mon.a (mon.0) 89 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:10.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:10 smithi161 bash[19769]: audit 2024-02-13T03:14:10.349613+0000 mon.a (mon.0) 90 : audit [DBG] from='client.? v1:172.21.15.161:0/2445185517' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-13T03:14:10.940 INFO:teuthology.orchestra.run.smithi161.stderr:firewalld does not appear to be present 2024-02-13T03:14:10.940 INFO:teuthology.orchestra.run.smithi161.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-13T03:14:10.944 INFO:teuthology.orchestra.run.smithi161.stderr:Ceph Dashboard is now available at: 2024-02-13T03:14:10.945 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-02-13T03:14:10.945 INFO:teuthology.orchestra.run.smithi161.stderr: URL: https://smithi161.front.sepia.ceph.com:8443/ 2024-02-13T03:14:10.945 INFO:teuthology.orchestra.run.smithi161.stderr: User: admin 2024-02-13T03:14:10.945 INFO:teuthology.orchestra.run.smithi161.stderr: Password: 1xgkv9cfy1 2024-02-13T03:14:10.945 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-02-13T03:14:10.945 INFO:teuthology.orchestra.run.smithi161.stderr:Enabling autotune for osd_memory_target 2024-02-13T03:14:12.032 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.029+0000 7f8693096700 1 Processor -- start 2024-02-13T03:14:12.032 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.029+0000 7f8693096700 1 -- start start 2024-02-13T03:14:12.033 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.029+0000 7f8693096700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f868c0fc290 con 0x7f868c0fb760 2024-02-13T03:14:12.033 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.029+0000 7f8690e32700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f868c0fb760 0x7f868c0fbb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:55724/0 (socket says 172.21.15.161:55724) 2024-02-13T03:14:12.033 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.029+0000 7f8690e32700 1 -- 172.21.15.161:0/282820642 learned_addr learned my addr 172.21.15.161:0/282820642 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:12.034 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.029+0000 7f86837fe700 1 -- 172.21.15.161:0/282820642 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243545616 0 0) 0x7f868c0fc290 con 0x7f868c0fb760 2024-02-13T03:14:12.034 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.029+0000 7f86837fe700 1 -- 172.21.15.161:0/282820642 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86740033f0 con 0x7f868c0fb760 2024-02-13T03:14:12.034 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f86837fe700 1 -- 172.21.15.161:0/282820642 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 876977969 0 0) 0x7f86740033f0 con 0x7f868c0fb760 2024-02-13T03:14:12.034 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f86837fe700 1 -- 172.21.15.161:0/282820642 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c100ce0 con 0x7f868c0fb760 2024-02-13T03:14:12.034 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f86837fe700 1 -- 172.21.15.161:0/282820642 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f867c001590 con 0x7f868c0fb760 2024-02-13T03:14:12.034 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f86837fe700 1 -- 172.21.15.161:0/282820642 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f867c001d20 con 0x7f868c0fb760 2024-02-13T03:14:12.034 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f86837fe700 1 -- 172.21.15.161:0/282820642 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f867c005740 con 0x7f868c0fb760 2024-02-13T03:14:12.035 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f8693096700 1 -- 172.21.15.161:0/282820642 >> v1:172.21.15.161:6789/0 conn(0x7f868c0fb760 legacy=0x7f868c0fbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:12.035 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f8693096700 1 -- 172.21.15.161:0/282820642 shutdown_connections 2024-02-13T03:14:12.035 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f8693096700 1 -- 172.21.15.161:0/282820642 >> 172.21.15.161:0/282820642 conn(0x7f868c0f6f40 msgr2=0x7f868c0f9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f8693096700 1 -- 172.21.15.161:0/282820642 shutdown_connections 2024-02-13T03:14:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f8693096700 1 -- 172.21.15.161:0/282820642 wait complete. 2024-02-13T03:14:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f8693096700 1 Processor -- start 2024-02-13T03:14:12.036 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f8693096700 1 -- start start 2024-02-13T03:14:12.037 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f8693096700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f868c10a880 con 0x7f868c0fb760 2024-02-13T03:14:12.037 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f8690e32700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f868c0fb760 0x7f868c10a170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:55728/0 (socket says 172.21.15.161:55728) 2024-02-13T03:14:12.037 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f8690e32700 1 -- 172.21.15.161:0/1692612716 learned_addr learned my addr 172.21.15.161:0/1692612716 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:12.037 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f8681ffb700 1 -- 172.21.15.161:0/1692612716 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813196230 0 0) 0x7f868c10a880 con 0x7f868c0fb760 2024-02-13T03:14:12.038 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f8681ffb700 1 -- 172.21.15.161:0/1692612716 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8674005210 con 0x7f868c0fb760 2024-02-13T03:14:12.038 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.033+0000 7f8681ffb700 1 -- 172.21.15.161:0/1692612716 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1711808217 0 0) 0x7f8674005210 con 0x7f868c0fb760 2024-02-13T03:14:12.038 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.037+0000 7f8681ffb700 1 -- 172.21.15.161:0/1692612716 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f868c10a880 con 0x7f868c0fb760 2024-02-13T03:14:12.038 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.037+0000 7f8681ffb700 1 -- 172.21.15.161:0/1692612716 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f867c000ed0 con 0x7f868c0fb760 2024-02-13T03:14:12.039 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.037+0000 7f8681ffb700 1 -- 172.21.15.161:0/1692612716 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2301626628 0 0) 0x7f868c10a880 con 0x7f868c0fb760 2024-02-13T03:14:12.039 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.037+0000 7f8681ffb700 1 -- 172.21.15.161:0/1692612716 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c10aa10 con 0x7f868c0fb760 2024-02-13T03:14:12.039 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.037+0000 7f8693096700 1 -- 172.21.15.161:0/1692612716 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c10acc0 con 0x7f868c0fb760 2024-02-13T03:14:12.039 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.037+0000 7f8693096700 1 -- 172.21.15.161:0/1692612716 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c10e9b0 con 0x7f868c0fb760 2024-02-13T03:14:12.040 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.037+0000 7f8681ffb700 1 -- 172.21.15.161:0/1692612716 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f867c0010f0 con 0x7f868c0fb760 2024-02-13T03:14:12.040 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.037+0000 7f8681ffb700 1 -- 172.21.15.161:0/1692612716 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f867c006040 con 0x7f868c0fb760 2024-02-13T03:14:12.040 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.037+0000 7f8681ffb700 1 -- 172.21.15.161:0/1692612716 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7f867c001880 con 0x7f868c0fb760 2024-02-13T03:14:12.040 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.037+0000 7f8681ffb700 1 -- 172.21.15.161:0/1692612716 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 544748005 0 0) 0x7f867c03ecf0 con 0x7f868c0fb760 2024-02-13T03:14:12.041 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.037+0000 7f8693096700 1 -- 172.21.15.161:0/1692612716 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8678004f50 con 0x7f868c0fb760 2024-02-13T03:14:12.048 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.045+0000 7f8681ffb700 1 -- 172.21.15.161:0/1692612716 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f867c005a30 con 0x7f868c0fb760 2024-02-13T03:14:12.368 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.365+0000 7f8693096700 1 -- 172.21.15.161:0/1692612716 --> v1:172.21.15.161:6789/0 -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f8678005780 con 0x7f868c0fb760 2024-02-13T03:14:12.368 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.365+0000 7f8681ffb700 1 -- 172.21.15.161:0/1692612716 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (unknown 689008156 0 0) 0x7f867c0016e0 con 0x7f868c0fb760 2024-02-13T03:14:12.371 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.369+0000 7f8693096700 1 -- 172.21.15.161:0/1692612716 >> v1:172.21.15.161:6800/1187535987 conn(0x7f8674033aa0 legacy=0x7f8674035f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:12.371 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.369+0000 7f8693096700 1 -- 172.21.15.161:0/1692612716 >> v1:172.21.15.161:6789/0 conn(0x7f868c0fb760 legacy=0x7f868c10a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:12.372 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.369+0000 7f8693096700 1 -- 172.21.15.161:0/1692612716 shutdown_connections 2024-02-13T03:14:12.372 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.369+0000 7f8693096700 1 -- 172.21.15.161:0/1692612716 >> 172.21.15.161:0/1692612716 conn(0x7f868c0f6f40 msgr2=0x7f868c0f7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:12.372 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.369+0000 7f8693096700 1 -- 172.21.15.161:0/1692612716 shutdown_connections 2024-02-13T03:14:12.372 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:12.369+0000 7f8693096700 1 -- 172.21.15.161:0/1692612716 wait complete. 2024-02-13T03:14:13.883 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf862b3700 1 Processor -- start 2024-02-13T03:14:13.883 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf862b3700 1 -- start start 2024-02-13T03:14:13.883 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf862b3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcf80072080 con 0x7fcf80071c60 2024-02-13T03:14:13.883 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf7ffff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fcf80071c60 0x7fcf80070200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:55734/0 (socket says 172.21.15.161:55734) 2024-02-13T03:14:13.884 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf7ffff700 1 -- 172.21.15.161:0/2514222993 learned_addr learned my addr 172.21.15.161:0/2514222993 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:13.884 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf7f7fe700 1 -- 172.21.15.161:0/2514222993 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705845118 0 0) 0x7fcf80072080 con 0x7fcf80071c60 2024-02-13T03:14:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf7f7fe700 1 -- 172.21.15.161:0/2514222993 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf6c0033f0 con 0x7fcf80071c60 2024-02-13T03:14:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf7f7fe700 1 -- 172.21.15.161:0/2514222993 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2948704890 0 0) 0x7fcf6c0033f0 con 0x7fcf80071c60 2024-02-13T03:14:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf7f7fe700 1 -- 172.21.15.161:0/2514222993 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf80070910 con 0x7fcf80071c60 2024-02-13T03:14:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf7f7fe700 1 -- 172.21.15.161:0/2514222993 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fcf70001590 con 0x7fcf80071c60 2024-02-13T03:14:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf7f7fe700 1 -- 172.21.15.161:0/2514222993 <== mon.0 v1:172.21.15.161:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fcf70001d20 con 0x7fcf80071c60 2024-02-13T03:14:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf7f7fe700 1 -- 172.21.15.161:0/2514222993 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fcf70005740 con 0x7fcf80071c60 2024-02-13T03:14:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf862b3700 1 -- 172.21.15.161:0/2514222993 >> v1:172.21.15.161:6789/0 conn(0x7fcf80071c60 legacy=0x7fcf80070200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf862b3700 1 -- 172.21.15.161:0/2514222993 shutdown_connections 2024-02-13T03:14:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf862b3700 1 -- 172.21.15.161:0/2514222993 >> 172.21.15.161:0/2514222993 conn(0x7fcf800f6f40 msgr2=0x7fcf800f9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:13.885 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf862b3700 1 -- 172.21.15.161:0/2514222993 shutdown_connections 2024-02-13T03:14:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.881+0000 7fcf862b3700 1 -- 172.21.15.161:0/2514222993 wait complete. 2024-02-13T03:14:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf862b3700 1 Processor -- start 2024-02-13T03:14:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf862b3700 1 -- start start 2024-02-13T03:14:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf862b3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcf8010ca60 con 0x7fcf80071c60 2024-02-13T03:14:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf7ffff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fcf80071c60 0x7fcf8010c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:55746/0 (socket says 172.21.15.161:55746) 2024-02-13T03:14:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf7ffff700 1 -- 172.21.15.161:0/2924541883 learned_addr learned my addr 172.21.15.161:0/2924541883 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf7df76700 1 -- 172.21.15.161:0/2924541883 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 182496779 0 0) 0x7fcf8010ca60 con 0x7fcf80071c60 2024-02-13T03:14:13.887 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf7df76700 1 -- 172.21.15.161:0/2924541883 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf6c005210 con 0x7fcf80071c60 2024-02-13T03:14:13.888 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf7df76700 1 -- 172.21.15.161:0/2924541883 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2234682739 0 0) 0x7fcf6c005210 con 0x7fcf80071c60 2024-02-13T03:14:13.888 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf7df76700 1 -- 172.21.15.161:0/2924541883 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcf8010ca60 con 0x7fcf80071c60 2024-02-13T03:14:13.888 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf7df76700 1 -- 172.21.15.161:0/2924541883 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fcf70004870 con 0x7fcf80071c60 2024-02-13T03:14:13.889 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf7df76700 1 -- 172.21.15.161:0/2924541883 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1120062160 0 0) 0x7fcf8010ca60 con 0x7fcf80071c60 2024-02-13T03:14:13.889 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf7df76700 1 -- 172.21.15.161:0/2924541883 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8010cbf0 con 0x7fcf80071c60 2024-02-13T03:14:13.889 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf862b3700 1 -- 172.21.15.161:0/2924541883 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8010cea0 con 0x7fcf80071c60 2024-02-13T03:14:13.889 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf862b3700 1 -- 172.21.15.161:0/2924541883 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf80110b90 con 0x7fcf80071c60 2024-02-13T03:14:13.889 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf7df76700 1 -- 172.21.15.161:0/2924541883 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fcf700010e0 con 0x7fcf80071c60 2024-02-13T03:14:13.890 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf7df76700 1 -- 172.21.15.161:0/2924541883 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7fcf70006080 con 0x7fcf80071c60 2024-02-13T03:14:13.890 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.885+0000 7fcf7df76700 1 -- 172.21.15.161:0/2924541883 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7fcf700062c0 con 0x7fcf80071c60 2024-02-13T03:14:13.890 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.889+0000 7fcf7df76700 1 -- 172.21.15.161:0/2924541883 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 544748005 0 0) 0x7fcf7003fcc0 con 0x7fcf80071c60 2024-02-13T03:14:13.890 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.889+0000 7fcf862b3700 1 -- 172.21.15.161:0/2924541883 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf60004f50 con 0x7fcf80071c60 2024-02-13T03:14:13.896 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:13.893+0000 7fcf7df76700 1 -- 172.21.15.161:0/2924541883 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fcf70007ad0 con 0x7fcf80071c60 2024-02-13T03:14:14.394 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:14.393+0000 7fcf862b3700 1 -- 172.21.15.161:0/2924541883 --> v1:172.21.15.161:6789/0 -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fcf60005780 con 0x7fcf80071c60 2024-02-13T03:14:14.396 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:14.393+0000 7fcf7df76700 1 -- 172.21.15.161:0/2924541883 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v24)=0 set mgr/dashboard/cluster/status v24) v1 ==== 153+0+0 (unknown 2554848587 0 0) 0x7fcf700132b0 con 0x7fcf80071c60 2024-02-13T03:14:14.399 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:14.397+0000 7fcf862b3700 1 -- 172.21.15.161:0/2924541883 >> v1:172.21.15.161:6800/1187535987 conn(0x7fcf6c0338e0 legacy=0x7fcf6c035da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:14.397+0000 7fcf862b3700 1 -- 172.21.15.161:0/2924541883 >> v1:172.21.15.161:6789/0 conn(0x7fcf80071c60 legacy=0x7fcf8010c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:14.397+0000 7fcf862b3700 1 -- 172.21.15.161:0/2924541883 shutdown_connections 2024-02-13T03:14:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:14.397+0000 7fcf862b3700 1 -- 172.21.15.161:0/2924541883 >> 172.21.15.161:0/2924541883 conn(0x7fcf800f6f40 msgr2=0x7fcf800fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:14.400 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:14.397+0000 7fcf862b3700 1 -- 172.21.15.161:0/2924541883 shutdown_connections 2024-02-13T03:14:14.401 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: 2024-02-13T03:14:14.397+0000 7fcf862b3700 1 -- 172.21.15.161:0/2924541883 wait complete. 2024-02-13T03:14:14.401 INFO:teuthology.orchestra.run.smithi161.stderr:/usr/bin/ceph: set mgr/dashboard/cluster/status 2024-02-13T03:14:14.986 INFO:teuthology.orchestra.run.smithi161.stderr:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-13T03:14:14.987 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-02-13T03:14:14.987 INFO:teuthology.orchestra.run.smithi161.stderr: sudo /usr/sbin/cephadm shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-13T03:14:14.987 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-02-13T03:14:14.987 INFO:teuthology.orchestra.run.smithi161.stderr:Or, if you are only running a single cluster on this host: 2024-02-13T03:14:14.987 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-02-13T03:14:14.987 INFO:teuthology.orchestra.run.smithi161.stderr: sudo /usr/sbin/cephadm shell 2024-02-13T03:14:14.987 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-02-13T03:14:14.987 INFO:teuthology.orchestra.run.smithi161.stderr:Please consider enabling telemetry to help improve Ceph: 2024-02-13T03:14:14.987 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-02-13T03:14:14.987 INFO:teuthology.orchestra.run.smithi161.stderr: ceph telemetry on 2024-02-13T03:14:14.987 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-02-13T03:14:14.987 INFO:teuthology.orchestra.run.smithi161.stderr:For more information see: 2024-02-13T03:14:14.988 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-02-13T03:14:14.988 INFO:teuthology.orchestra.run.smithi161.stderr: https://docs.ceph.com/en/pacific/mgr/telemetry/ 2024-02-13T03:14:14.988 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-02-13T03:14:14.988 INFO:teuthology.orchestra.run.smithi161.stderr:Bootstrap complete. 2024-02-13T03:14:15.022 INFO:tasks.cephadm:Fetching config... 2024-02-13T03:14:15.023 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:14:15.023 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-13T03:14:15.070 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-13T03:14:15.070 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:14:15.071 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-13T03:14:15.119 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-13T03:14:15.119 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:14:15.119 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-13T03:14:15.179 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-13T03:14:15.179 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:14:15.180 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-13T03:14:15.226 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-13T03:14:15.227 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC17oVVUIIh7asXf9BVbeHf8GEaNZ0Zty5Wrxx328tMSZ1L/xstI/rjeoHZxdEVT1j94rXkyWGqp3W8tMSTRdOiuT2CHp+9mjwgTg5GijqLYTi0ZmWJGDI2ALmUGe6lq/n9Yu0bU0wD4Bbh0kCio/Ue6cBBmYRKM0tEKMP7zus6sIy6oqIbnEjqgCR2a1jrIDEiZTChgNkPGTSy4grLWrRCoYFeJsEw4y6z8SbUTGkytH4Z1JJwZKcTBK33i+0BI0+nRgPGIhziB67Oaq7JP6z1M0sRVBsR+2gB6hN5vvYRT8074J7vv+WU9MUeeCh62xE3BWIbN+MLRzzK/QsCGTatV3gLzl94v3UsfPKGXdsqhgB6W4CYos512HwEM0/s+E+BKpGULlwTcJfSydM/gvOYwpIT6F78P/XL6XEo5rcxdcaV8GXNP50vLPVmyrAyWzdCp2s/bbhEJGhFvALBz8Ldaz+aNpWSl/QN50Iwgjv19dSy/02BuhLrf6zcV5fBh9k= ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-13T03:14:15.289 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC17oVVUIIh7asXf9BVbeHf8GEaNZ0Zty5Wrxx328tMSZ1L/xstI/rjeoHZxdEVT1j94rXkyWGqp3W8tMSTRdOiuT2CHp+9mjwgTg5GijqLYTi0ZmWJGDI2ALmUGe6lq/n9Yu0bU0wD4Bbh0kCio/Ue6cBBmYRKM0tEKMP7zus6sIy6oqIbnEjqgCR2a1jrIDEiZTChgNkPGTSy4grLWrRCoYFeJsEw4y6z8SbUTGkytH4Z1JJwZKcTBK33i+0BI0+nRgPGIhziB67Oaq7JP6z1M0sRVBsR+2gB6hN5vvYRT8074J7vv+WU9MUeeCh62xE3BWIbN+MLRzzK/QsCGTatV3gLzl94v3UsfPKGXdsqhgB6W4CYos512HwEM0/s+E+BKpGULlwTcJfSydM/gvOYwpIT6F78P/XL6XEo5rcxdcaV8GXNP50vLPVmyrAyWzdCp2s/bbhEJGhFvALBz8Ldaz+aNpWSl/QN50Iwgjv19dSy/02BuhLrf6zcV5fBh9k= ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:14:15.299 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC17oVVUIIh7asXf9BVbeHf8GEaNZ0Zty5Wrxx328tMSZ1L/xstI/rjeoHZxdEVT1j94rXkyWGqp3W8tMSTRdOiuT2CHp+9mjwgTg5GijqLYTi0ZmWJGDI2ALmUGe6lq/n9Yu0bU0wD4Bbh0kCio/Ue6cBBmYRKM0tEKMP7zus6sIy6oqIbnEjqgCR2a1jrIDEiZTChgNkPGTSy4grLWrRCoYFeJsEw4y6z8SbUTGkytH4Z1JJwZKcTBK33i+0BI0+nRgPGIhziB67Oaq7JP6z1M0sRVBsR+2gB6hN5vvYRT8074J7vv+WU9MUeeCh62xE3BWIbN+MLRzzK/QsCGTatV3gLzl94v3UsfPKGXdsqhgB6W4CYos512HwEM0/s+E+BKpGULlwTcJfSydM/gvOYwpIT6F78P/XL6XEo5rcxdcaV8GXNP50vLPVmyrAyWzdCp2s/bbhEJGhFvALBz8Ldaz+aNpWSl/QN50Iwgjv19dSy/02BuhLrf6zcV5fBh9k= ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-13T03:14:15.323 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC17oVVUIIh7asXf9BVbeHf8GEaNZ0Zty5Wrxx328tMSZ1L/xstI/rjeoHZxdEVT1j94rXkyWGqp3W8tMSTRdOiuT2CHp+9mjwgTg5GijqLYTi0ZmWJGDI2ALmUGe6lq/n9Yu0bU0wD4Bbh0kCio/Ue6cBBmYRKM0tEKMP7zus6sIy6oqIbnEjqgCR2a1jrIDEiZTChgNkPGTSy4grLWrRCoYFeJsEw4y6z8SbUTGkytH4Z1JJwZKcTBK33i+0BI0+nRgPGIhziB67Oaq7JP6z1M0sRVBsR+2gB6hN5vvYRT8074J7vv+WU9MUeeCh62xE3BWIbN+MLRzzK/QsCGTatV3gLzl94v3UsfPKGXdsqhgB6W4CYos512HwEM0/s+E+BKpGULlwTcJfSydM/gvOYwpIT6F78P/XL6XEo5rcxdcaV8GXNP50vLPVmyrAyWzdCp2s/bbhEJGhFvALBz8Ldaz+aNpWSl/QN50Iwgjv19dSy/02BuhLrf6zcV5fBh9k= ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:14:15.335 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-13T03:14:15.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:15 smithi161 bash[19769]: audit 2024-02-13T03:14:14.396747+0000 mon.a (mon.0) 91 : audit [INF] from='client.? v1:172.21.15.161:0/2924541883' entity='client.admin' 2024-02-13T03:14:19.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.864+0000 7efff60dd700 1 -- 172.21.15.161:0/2881207298 >> v1:172.21.15.161:6789/0 conn(0x7efff00f7570 legacy=0x7efff00f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.864+0000 7efff60dd700 1 -- 172.21.15.161:0/2881207298 shutdown_connections 2024-02-13T03:14:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.864+0000 7efff60dd700 1 -- 172.21.15.161:0/2881207298 >> 172.21.15.161:0/2881207298 conn(0x7efff00f2cd0 msgr2=0x7efff00f50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.864+0000 7efff60dd700 1 -- 172.21.15.161:0/2881207298 shutdown_connections 2024-02-13T03:14:19.870 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7efff60dd700 1 -- 172.21.15.161:0/2881207298 wait complete. 2024-02-13T03:14:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7efff60dd700 1 Processor -- start 2024-02-13T03:14:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7efff60dd700 1 -- start start 2024-02-13T03:14:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7efff60dd700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efff01071e0 con 0x7efff00f7570 2024-02-13T03:14:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7effef7fe700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7efff00f7570 0x7efff0106ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56302/0 (socket says 172.21.15.161:56302) 2024-02-13T03:14:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7effef7fe700 1 -- 172.21.15.161:0/964553343 learned_addr learned my addr 172.21.15.161:0/964553343 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7effed7fa700 1 -- 172.21.15.161:0/964553343 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3937503412 0 0) 0x7efff01071e0 con 0x7efff00f7570 2024-02-13T03:14:19.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7effed7fa700 1 -- 172.21.15.161:0/964553343 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effdc005210 con 0x7efff00f7570 2024-02-13T03:14:19.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7effed7fa700 1 -- 172.21.15.161:0/964553343 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1811224751 0 0) 0x7effdc005210 con 0x7efff00f7570 2024-02-13T03:14:19.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7effed7fa700 1 -- 172.21.15.161:0/964553343 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efff01071e0 con 0x7efff00f7570 2024-02-13T03:14:19.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7effed7fa700 1 -- 172.21.15.161:0/964553343 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7effe0000fe0 con 0x7efff00f7570 2024-02-13T03:14:19.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7effed7fa700 1 -- 172.21.15.161:0/964553343 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1327003530 0 0) 0x7efff01071e0 con 0x7efff00f7570 2024-02-13T03:14:19.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7effed7fa700 1 -- 172.21.15.161:0/964553343 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff0107370 con 0x7efff00f7570 2024-02-13T03:14:19.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7efff60dd700 1 -- 172.21.15.161:0/964553343 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff0107620 con 0x7efff00f7570 2024-02-13T03:14:19.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7efff60dd700 1 -- 172.21.15.161:0/964553343 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efff010b310 con 0x7efff00f7570 2024-02-13T03:14:19.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7effed7fa700 1 -- 172.21.15.161:0/964553343 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7effe0001680 con 0x7efff00f7570 2024-02-13T03:14:19.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7effed7fa700 1 -- 172.21.15.161:0/964553343 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7effe00039a0 con 0x7efff00f7570 2024-02-13T03:14:19.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.868+0000 7effed7fa700 1 -- 172.21.15.161:0/964553343 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7effe0003b20 con 0x7efff00f7570 2024-02-13T03:14:19.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.872+0000 7effed7fa700 1 -- 172.21.15.161:0/964553343 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 544748005 0 0) 0x7effe003fb70 con 0x7efff00f7570 2024-02-13T03:14:19.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.872+0000 7efff60dd700 1 -- 172.21.15.161:0/964553343 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd0004f50 con 0x7efff00f7570 2024-02-13T03:14:19.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:19.880+0000 7effed7fa700 1 -- 172.21.15.161:0/964553343 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7effe00117c0 con 0x7efff00f7570 2024-02-13T03:14:20.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:20.072+0000 7efff60dd700 1 -- 172.21.15.161:0/964553343 --> v1:172.21.15.161:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7effd0005780 con 0x7efff00f7570 2024-02-13T03:14:20.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:20.080+0000 7effed7fa700 1 -- 172.21.15.161:0/964553343 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (unknown 3028693289 0 0) 0x7effe0013140 con 0x7efff00f7570 2024-02-13T03:14:20.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:20.084+0000 7efff60dd700 1 -- 172.21.15.161:0/964553343 >> v1:172.21.15.161:6800/1187535987 conn(0x7effdc0339d0 legacy=0x7effdc035e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:20.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:20.084+0000 7efff60dd700 1 -- 172.21.15.161:0/964553343 >> v1:172.21.15.161:6789/0 conn(0x7efff00f7570 legacy=0x7efff0106ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:20.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:20.084+0000 7efff60dd700 1 -- 172.21.15.161:0/964553343 shutdown_connections 2024-02-13T03:14:20.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:20.084+0000 7efff60dd700 1 -- 172.21.15.161:0/964553343 >> 172.21.15.161:0/964553343 conn(0x7efff00f2cd0 msgr2=0x7efff00f50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:20.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:20.084+0000 7efff60dd700 1 -- 172.21.15.161:0/964553343 shutdown_connections 2024-02-13T03:14:20.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:20.084+0000 7efff60dd700 1 -- 172.21.15.161:0/964553343 wait complete. 2024-02-13T03:14:20.687 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-13T03:14:20.687 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-13T03:14:21.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:21 smithi161 bash[19769]: audit 2024-02-13T03:14:20.081857+0000 mon.a (mon.0) 92 : audit [INF] from='client.? v1:172.21.15.161:0/964553343' entity='client.admin' 2024-02-13T03:14:23.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:22 smithi161 bash[19769]: audit 2024-02-13T03:14:21.864037+0000 mon.a (mon.0) 93 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:23.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:22 smithi161 bash[19769]: audit 2024-02-13T03:14:22.161703+0000 mon.a (mon.0) 94 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:14:23.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:22 smithi161 bash[19769]: audit 2024-02-13T03:14:22.167500+0000 mon.a (mon.0) 95 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:23.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:22 smithi161 bash[19769]: audit 2024-02-13T03:14:22.169242+0000 mon.a (mon.0) 96 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:14:23.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:22 smithi161 bash[19769]: audit 2024-02-13T03:14:22.174726+0000 mon.a (mon.0) 97 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:23.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.636+0000 7f39865fd700 1 -- 172.21.15.161:0/2814472360 >> v1:172.21.15.161:6789/0 conn(0x7f39800f5bf0 legacy=0x7f39800f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:23.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.636+0000 7f39865fd700 1 -- 172.21.15.161:0/2814472360 shutdown_connections 2024-02-13T03:14:23.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.636+0000 7f39865fd700 1 -- 172.21.15.161:0/2814472360 >> 172.21.15.161:0/2814472360 conn(0x7f39800f1350 msgr2=0x7f39800f3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:23.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.636+0000 7f39865fd700 1 -- 172.21.15.161:0/2814472360 shutdown_connections 2024-02-13T03:14:23.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.636+0000 7f39865fd700 1 -- 172.21.15.161:0/2814472360 wait complete. 2024-02-13T03:14:23.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f39865fd700 1 Processor -- start 2024-02-13T03:14:23.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f39865fd700 1 -- start start 2024-02-13T03:14:23.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f39865fd700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f39800f2850 con 0x7f39800f5bf0 2024-02-13T03:14:23.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f397ffff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f39800f5bf0 0x7f3980109770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56308/0 (socket says 172.21.15.161:56308) 2024-02-13T03:14:23.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f397ffff700 1 -- 172.21.15.161:0/1832197874 learned_addr learned my addr 172.21.15.161:0/1832197874 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:23.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f397d7fa700 1 -- 172.21.15.161:0/1832197874 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743906879 0 0) 0x7f39800f2850 con 0x7f39800f5bf0 2024-02-13T03:14:23.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f397d7fa700 1 -- 172.21.15.161:0/1832197874 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3968005260 con 0x7f39800f5bf0 2024-02-13T03:14:23.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f397d7fa700 1 -- 172.21.15.161:0/1832197874 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2226691706 0 0) 0x7f3968005260 con 0x7f39800f5bf0 2024-02-13T03:14:23.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f397d7fa700 1 -- 172.21.15.161:0/1832197874 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f39800f2850 con 0x7f39800f5bf0 2024-02-13T03:14:23.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f397d7fa700 1 -- 172.21.15.161:0/1832197874 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f3970002060 con 0x7f39800f5bf0 2024-02-13T03:14:23.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f397d7fa700 1 -- 172.21.15.161:0/1832197874 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2777882966 0 0) 0x7f39800f2850 con 0x7f39800f5bf0 2024-02-13T03:14:23.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f397d7fa700 1 -- 172.21.15.161:0/1832197874 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39800f29e0 con 0x7f39800f5bf0 2024-02-13T03:14:23.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f39865fd700 1 -- 172.21.15.161:0/1832197874 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398010ae90 con 0x7f39800f5bf0 2024-02-13T03:14:23.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f39865fd700 1 -- 172.21.15.161:0/1832197874 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f398010b320 con 0x7f39800f5bf0 2024-02-13T03:14:23.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f397d7fa700 1 -- 172.21.15.161:0/1832197874 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f3970002850 con 0x7f39800f5bf0 2024-02-13T03:14:23.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f397d7fa700 1 -- 172.21.15.161:0/1832197874 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f3970003890 con 0x7f39800f5bf0 2024-02-13T03:14:23.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.640+0000 7f39865fd700 1 -- 172.21.15.161:0/1832197874 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396c004f50 con 0x7f39800f5bf0 2024-02-13T03:14:23.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.644+0000 7f397d7fa700 1 -- 172.21.15.161:0/1832197874 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7f3970003a10 con 0x7f39800f5bf0 2024-02-13T03:14:23.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.644+0000 7f397d7fa700 1 -- 172.21.15.161:0/1832197874 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 544748005 0 0) 0x7f397003ed40 con 0x7f39800f5bf0 2024-02-13T03:14:23.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.652+0000 7f397d7fa700 1 -- 172.21.15.161:0/1832197874 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f3970014a80 con 0x7f39800f5bf0 2024-02-13T03:14:23.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.840+0000 7f39865fd700 1 -- 172.21.15.161:0/1832197874 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f396c000bc0 con 0x7f3968033a00 2024-02-13T03:14:23.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.844+0000 7f397d7fa700 1 -- 172.21.15.161:0/1832197874 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f396c000bc0 con 0x7f3968033a00 2024-02-13T03:14:23.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.848+0000 7f39865fd700 1 -- 172.21.15.161:0/1832197874 >> v1:172.21.15.161:6800/1187535987 conn(0x7f3968033a00 legacy=0x7f3968035ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:23.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.848+0000 7f39865fd700 1 -- 172.21.15.161:0/1832197874 >> v1:172.21.15.161:6789/0 conn(0x7f39800f5bf0 legacy=0x7f3980109770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:23.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.848+0000 7f39865fd700 1 -- 172.21.15.161:0/1832197874 shutdown_connections 2024-02-13T03:14:23.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.848+0000 7f39865fd700 1 -- 172.21.15.161:0/1832197874 >> 172.21.15.161:0/1832197874 conn(0x7f39800f1350 msgr2=0x7f39800f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:23.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.848+0000 7f39865fd700 1 -- 172.21.15.161:0/1832197874 shutdown_connections 2024-02-13T03:14:23.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:23.848+0000 7f39865fd700 1 -- 172.21.15.161:0/1832197874 wait complete. 2024-02-13T03:14:24.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:23 smithi161 bash[19769]: cluster 2024-02-13T03:14:22.925473+0000 mgr.y (mgr.14146) 5 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:24.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:23 smithi161 bash[19769]: audit 2024-02-13T03:14:23.848441+0000 mon.a (mon.0) 98 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:24.419 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-02-13T03:14:24.419 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-02-13T03:14:24.419 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-02-13T03:14:24.426 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-02-13T03:14:24.426 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-13T03:14:24.475 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-02-13T03:14:24.475 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch host add smithi183 2024-02-13T03:14:25.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:24 smithi161 bash[19769]: audit 2024-02-13T03:14:23.893370+0000 mon.a (mon.0) 99 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:14:25.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:24 smithi161 bash[19769]: audit 2024-02-13T03:14:23.897736+0000 mon.a (mon.0) 100 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:14:25.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:24 smithi161 bash[19769]: audit 2024-02-13T03:14:23.898979+0000 mon.a (mon.0) 101 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:14:25.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:24 smithi161 bash[19769]: cephadm 2024-02-13T03:14:23.901454+0000 mgr.y (mgr.14146) 7 : cephadm [INF] Updating smithi161:/etc/ceph/ceph.conf 2024-02-13T03:14:25.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:24 smithi161 bash[19769]: cephadm 2024-02-13T03:14:23.920762+0000 mgr.y (mgr.14146) 8 : cephadm [INF] Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-02-13T03:14:25.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:24 smithi161 bash[19769]: audit 2024-02-13T03:14:23.970729+0000 mon.a (mon.0) 102 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:25.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:24 smithi161 bash[19769]: audit 2024-02-13T03:14:23.976606+0000 mon.a (mon.0) 103 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:26.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:25 smithi161 bash[19769]: cluster 2024-02-13T03:14:24.925897+0000 mgr.y (mgr.14146) 9 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:28.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:28 smithi161 bash[19769]: cluster 2024-02-13T03:14:26.926282+0000 mgr.y (mgr.14146) 10 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:30.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:30 smithi161 bash[19769]: cluster 2024-02-13T03:14:28.926672+0000 mgr.y (mgr.14146) 11 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:32.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:32 smithi161 bash[19769]: cluster 2024-02-13T03:14:30.927119+0000 mgr.y (mgr.14146) 12 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:34.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:34 smithi161 bash[19769]: cluster 2024-02-13T03:14:32.927501+0000 mgr.y (mgr.14146) 13 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:36.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:36 smithi161 bash[19769]: cluster 2024-02-13T03:14:34.927918+0000 mgr.y (mgr.14146) 14 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:38.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:38 smithi161 bash[19769]: cluster 2024-02-13T03:14:36.928323+0000 mgr.y (mgr.14146) 15 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:40.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:40 smithi161 bash[19769]: cluster 2024-02-13T03:14:38.928693+0000 mgr.y (mgr.14146) 16 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:42.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:42 smithi161 bash[19769]: cluster 2024-02-13T03:14:40.929084+0000 mgr.y (mgr.14146) 17 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:44.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:44 smithi161 bash[19769]: cluster 2024-02-13T03:14:42.929483+0000 mgr.y (mgr.14146) 18 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:46.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:46 smithi161 bash[19769]: cluster 2024-02-13T03:14:44.929888+0000 mgr.y (mgr.14146) 19 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:48.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:48 smithi161 bash[19769]: cluster 2024-02-13T03:14:46.930288+0000 mgr.y (mgr.14146) 20 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:49.794 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.794+0000 7f5ab3fff700 1 -- 172.21.15.183:0/1501468678 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f5aa4002e50 con 0x7f5ab40f7a70 2024-02-13T03:14:49.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.794+0000 7f5abb4f6700 1 -- 172.21.15.183:0/1501468678 >> v1:172.21.15.161:6789/0 conn(0x7f5ab40f7a70 legacy=0x7f5ab40f7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:49.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.794+0000 7f5abb4f6700 1 -- 172.21.15.183:0/1501468678 shutdown_connections 2024-02-13T03:14:49.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.794+0000 7f5abb4f6700 1 -- 172.21.15.183:0/1501468678 >> 172.21.15.183:0/1501468678 conn(0x7f5ab40f1350 msgr2=0x7f5ab40f3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:49.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.794+0000 7f5abb4f6700 1 -- 172.21.15.183:0/1501468678 shutdown_connections 2024-02-13T03:14:49.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.794+0000 7f5abb4f6700 1 -- 172.21.15.183:0/1501468678 wait complete. 2024-02-13T03:14:49.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.794+0000 7f5abb4f6700 1 Processor -- start 2024-02-13T03:14:49.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.794+0000 7f5abb4f6700 1 -- start start 2024-02-13T03:14:49.796 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.794+0000 7f5abb4f6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5ab40f6360 con 0x7f5ab40f7a70 2024-02-13T03:14:49.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5ab9292700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f5ab40f7a70 0x7f5ab4109740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.183:58374/0 (socket says 172.21.15.183:58374) 2024-02-13T03:14:49.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5ab9292700 1 -- 172.21.15.183:0/3591255525 learned_addr learned my addr 172.21.15.183:0/3591255525 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:14:49.797 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5ab27fc700 1 -- 172.21.15.183:0/3591255525 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903546945 0 0) 0x7f5ab40f6360 con 0x7f5ab40f7a70 2024-02-13T03:14:49.798 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5ab27fc700 1 -- 172.21.15.183:0/3591255525 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a9c005260 con 0x7f5ab40f7a70 2024-02-13T03:14:49.798 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5ab27fc700 1 -- 172.21.15.183:0/3591255525 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1175641680 0 0) 0x7f5a9c005260 con 0x7f5ab40f7a70 2024-02-13T03:14:49.798 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5ab27fc700 1 -- 172.21.15.183:0/3591255525 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ab40f6360 con 0x7f5ab40f7a70 2024-02-13T03:14:49.799 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5ab27fc700 1 -- 172.21.15.183:0/3591255525 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f5aa4002020 con 0x7f5ab40f7a70 2024-02-13T03:14:49.799 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5ab27fc700 1 -- 172.21.15.183:0/3591255525 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3704914958 0 0) 0x7f5ab40f6360 con 0x7f5ab40f7a70 2024-02-13T03:14:49.799 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5ab27fc700 1 -- 172.21.15.183:0/3591255525 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab40f64f0 con 0x7f5ab40f7a70 2024-02-13T03:14:49.799 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5abb4f6700 1 -- 172.21.15.183:0/3591255525 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab410ae60 con 0x7f5ab40f7a70 2024-02-13T03:14:49.799 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5abb4f6700 1 -- 172.21.15.183:0/3591255525 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab410b230 con 0x7f5ab40f7a70 2024-02-13T03:14:49.799 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5ab27fc700 1 -- 172.21.15.183:0/3591255525 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f5aa4002840 con 0x7f5ab40f7a70 2024-02-13T03:14:49.799 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5ab27fc700 1 -- 172.21.15.183:0/3591255525 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f5aa4003990 con 0x7f5ab40f7a70 2024-02-13T03:14:49.800 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5ab27fc700 1 -- 172.21.15.183:0/3591255525 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7f5aa4003bd0 con 0x7f5ab40f7a70 2024-02-13T03:14:49.800 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5abb4f6700 1 -- 172.21.15.183:0/3591255525 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa0004f50 con 0x7f5ab40f7a70 2024-02-13T03:14:49.800 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.798+0000 7f5ab27fc700 1 -- 172.21.15.183:0/3591255525 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 544748005 0 0) 0x7f5aa403fd30 con 0x7f5ab40f7a70 2024-02-13T03:14:49.811 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:49.810+0000 7f5ab27fc700 1 -- 172.21.15.183:0/3591255525 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f5aa40116f0 con 0x7f5ab40f7a70 2024-02-13T03:14:50.002 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:50.002+0000 7f5abb4f6700 1 -- 172.21.15.183:0/3591255525 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}) v1 -- 0x7f5aa0000bc0 con 0x7f5a9c033920 2024-02-13T03:14:50.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:50 smithi161 bash[19769]: cluster 2024-02-13T03:14:48.930672+0000 mgr.y (mgr.14146) 21 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:50.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:50 smithi161 bash[20075]: Warning: Permanently added 'smithi183,172.21.15.183' (ECDSA) to the list of known hosts. 2024-02-13T03:14:50.849 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:50.846+0000 7f5ab27fc700 1 -- 172.21.15.183:0/3591255525 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (unknown 0 0 696227864) 0x7f5aa0000bc0 con 0x7f5a9c033920 2024-02-13T03:14:50.849 INFO:teuthology.orchestra.run.smithi183.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-02-13T03:14:50.852 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:50.850+0000 7f5abb4f6700 1 -- 172.21.15.183:0/3591255525 >> v1:172.21.15.161:6800/1187535987 conn(0x7f5a9c033920 legacy=0x7f5a9c035de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:50.853 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:50.850+0000 7f5abb4f6700 1 -- 172.21.15.183:0/3591255525 >> v1:172.21.15.161:6789/0 conn(0x7f5ab40f7a70 legacy=0x7f5ab4109740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:50.853 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:50.850+0000 7f5abb4f6700 1 -- 172.21.15.183:0/3591255525 shutdown_connections 2024-02-13T03:14:50.853 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:50.850+0000 7f5abb4f6700 1 -- 172.21.15.183:0/3591255525 >> 172.21.15.183:0/3591255525 conn(0x7f5ab40f1350 msgr2=0x7f5ab40fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:50.853 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:50.850+0000 7f5abb4f6700 1 -- 172.21.15.183:0/3591255525 shutdown_connections 2024-02-13T03:14:50.853 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:50.850+0000 7f5abb4f6700 1 -- 172.21.15.183:0/3591255525 wait complete. 2024-02-13T03:14:51.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:14:50 smithi161 bash[20075]: Warning: Permanently added '172.21.15.183' (ECDSA) to the list of known hosts. 2024-02-13T03:14:51.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:51 smithi161 bash[19769]: audit 2024-02-13T03:14:50.003606+0000 mgr.y (mgr.14146) 22 : audit [DBG] from='client.14170 v1:172.21.15.183:0/3591255525' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:14:51.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:51 smithi161 bash[19769]: audit 2024-02-13T03:14:50.847488+0000 mon.a (mon.0) 104 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:51.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:51 smithi161 bash[19769]: audit 2024-02-13T03:14:50.878397+0000 mon.a (mon.0) 105 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:14:51.576 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch host ls --format=json 2024-02-13T03:14:52.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:52 smithi161 bash[19769]: cephadm 2024-02-13T03:14:50.848269+0000 mgr.y (mgr.14146) 23 : cephadm [INF] Added host smithi183 2024-02-13T03:14:52.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:52 smithi161 bash[19769]: cluster 2024-02-13T03:14:50.931278+0000 mgr.y (mgr.14146) 24 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:52.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:52 smithi161 bash[19769]: audit 2024-02-13T03:14:51.461744+0000 mon.a (mon.0) 106 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:54.122 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19cc0bb700 1 -- 172.21.15.183:0/1171634286 >> v1:172.21.15.161:6789/0 conn(0x7f19c40f9790 legacy=0x7f19c40f9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:54.122 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19cc0bb700 1 -- 172.21.15.183:0/1171634286 shutdown_connections 2024-02-13T03:14:54.122 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19cc0bb700 1 -- 172.21.15.183:0/1171634286 >> 172.21.15.183:0/1171634286 conn(0x7f19c4062360 msgr2=0x7f19c40f73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:54.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19cc0bb700 1 -- 172.21.15.183:0/1171634286 shutdown_connections 2024-02-13T03:14:54.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19cc0bb700 1 -- 172.21.15.183:0/1171634286 wait complete. 2024-02-13T03:14:54.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19cc0bb700 1 Processor -- start 2024-02-13T03:14:54.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19cc0bb700 1 -- start start 2024-02-13T03:14:54.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19cc0bb700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f19c41094c0 con 0x7f19c40f9790 2024-02-13T03:14:54.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19c9e57700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f19c40f9790 0x7f19c4108db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.183:58392/0 (socket says 172.21.15.183:58392) 2024-02-13T03:14:54.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19c9e57700 1 -- 172.21.15.183:0/788834765 learned_addr learned my addr 172.21.15.183:0/788834765 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:14:54.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19c2ffd700 1 -- 172.21.15.183:0/788834765 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391647746 0 0) 0x7f19c41094c0 con 0x7f19c40f9790 2024-02-13T03:14:54.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19c2ffd700 1 -- 172.21.15.183:0/788834765 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19ac005260 con 0x7f19c40f9790 2024-02-13T03:14:54.125 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19c2ffd700 1 -- 172.21.15.183:0/788834765 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1658569850 0 0) 0x7f19ac005260 con 0x7f19c40f9790 2024-02-13T03:14:54.125 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19c2ffd700 1 -- 172.21.15.183:0/788834765 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19c41094c0 con 0x7f19c40f9790 2024-02-13T03:14:54.125 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19c2ffd700 1 -- 172.21.15.183:0/788834765 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f19b8002060 con 0x7f19c40f9790 2024-02-13T03:14:54.125 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19c2ffd700 1 -- 172.21.15.183:0/788834765 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2965086441 0 0) 0x7f19c41094c0 con 0x7f19c40f9790 2024-02-13T03:14:54.125 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19c2ffd700 1 -- 172.21.15.183:0/788834765 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c4109650 con 0x7f19c40f9790 2024-02-13T03:14:54.125 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19cc0bb700 1 -- 172.21.15.183:0/788834765 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c410d270 con 0x7f19c40f9790 2024-02-13T03:14:54.126 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19cc0bb700 1 -- 172.21.15.183:0/788834765 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c410d5c0 con 0x7f19c40f9790 2024-02-13T03:14:54.126 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19c2ffd700 1 -- 172.21.15.183:0/788834765 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f19b8002780 con 0x7f19c40f9790 2024-02-13T03:14:54.126 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.122+0000 7f19c2ffd700 1 -- 172.21.15.183:0/788834765 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f19b8003780 con 0x7f19c40f9790 2024-02-13T03:14:54.126 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.126+0000 7f19cc0bb700 1 -- 172.21.15.183:0/788834765 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b0004f50 con 0x7f19c40f9790 2024-02-13T03:14:54.126 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.126+0000 7f19c2ffd700 1 -- 172.21.15.183:0/788834765 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7f19b8003900 con 0x7f19c40f9790 2024-02-13T03:14:54.127 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.126+0000 7f19c2ffd700 1 -- 172.21.15.183:0/788834765 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 544748005 0 0) 0x7f19b803ec10 con 0x7f19c40f9790 2024-02-13T03:14:54.137 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.134+0000 7f19c2ffd700 1 -- 172.21.15.183:0/788834765 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f19b8013ae0 con 0x7f19c40f9790 2024-02-13T03:14:54.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:53 smithi161 bash[19769]: audit 2024-02-13T03:14:52.767252+0000 mon.a (mon.0) 107 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:54.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:53 smithi161 bash[19769]: cluster 2024-02-13T03:14:52.931729+0000 mgr.y (mgr.14146) 25 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:54.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.338+0000 7f19cc0bb700 1 -- 172.21.15.183:0/788834765 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19b0000bc0 con 0x7f19ac033a90 2024-02-13T03:14:54.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.342+0000 7f19c2ffd700 1 -- 172.21.15.183:0/788834765 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (unknown 0 0 1639208350) 0x7f19b0000bc0 con 0x7f19ac033a90 2024-02-13T03:14:54.342 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-02-13T03:14:54.343 INFO:teuthology.orchestra.run.smithi183.stdout:[{"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-02-13T03:14:54.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.342+0000 7f19cc0bb700 1 -- 172.21.15.183:0/788834765 >> v1:172.21.15.161:6800/1187535987 conn(0x7f19ac033a90 legacy=0x7f19ac035f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:54.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.342+0000 7f19cc0bb700 1 -- 172.21.15.183:0/788834765 >> v1:172.21.15.161:6789/0 conn(0x7f19c40f9790 legacy=0x7f19c4108db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:54.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.342+0000 7f19cc0bb700 1 -- 172.21.15.183:0/788834765 shutdown_connections 2024-02-13T03:14:54.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.342+0000 7f19cc0bb700 1 -- 172.21.15.183:0/788834765 >> 172.21.15.183:0/788834765 conn(0x7f19c4062360 msgr2=0x7f19c40f73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:54.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.342+0000 7f19cc0bb700 1 -- 172.21.15.183:0/788834765 shutdown_connections 2024-02-13T03:14:54.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:14:54.342+0000 7f19cc0bb700 1 -- 172.21.15.183:0/788834765 wait complete. 2024-02-13T03:14:55.096 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-13T03:14:55.097 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd crush tunables default 2024-02-13T03:14:55.109 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:54 smithi161 bash[19769]: audit 2024-02-13T03:14:54.341048+0000 mgr.y (mgr.14146) 26 : audit [DBG] from='client.14172 v1:172.21.15.183:0/788834765' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T03:14:56.030 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:55 smithi161 bash[19769]: cluster 2024-02-13T03:14:54.932167+0000 mgr.y (mgr.14146) 27 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:57.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.712+0000 7ff1f662b700 1 -- 172.21.15.161:0/3862472480 >> v1:172.21.15.161:6789/0 conn(0x7ff1f006b190 legacy=0x7ff1f006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:57.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.712+0000 7ff1f662b700 1 -- 172.21.15.161:0/3862472480 shutdown_connections 2024-02-13T03:14:57.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.712+0000 7ff1f662b700 1 -- 172.21.15.161:0/3862472480 >> 172.21.15.161:0/3862472480 conn(0x7ff1f00f3190 msgr2=0x7ff1f00f55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:57.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.712+0000 7ff1f662b700 1 -- 172.21.15.161:0/3862472480 shutdown_connections 2024-02-13T03:14:57.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.712+0000 7ff1f662b700 1 -- 172.21.15.161:0/3862472480 wait complete. 2024-02-13T03:14:57.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.712+0000 7ff1f662b700 1 Processor -- start 2024-02-13T03:14:57.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.712+0000 7ff1f662b700 1 -- start start 2024-02-13T03:14:57.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.712+0000 7ff1f662b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff1f010f590 con 0x7ff1f006b190 2024-02-13T03:14:57.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.712+0000 7ff1effff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff1f006b190 0x7ff1f010ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:42212/0 (socket says 172.21.15.161:42212) 2024-02-13T03:14:57.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.712+0000 7ff1effff700 1 -- 172.21.15.161:0/2975916710 learned_addr learned my addr 172.21.15.161:0/2975916710 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:14:57.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1ed7fa700 1 -- 172.21.15.161:0/2975916710 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700648644 0 0) 0x7ff1f010f590 con 0x7ff1f006b190 2024-02-13T03:14:57.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1ed7fa700 1 -- 172.21.15.161:0/2975916710 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1d8005210 con 0x7ff1f006b190 2024-02-13T03:14:57.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1ed7fa700 1 -- 172.21.15.161:0/2975916710 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1921251252 0 0) 0x7ff1d8005210 con 0x7ff1f006b190 2024-02-13T03:14:57.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1ed7fa700 1 -- 172.21.15.161:0/2975916710 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1f010f590 con 0x7ff1f006b190 2024-02-13T03:14:57.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1ed7fa700 1 -- 172.21.15.161:0/2975916710 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7ff1e4000fe0 con 0x7ff1f006b190 2024-02-13T03:14:57.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1ed7fa700 1 -- 172.21.15.161:0/2975916710 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2050386927 0 0) 0x7ff1f010f590 con 0x7ff1f006b190 2024-02-13T03:14:57.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1ed7fa700 1 -- 172.21.15.161:0/2975916710 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f010f720 con 0x7ff1f006b190 2024-02-13T03:14:57.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1f662b700 1 -- 172.21.15.161:0/2975916710 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f010f9d0 con 0x7ff1f006b190 2024-02-13T03:14:57.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1f662b700 1 -- 172.21.15.161:0/2975916710 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f0111e50 con 0x7ff1f006b190 2024-02-13T03:14:57.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1ed7fa700 1 -- 172.21.15.161:0/2975916710 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7ff1e4001980 con 0x7ff1f006b190 2024-02-13T03:14:57.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1ed7fa700 1 -- 172.21.15.161:0/2975916710 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7ff1e4003870 con 0x7ff1f006b190 2024-02-13T03:14:57.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1ed7fa700 1 -- 172.21.15.161:0/2975916710 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7ff1e4003a70 con 0x7ff1f006b190 2024-02-13T03:14:57.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1f662b700 1 -- 172.21.15.161:0/2975916710 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc004f50 con 0x7ff1f006b190 2024-02-13T03:14:57.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.716+0000 7ff1ed7fa700 1 -- 172.21.15.161:0/2975916710 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 544748005 0 0) 0x7ff1e403efb0 con 0x7ff1f006b190 2024-02-13T03:14:57.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.724+0000 7ff1ed7fa700 1 -- 172.21.15.161:0/2975916710 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7ff1e40147e0 con 0x7ff1f006b190 2024-02-13T03:14:57.921 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:57.916+0000 7ff1f662b700 1 -- 172.21.15.161:0/2975916710 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff1dc005200 con 0x7ff1f006b190 2024-02-13T03:14:58.128 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:58.124+0000 7ff1ed7fa700 1 -- 172.21.15.161:0/2975916710 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (unknown 3126668360 0 0) 0x7ff1e4012c50 con 0x7ff1f006b190 2024-02-13T03:14:58.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:58.128+0000 7ff1f662b700 1 -- 172.21.15.161:0/2975916710 >> v1:172.21.15.161:6800/1187535987 conn(0x7ff1d80338d0 legacy=0x7ff1d8035d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:58.132 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:58.128+0000 7ff1f662b700 1 -- 172.21.15.161:0/2975916710 >> v1:172.21.15.161:6789/0 conn(0x7ff1f006b190 legacy=0x7ff1f010ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:14:58.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:58.128+0000 7ff1f662b700 1 -- 172.21.15.161:0/2975916710 shutdown_connections 2024-02-13T03:14:58.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:58.128+0000 7ff1f662b700 1 -- 172.21.15.161:0/2975916710 >> 172.21.15.161:0/2975916710 conn(0x7ff1f00f3190 msgr2=0x7ff1f00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:14:58.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:58.128+0000 7ff1f662b700 1 -- 172.21.15.161:0/2975916710 shutdown_connections 2024-02-13T03:14:58.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:14:58.128+0000 7ff1f662b700 1 -- 172.21.15.161:0/2975916710 wait complete. 2024-02-13T03:14:58.134 INFO:teuthology.orchestra.run.smithi161.stderr:adjusted tunables profile to default 2024-02-13T03:14:58.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:58 smithi161 bash[19769]: cluster 2024-02-13T03:14:56.932598+0000 mgr.y (mgr.14146) 28 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:14:58.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:58 smithi161 bash[19769]: audit 2024-02-13T03:14:57.921287+0000 mon.a (mon.0) 108 : audit [INF] from='client.? v1:172.21.15.161:0/2975916710' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-13T03:14:58.701 INFO:tasks.cephadm:Adding mon.a on smithi161 2024-02-13T03:14:58.701 INFO:tasks.cephadm:Adding mon.c on smithi161 2024-02-13T03:14:58.701 INFO:tasks.cephadm:Adding mon.b on smithi183 2024-02-13T03:14:58.701 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch apply mon '3;smithi161:[v1:172.21.15.161:6789]=a;smithi161:[v1:172.21.15.161:6790]=c;smithi183:[v1:172.21.15.183:6789]=b' 2024-02-13T03:14:59.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:59 smithi161 bash[19769]: audit 2024-02-13T03:14:58.128207+0000 mon.a (mon.0) 109 : audit [INF] from='client.? v1:172.21.15.161:0/2975916710' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-13T03:14:59.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:59 smithi161 bash[19769]: cluster 2024-02-13T03:14:58.128328+0000 mon.a (mon.0) 110 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-13T03:14:59.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:59 smithi161 bash[19769]: audit 2024-02-13T03:14:58.243542+0000 mon.a (mon.0) 111 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:59.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:59 smithi161 bash[19769]: audit 2024-02-13T03:14:58.541148+0000 mon.a (mon.0) 112 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:59.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:59 smithi161 bash[19769]: audit 2024-02-13T03:14:58.542699+0000 mon.a (mon.0) 113 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:14:59.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:59 smithi161 bash[19769]: audit 2024-02-13T03:14:58.544620+0000 mon.a (mon.0) 114 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:14:59.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:59 smithi161 bash[19769]: audit 2024-02-13T03:14:58.546168+0000 mon.a (mon.0) 115 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:14:59.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:59 smithi161 bash[19769]: cephadm 2024-02-13T03:14:58.548555+0000 mgr.y (mgr.14146) 29 : cephadm [INF] Updating smithi183:/etc/ceph/ceph.conf 2024-02-13T03:14:59.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:59 smithi161 bash[19769]: audit 2024-02-13T03:14:58.600525+0000 mon.a (mon.0) 116 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:14:59.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:14:59 smithi161 bash[19769]: audit 2024-02-13T03:14:58.606353+0000 mon.a (mon.0) 117 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:00.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:00 smithi161 bash[19769]: cephadm 2024-02-13T03:14:58.570297+0000 mgr.y (mgr.14146) 30 : cephadm [INF] Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-02-13T03:15:00.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:00 smithi161 bash[19769]: cluster 2024-02-13T03:14:58.933021+0000 mgr.y (mgr.14146) 31 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:01.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.369+0000 7f288837d700 1 -- 172.21.15.183:0/3039907826 >> v1:172.21.15.161:6789/0 conn(0x7f28800fc500 legacy=0x7f28800fc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:01.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.369+0000 7f288837d700 1 -- 172.21.15.183:0/3039907826 shutdown_connections 2024-02-13T03:15:01.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.369+0000 7f288837d700 1 -- 172.21.15.183:0/3039907826 >> 172.21.15.183:0/3039907826 conn(0x7f288006c390 msgr2=0x7f288006c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:15:01.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.369+0000 7f288837d700 1 -- 172.21.15.183:0/3039907826 shutdown_connections 2024-02-13T03:15:01.373 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.369+0000 7f288837d700 1 -- 172.21.15.183:0/3039907826 wait complete. 2024-02-13T03:15:01.374 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f288837d700 1 Processor -- start 2024-02-13T03:15:01.374 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f288837d700 1 -- start start 2024-02-13T03:15:01.374 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f288837d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f288010d260 con 0x7f28800fc500 2024-02-13T03:15:01.374 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f2886119700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f28800fc500 0x7f288010cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.183:59312/0 (socket says 172.21.15.183:59312) 2024-02-13T03:15:01.374 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f2886119700 1 -- 172.21.15.183:0/1566500195 learned_addr learned my addr 172.21.15.183:0/1566500195 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:15:01.375 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f28777fe700 1 -- 172.21.15.183:0/1566500195 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241268545 0 0) 0x7f288010d260 con 0x7f28800fc500 2024-02-13T03:15:01.375 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f28777fe700 1 -- 172.21.15.183:0/1566500195 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f286c005210 con 0x7f28800fc500 2024-02-13T03:15:01.376 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f28777fe700 1 -- 172.21.15.183:0/1566500195 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2226816774 0 0) 0x7f286c005210 con 0x7f28800fc500 2024-02-13T03:15:01.376 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f28777fe700 1 -- 172.21.15.183:0/1566500195 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f288010d260 con 0x7f28800fc500 2024-02-13T03:15:01.376 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f28777fe700 1 -- 172.21.15.183:0/1566500195 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f287c000fe0 con 0x7f28800fc500 2024-02-13T03:15:01.376 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f28777fe700 1 -- 172.21.15.183:0/1566500195 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3080304103 0 0) 0x7f288010d260 con 0x7f28800fc500 2024-02-13T03:15:01.376 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f28777fe700 1 -- 172.21.15.183:0/1566500195 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288010d3f0 con 0x7f28800fc500 2024-02-13T03:15:01.376 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f288837d700 1 -- 172.21.15.183:0/1566500195 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288010d6a0 con 0x7f28800fc500 2024-02-13T03:15:01.377 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f288837d700 1 -- 172.21.15.183:0/1566500195 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f288010fba0 con 0x7f28800fc500 2024-02-13T03:15:01.377 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f28777fe700 1 -- 172.21.15.183:0/1566500195 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f287c001940 con 0x7f28800fc500 2024-02-13T03:15:01.377 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f28777fe700 1 -- 172.21.15.183:0/1566500195 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f287c003790 con 0x7f28800fc500 2024-02-13T03:15:01.378 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.373+0000 7f288837d700 1 -- 172.21.15.183:0/1566500195 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2864004f50 con 0x7f28800fc500 2024-02-13T03:15:01.378 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.377+0000 7f28777fe700 1 -- 172.21.15.183:0/1566500195 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7f287c0022b0 con 0x7f28800fc500 2024-02-13T03:15:01.378 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.377+0000 7f28777fe700 1 -- 172.21.15.183:0/1566500195 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 1047224232 0 0) 0x7f287c03ecd0 con 0x7f28800fc500 2024-02-13T03:15:01.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.385+0000 7f28777fe700 1 -- 172.21.15.183:0/1566500195 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f287c013ad0 con 0x7f28800fc500 2024-02-13T03:15:01.582 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.581+0000 7f288837d700 1 -- 172.21.15.183:0/1566500195 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi161:[v1:172.21.15.161:6789]=a;smithi161:[v1:172.21.15.161:6790]=c;smithi183:[v1:172.21.15.183:6789]=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f2864000ca0 con 0x7f286c033a40 2024-02-13T03:15:01.599 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.597+0000 7f28777fe700 1 -- 172.21.15.183:0/1566500195 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7f2864000ca0 con 0x7f286c033a40 2024-02-13T03:15:01.599 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mon update... 2024-02-13T03:15:01.602 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.601+0000 7f288837d700 1 -- 172.21.15.183:0/1566500195 >> v1:172.21.15.161:6800/1187535987 conn(0x7f286c033a40 legacy=0x7f286c035f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:01.602 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.601+0000 7f288837d700 1 -- 172.21.15.183:0/1566500195 >> v1:172.21.15.161:6789/0 conn(0x7f28800fc500 legacy=0x7f288010cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:01.602 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.601+0000 7f288837d700 1 -- 172.21.15.183:0/1566500195 shutdown_connections 2024-02-13T03:15:01.603 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.601+0000 7f288837d700 1 -- 172.21.15.183:0/1566500195 >> 172.21.15.183:0/1566500195 conn(0x7f288006c390 msgr2=0x7f2880060da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:15:01.603 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.601+0000 7f288837d700 1 -- 172.21.15.183:0/1566500195 shutdown_connections 2024-02-13T03:15:01.603 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:01.601+0000 7f288837d700 1 -- 172.21.15.183:0/1566500195 wait complete. 2024-02-13T03:15:02.326 DEBUG:teuthology.orchestra.run.smithi161:mon.c> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mon.c.service 2024-02-13T03:15:02.329 DEBUG:teuthology.orchestra.run.smithi183:mon.b> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mon.b.service 2024-02-13T03:15:02.331 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-13T03:15:02.331 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-13T03:15:02.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:02 smithi161 bash[19769]: cluster 2024-02-13T03:15:00.933406+0000 mgr.y (mgr.14146) 32 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:02.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:02 smithi161 bash[19769]: audit 2024-02-13T03:15:01.598187+0000 mon.a (mon.0) 118 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:02.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:02 smithi161 bash[19769]: audit 2024-02-13T03:15:01.629859+0000 mon.a (mon.0) 119 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:02.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:02 smithi161 bash[19769]: audit 2024-02-13T03:15:01.633747+0000 mon.a (mon.0) 120 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:02.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:02 smithi161 bash[19769]: audit 2024-02-13T03:15:01.635568+0000 mon.a (mon.0) 121 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:02.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:02 smithi161 bash[19769]: audit 2024-02-13T03:15:01.642961+0000 mon.a (mon.0) 122 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:02.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:02 smithi161 bash[19769]: audit 2024-02-13T03:15:01.645444+0000 mon.a (mon.0) 123 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T03:15:02.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:02 smithi161 bash[19769]: audit 2024-02-13T03:15:01.646819+0000 mon.a (mon.0) 124 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:02.444 INFO:journalctl@ceph.mon.c.smithi161.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:15:02.604 INFO:journalctl@ceph.mon.b.smithi183.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:15:03.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:03 smithi161 bash[19769]: audit 2024-02-13T03:15:01.584282+0000 mgr.y (mgr.14146) 33 : audit [DBG] from='client.14176 v1:172.21.15.183:0/1566500195' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi161:[v1:172.21.15.161:6789]=a;smithi161:[v1:172.21.15.161:6790]=c;smithi183:[v1:172.21.15.183:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:15:03.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:03 smithi161 bash[19769]: cephadm 2024-02-13T03:15:01.590447+0000 mgr.y (mgr.14146) 34 : cephadm [INF] Saving service mon spec with placement smithi161:[v1:172.21.15.161:6789]=a;smithi161:[v1:172.21.15.161:6790]=c;smithi183:[v1:172.21.15.183:6789]=b;count:3 2024-02-13T03:15:03.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:03 smithi161 bash[19769]: cephadm 2024-02-13T03:15:01.647953+0000 mgr.y (mgr.14146) 35 : cephadm [INF] Deploying daemon mon.b on smithi183 2024-02-13T03:15:04.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:04 smithi161 bash[19769]: cluster 2024-02-13T03:15:02.933907+0000 mgr.y (mgr.14146) 36 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:05.972 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.969+0000 7f82f366e700 1 -- 172.21.15.183:0/825111962 >> v1:172.21.15.161:6789/0 conn(0x7f82ec0fd5c0 legacy=0x7f82ec0fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:05.972 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.969+0000 7f82f366e700 1 -- 172.21.15.183:0/825111962 shutdown_connections 2024-02-13T03:15:05.972 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.969+0000 7f82f366e700 1 -- 172.21.15.183:0/825111962 >> 172.21.15.183:0/825111962 conn(0x7f82ec0f3180 msgr2=0x7f82ec0f55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:15:05.972 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.969+0000 7f82f366e700 1 -- 172.21.15.183:0/825111962 shutdown_connections 2024-02-13T03:15:05.972 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.969+0000 7f82f366e700 1 -- 172.21.15.183:0/825111962 wait complete. 2024-02-13T03:15:05.972 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.969+0000 7f82f366e700 1 Processor -- start 2024-02-13T03:15:05.972 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.969+0000 7f82f366e700 1 -- start start 2024-02-13T03:15:05.972 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.969+0000 7f82f366e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f82ec06cf80 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.973 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.969+0000 7f82f140a700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f82ec0fd5c0 0x7f82ec06b170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.183:59318/0 (socket says 172.21.15.183:59318) 2024-02-13T03:15:05.973 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.969+0000 7f82f140a700 1 -- 172.21.15.183:0/2185510240 learned_addr learned my addr 172.21.15.183:0/2185510240 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:15:05.973 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.969+0000 7f82e27fc700 1 -- 172.21.15.183:0/2185510240 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889458272 0 0) 0x7f82ec06cf80 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.973 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.969+0000 7f82e27fc700 1 -- 172.21.15.183:0/2185510240 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82d8005210 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.973 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.973+0000 7f82e27fc700 1 -- 172.21.15.183:0/2185510240 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3883398585 0 0) 0x7f82d8005210 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.973 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.973+0000 7f82e27fc700 1 -- 172.21.15.183:0/2185510240 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f82ec06cf80 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.973 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.973+0000 7f82e27fc700 1 -- 172.21.15.183:0/2185510240 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f82e8001430 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.974 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.973+0000 7f82e27fc700 1 -- 172.21.15.183:0/2185510240 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2437517887 0 0) 0x7f82ec06cf80 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.974 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.973+0000 7f82e27fc700 1 -- 172.21.15.183:0/2185510240 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec06b880 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.974 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.973+0000 7f82f366e700 1 -- 172.21.15.183:0/2185510240 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ec06bb30 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.974 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.973+0000 7f82f366e700 1 -- 172.21.15.183:0/2185510240 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ec111f40 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.975 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.973+0000 7f82e27fc700 1 -- 172.21.15.183:0/2185510240 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f82e8001100 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.975 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.973+0000 7f82e27fc700 1 -- 172.21.15.183:0/2185510240 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1592025111 0 0) 0x7f82e8003810 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.975 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.973+0000 7f82e27fc700 1 -- 172.21.15.183:0/2185510240 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7f82e8003a10 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.975 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.973+0000 7f82f366e700 1 -- 172.21.15.183:0/2185510240 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d0004f50 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.975 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.973+0000 7f82e27fc700 1 -- 172.21.15.183:0/2185510240 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 1047224232 0 0) 0x7f82e803ef50 con 0x7f82ec0fd5c0 2024-02-13T03:15:05.987 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:05.985+0000 7f82e27fc700 1 -- 172.21.15.183:0/2185510240 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f82e8013f10 con 0x7f82ec0fd5c0 2024-02-13T03:15:06.232 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:06.229+0000 7f82f366e700 1 -- 172.21.15.183:0/2185510240 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f82d0005200 con 0x7f82ec0fd5c0 2024-02-13T03:15:06.233 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:06.233+0000 7f82e27fc700 1 -- 172.21.15.183:0/2185510240 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+675 (unknown 2237029548 0 3258654500) 0x7f82e8013d30 con 0x7f82ec0fd5c0 2024-02-13T03:15:06.234 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-02-13T03:15:06.234 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":1,"fsid":"b9b3ef98-ca1d-11ee-95b9-87774f69a715","modified":"2024-02-13T03:13:01.970936Z","created":"2024-02-13T03:13:01.970936Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-13T03:15:06.237 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:06.233+0000 7f82f366e700 1 -- 172.21.15.183:0/2185510240 >> v1:172.21.15.161:6800/1187535987 conn(0x7f82d8033920 legacy=0x7f82d8035de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:06.237 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:06.233+0000 7f82f366e700 1 -- 172.21.15.183:0/2185510240 >> v1:172.21.15.161:6789/0 conn(0x7f82ec0fd5c0 legacy=0x7f82ec06b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:06.237 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:06.233+0000 7f82f366e700 1 -- 172.21.15.183:0/2185510240 shutdown_connections 2024-02-13T03:15:06.237 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:06.233+0000 7f82f366e700 1 -- 172.21.15.183:0/2185510240 >> 172.21.15.183:0/2185510240 conn(0x7f82ec0f3180 msgr2=0x7f82ec0ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:15:06.237 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:06.237+0000 7f82f366e700 1 -- 172.21.15.183:0/2185510240 shutdown_connections 2024-02-13T03:15:06.237 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:06.237+0000 7f82f366e700 1 -- 172.21.15.183:0/2185510240 wait complete. 2024-02-13T03:15:06.238 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 1 2024-02-13T03:15:06.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:06 smithi161 bash[19769]: cluster 2024-02-13T03:15:04.934394+0000 mgr.y (mgr.14146) 37 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:07.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:07 smithi161 bash[19769]: audit 2024-02-13T03:15:06.233233+0000 mon.a (mon.0) 125 : audit [DBG] from='client.? v1:172.21.15.183:0/2185510240' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T03:15:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:07 smithi183 systemd[1]: Started Ceph mon.b for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:15:07.898 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-13T03:15:07.898 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-13T03:15:08.616 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:08 smithi161 bash[19769]: cluster 2024-02-13T03:15:06.934811+0000 mgr.y (mgr.14146) 38 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:08.616 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:08 smithi161 bash[19769]: audit 2024-02-13T03:15:07.241028+0000 mon.a (mon.0) 126 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:08.616 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:08 smithi161 bash[19769]: audit 2024-02-13T03:15:07.242889+0000 mon.a (mon.0) 127 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T03:15:08.616 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:08 smithi161 bash[19769]: audit 2024-02-13T03:15:07.244556+0000 mon.a (mon.0) 128 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:08.616 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:08 smithi161 bash[19769]: cephadm 2024-02-13T03:15:07.246075+0000 mgr.y (mgr.14146) 39 : cephadm [INF] Deploying daemon mon.c on smithi161 2024-02-13T03:15:08.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.389+0000 7f25dffc6880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-13T03:15:08.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.389+0000 7f25dffc6880 0 ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable), process ceph-mon, pid 7 2024-02-13T03:15:08.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.389+0000 7f25dffc6880 0 pidfile_write: ignore empty --pid-file 2024-02-13T03:15:08.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 0 load: jerasure load: lrc load: isa 2024-02-13T03:15:08.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: RocksDB version: 6.8.1 2024-02-13T03:15:08.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-13T03:15:08.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Compile date Feb 8 2024 2024-02-13T03:15:08.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: DB SUMMARY 2024-02-13T03:15:08.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: CURRENT file: CURRENT 2024-02-13T03:15:08.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-13T03:15:08.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2024-02-13T03:15:08.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-13T03:15:08.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000003.log size: 476 ; 2024-02-13T03:15:08.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.error_if_exists: 0 2024-02-13T03:15:08.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.create_if_missing: 0 2024-02-13T03:15:08.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.paranoid_checks: 1 2024-02-13T03:15:08.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.env: 0x55f83ff33080 2024-02-13T03:15:08.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.fs: Posix File System 2024-02-13T03:15:08.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.info_log: 0x55f84149d980 2024-02-13T03:15:08.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-13T03:15:08.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.statistics: (nil) 2024-02-13T03:15:08.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.use_fsync: 0 2024-02-13T03:15:08.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.max_log_file_size: 0 2024-02-13T03:15:08.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-13T03:15:08.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-13T03:15:08.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-13T03:15:08.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-13T03:15:08.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.allow_fallocate: 1 2024-02-13T03:15:08.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-13T03:15:08.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-13T03:15:08.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.use_direct_reads: 0 2024-02-13T03:15:08.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-13T03:15:08.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-13T03:15:08.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.db_log_dir: 2024-02-13T03:15:08.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-02-13T03:15:08.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-13T03:15:08.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.max_subcompactions: 1 2024-02-13T03:15:08.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.max_background_flushes: -1 2024-02-13T03:15:08.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-13T03:15:08.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-13T03:15:08.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-13T03:15:08.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-13T03:15:08.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-13T03:15:08.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-13T03:15:08.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-13T03:15:08.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.write_buffer_manager: 0x55f84219c3f0 2024-02-13T03:15:08.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-13T03:15:08.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-13T03:15:08.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-13T03:15:08.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-13T03:15:08.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-13T03:15:08.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-13T03:15:08.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-13T03:15:08.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-13T03:15:08.744 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-13T03:15:08.744 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.unordered_write: 0 2024-02-13T03:15:08.744 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-13T03:15:08.744 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-13T03:15:08.744 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-13T03:15:08.744 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-13T03:15:08.745 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.row_cache: None 2024-02-13T03:15:08.745 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.wal_filter: None 2024-02-13T03:15:08.745 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-13T03:15:08.745 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-13T03:15:08.745 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.preserve_deletes: 0 2024-02-13T03:15:08.745 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.two_write_queues: 0 2024-02-13T03:15:08.745 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-13T03:15:08.745 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.atomic_flush: 0 2024-02-13T03:15:08.745 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-13T03:15:08.745 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-13T03:15:08.745 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-13T03:15:08.745 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.log_readahead_size: 0 2024-02-13T03:15:08.745 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-13T03:15:08.746 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.max_background_jobs: 2 2024-02-13T03:15:08.746 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.max_background_compactions: -1 2024-02-13T03:15:08.746 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-13T03:15:08.746 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-13T03:15:08.746 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-13T03:15:08.746 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-13T03:15:08.746 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-13T03:15:08.746 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-13T03:15:08.746 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-13T03:15:08.746 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.max_open_files: -1 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Compression algorithms supported: 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: kZSTD supported: 0 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: kXpressCompression supported: 0 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: kLZ4Compression supported: 1 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: kBZip2Compression supported: 0 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: kZlibCompression supported: 1 2024-02-13T03:15:08.747 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: kSnappyCompression supported: 1 2024-02-13T03:15:08.748 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.393+0000 7f25dffc6880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-13T03:15:08.748 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 2024-02-13T03:15:08.748 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-13T03:15:08.748 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-13T03:15:08.748 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.merge_operator: 2024-02-13T03:15:08.748 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compaction_filter: None 2024-02-13T03:15:08.748 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-13T03:15:08.748 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-13T03:15:08.748 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-13T03:15:08.748 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f84144bd28) 2024-02-13T03:15:08.748 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: cache_index_and_filter_blocks: 1 2024-02-13T03:15:08.748 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-13T03:15:08.748 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-13T03:15:08.749 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: pin_top_level_index_and_filter: 1 2024-02-13T03:15:08.749 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: index_type: 0 2024-02-13T03:15:08.749 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: data_block_index_type: 0 2024-02-13T03:15:08.749 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: index_shortening: 1 2024-02-13T03:15:08.749 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: data_block_hash_table_util_ratio: 0.750000 2024-02-13T03:15:08.749 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: hash_index_allow_collision: 1 2024-02-13T03:15:08.749 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: checksum: 1 2024-02-13T03:15:08.749 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: no_block_cache: 0 2024-02-13T03:15:08.749 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: block_cache: 0x55f841482f10 2024-02-13T03:15:08.749 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: block_cache_name: BinnedLRUCache 2024-02-13T03:15:08.749 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: block_cache_options: 2024-02-13T03:15:08.749 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: capacity : 536870912 2024-02-13T03:15:08.749 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: num_shard_bits : 4 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: strict_capacity_limit : 0 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: high_pri_pool_ratio: 0.000 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: block_cache_compressed: (nil) 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: persistent_cache: (nil) 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: block_size: 4096 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: block_size_deviation: 10 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: block_restart_interval: 16 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: index_block_restart_interval: 1 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: metadata_block_size: 4096 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: partition_filters: 0 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: use_delta_encoding: 1 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: whole_key_filtering: 1 2024-02-13T03:15:08.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: verify_compression: 0 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: read_amp_bytes_per_bit: 0 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: format_version: 2 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: enable_index_compression: 1 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: block_align: 0 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compression: NoCompression 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.num_levels: 7 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-13T03:15:08.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-13T03:15:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-13T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-13T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-13T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-13T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-13T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-13T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-13T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-13T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-13T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-13T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-13T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-13T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-13T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-13T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.table_properties_collectors: 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.inplace_update_support: 0 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.bloom_locality: 0 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.max_successive_merges: 0 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-13T03:15:08.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-13T03:15:08.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-13T03:15:08.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.ttl: 2592000 2024-02-13T03:15:08.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-13T03:15:08.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-13T03:15:08.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-13T03:15:08.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707794108400362, "job": 1, "event": "recovery_started", "log_files": [3]} 2024-02-13T03:15:08.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2024-02-13T03:15:08.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-13T03:15:08.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707794108401627, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1402, "table_properties": {"data_size": 488, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 366, "raw_average_value_size": 73, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1707794108, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-13T03:15:08.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.397+0000 7f25dffc6880 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2024-02-13T03:15:08.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.401+0000 7f25dffc6880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707794108402432, "job": 1, "event": "recovery_finished"} 2024-02-13T03:15:08.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.401+0000 7f25dffc6880 4 rocksdb: DB pointer 0x55f84154b800 2024-02-13T03:15:08.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.401+0000 7f25dffc6880 0 mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.401+0000 7f25dffc6880 0 using public_addrv v1:172.21.15.183:6789/0 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.401+0000 7f25dffc6880 0 starting mon.b rank -1 at public addrs v1:172.21.15.183:6789/0 at bind addrs v1:172.21.15.183:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.401+0000 7f25c6db7700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.401+0000 7f25c6db7700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: ** DB Stats ** 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: ** Compaction Stats [default] ** 2024-02-13T03:15:08.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: L0 1/0 1.37 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Sum 1/0 1.37 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: ** Compaction Stats [default] ** 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.1 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: AddFile(Total Files): cumulative 0, interval 0 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-13T03:15:08.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: AddFile(Keys): cumulative 0, interval 0 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Cumulative compaction: 0.00 GB write, 0.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Interval compaction: 0.00 GB write, 0.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: ** File Read Latency Histogram By Level [default] ** 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: ** Compaction Stats [default] ** 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: L0 1/0 1.37 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Sum 1/0 1.37 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: ** Compaction Stats [default] ** 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.1 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:08.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T03:15:08.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-13T03:15:08.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-13T03:15:08.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: AddFile(Total Files): cumulative 0, interval 0 2024-02-13T03:15:08.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-13T03:15:08.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: AddFile(Keys): cumulative 0, interval 0 2024-02-13T03:15:08.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Cumulative compaction: 0.00 GB write, 0.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T03:15:08.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T03:15:08.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-13T03:15:08.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: ** File Read Latency Histogram By Level [default] ** 2024-02-13T03:15:08.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.401+0000 7f25dffc6880 1 mon.b@-1(???) e0 preinit fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:15:08.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.429+0000 7f25c95bc700 0 mon.b@-1(synchronizing).mds e1 new map 2024-02-13T03:15:08.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.429+0000 7f25c95bc700 0 mon.b@-1(synchronizing).mds e1 print_map 2024-02-13T03:15:08.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: e1 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: legacy client fscid: -1 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: No filesystems configured 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.429+0000 7f25c95bc700 1 mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.433+0000 7f25c95bc700 1 mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.433+0000 7f25c95bc700 1 mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.433+0000 7f25c95bc700 1 mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.433+0000 7f25c95bc700 1 mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.433+0000 7f25c95bc700 1 mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.433+0000 7f25c95bc700 0 mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.433+0000 7f25c95bc700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T03:15:08.761 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: cluster 2024-02-13T03:15:06.934811+0000 mgr.y (mgr.14146) 38 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:08.762 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: audit 2024-02-13T03:15:07.241028+0000 mon.a (mon.0) 126 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:08.762 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: audit 2024-02-13T03:15:07.242889+0000 mon.a (mon.0) 127 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T03:15:08.762 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: audit 2024-02-13T03:15:07.244556+0000 mon.a (mon.0) 128 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:08.762 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: cephadm 2024-02-13T03:15:07.246075+0000 mgr.y (mgr.14146) 39 : cephadm [INF] Deploying daemon mon.c on smithi161 2024-02-13T03:15:08.762 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:08 smithi183 bash[20824]: debug 2024-02-13T03:15:08.433+0000 7f25c95bc700 1 mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-13T03:15:10.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:10 smithi161 systemd[1]: Started Ceph mon.c for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:15:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.947+0000 7fdfa8810880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-13T03:15:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.947+0000 7fdfa8810880 0 ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable), process ceph-mon, pid 7 2024-02-13T03:15:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.947+0000 7fdfa8810880 0 pidfile_write: ignore empty --pid-file 2024-02-13T03:15:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 0 load: jerasure load: lrc load: isa 2024-02-13T03:15:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: RocksDB version: 6.8.1 2024-02-13T03:15:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-13T03:15:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Compile date Feb 8 2024 2024-02-13T03:15:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: DB SUMMARY 2024-02-13T03:15:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: CURRENT file: CURRENT 2024-02-13T03:15:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-13T03:15:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2024-02-13T03:15:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-13T03:15:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000003.log size: 476 ; 2024-02-13T03:15:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.error_if_exists: 0 2024-02-13T03:15:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.create_if_missing: 0 2024-02-13T03:15:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.paranoid_checks: 1 2024-02-13T03:15:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.env: 0x55d9780af080 2024-02-13T03:15:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.fs: Posix File System 2024-02-13T03:15:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.info_log: 0x55d979ad3980 2024-02-13T03:15:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-13T03:15:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.statistics: (nil) 2024-02-13T03:15:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.use_fsync: 0 2024-02-13T03:15:12.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_log_file_size: 0 2024-02-13T03:15:12.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-13T03:15:12.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-13T03:15:12.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-13T03:15:12.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-13T03:15:12.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.allow_fallocate: 1 2024-02-13T03:15:12.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-13T03:15:12.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-13T03:15:12.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.use_direct_reads: 0 2024-02-13T03:15:12.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-13T03:15:12.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-13T03:15:12.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.db_log_dir: 2024-02-13T03:15:12.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-02-13T03:15:12.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-13T03:15:12.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_subcompactions: 1 2024-02-13T03:15:12.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_background_flushes: -1 2024-02-13T03:15:12.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-13T03:15:12.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-13T03:15:12.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-13T03:15:12.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-13T03:15:12.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-13T03:15:12.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-13T03:15:12.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-13T03:15:12.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.write_buffer_manager: 0x55d97a7d23f0 2024-02-13T03:15:12.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.unordered_write: 0 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.row_cache: None 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.wal_filter: None 2024-02-13T03:15:12.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.preserve_deletes: 0 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.two_write_queues: 0 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.atomic_flush: 0 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.log_readahead_size: 0 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_background_jobs: 2 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_background_compactions: -1 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-13T03:15:12.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_open_files: -1 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Compression algorithms supported: 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: kZSTD supported: 0 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: kXpressCompression supported: 0 2024-02-13T03:15:12.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-13T03:15:12.184 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: kLZ4Compression supported: 1 2024-02-13T03:15:12.184 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: kBZip2Compression supported: 0 2024-02-13T03:15:12.184 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: kZlibCompression supported: 1 2024-02-13T03:15:12.184 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: kSnappyCompression supported: 1 2024-02-13T03:15:12.184 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-13T03:15:12.184 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 2024-02-13T03:15:12.184 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-13T03:15:12.184 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-13T03:15:12.184 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.merge_operator: 2024-02-13T03:15:12.184 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compaction_filter: None 2024-02-13T03:15:12.184 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-13T03:15:12.184 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-13T03:15:12.184 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d979a81d28) 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: cache_index_and_filter_blocks: 1 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: pin_top_level_index_and_filter: 1 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: index_type: 0 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: data_block_index_type: 0 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: index_shortening: 1 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: data_block_hash_table_util_ratio: 0.750000 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: hash_index_allow_collision: 1 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: checksum: 1 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: no_block_cache: 0 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: block_cache: 0x55d979ab8f10 2024-02-13T03:15:12.185 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: block_cache_name: BinnedLRUCache 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: block_cache_options: 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: capacity : 536870912 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: num_shard_bits : 4 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: strict_capacity_limit : 0 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: high_pri_pool_ratio: 0.000 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: block_cache_compressed: (nil) 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: persistent_cache: (nil) 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: block_size: 4096 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: block_size_deviation: 10 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: block_restart_interval: 16 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: index_block_restart_interval: 1 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: metadata_block_size: 4096 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: partition_filters: 0 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: use_delta_encoding: 1 2024-02-13T03:15:12.186 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: whole_key_filtering: 1 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: verify_compression: 0 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: read_amp_bytes_per_bit: 0 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: format_version: 2 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: enable_index_compression: 1 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: block_align: 0 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compression: NoCompression 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.num_levels: 7 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-13T03:15:12.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-13T03:15:12.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-13T03:15:12.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-13T03:15:12.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-13T03:15:12.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-13T03:15:12.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-13T03:15:12.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-13T03:15:12.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-13T03:15:12.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-13T03:15:12.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-13T03:15:12.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-13T03:15:12.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-13T03:15:12.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-13T03:15:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-13T03:15:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-13T03:15:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-13T03:15:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-13T03:15:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-13T03:15:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-13T03:15:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-13T03:15:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-13T03:15:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-13T03:15:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-13T03:15:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-13T03:15:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-13T03:15:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-13T03:15:12.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-13T03:15:12.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-13T03:15:12.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-13T03:15:12.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-13T03:15:12.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-13T03:15:12.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-13T03:15:12.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-13T03:15:12.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-13T03:15:12.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-13T03:15:12.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-13T03:15:12.191 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-13T03:15:12.191 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-13T03:15:12.191 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-13T03:15:12.191 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-13T03:15:12.191 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-13T03:15:12.191 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-13T03:15:12.191 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-13T03:15:12.191 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-13T03:15:12.191 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-13T03:15:12.191 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-13T03:15:12.191 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.table_properties_collectors: 2024-02-13T03:15:12.191 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.inplace_update_support: 0 2024-02-13T03:15:12.192 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-13T03:15:12.192 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-13T03:15:12.192 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-13T03:15:12.192 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-13T03:15:12.192 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.bloom_locality: 0 2024-02-13T03:15:12.192 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.max_successive_merges: 0 2024-02-13T03:15:12.192 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-13T03:15:12.192 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-13T03:15:12.192 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-13T03:15:12.192 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-13T03:15:12.192 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.ttl: 2592000 2024-02-13T03:15:12.193 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.951+0000 7fdfa8810880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-13T03:15:12.193 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.955+0000 7fdfa8810880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-13T03:15:12.193 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.955+0000 7fdfa8810880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-13T03:15:12.193 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.955+0000 7fdfa8810880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707794111959068, "job": 1, "event": "recovery_started", "log_files": [3]} 2024-02-13T03:15:12.193 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.955+0000 7fdfa8810880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2024-02-13T03:15:12.194 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.955+0000 7fdfa8810880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-13T03:15:12.194 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.955+0000 7fdfa8810880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707794111960219, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1402, "table_properties": {"data_size": 488, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 366, "raw_average_value_size": 73, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1707794111, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-13T03:15:12.194 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.955+0000 7fdfa8810880 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2024-02-13T03:15:12.194 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.955+0000 7fdfa8810880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707794111961322, "job": 1, "event": "recovery_finished"} 2024-02-13T03:15:12.194 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.959+0000 7fdfa8810880 4 rocksdb: DB pointer 0x55d979b81800 2024-02-13T03:15:12.194 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.959+0000 7fdf8f601700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-13T03:15:12.194 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.959+0000 7fdf8f601700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-13T03:15:12.194 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: ** DB Stats ** 2024-02-13T03:15:12.194 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T03:15:12.194 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-13T03:15:12.194 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T03:15:12.194 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T03:15:12.194 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: ** Compaction Stats [default] ** 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: L0 1/0 1.37 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Sum 1/0 1.37 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: ** Compaction Stats [default] ** 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T03:15:12.195 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-13T03:15:12.196 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-13T03:15:12.196 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: AddFile(Total Files): cumulative 0, interval 0 2024-02-13T03:15:12.196 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-13T03:15:12.196 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: AddFile(Keys): cumulative 0, interval 0 2024-02-13T03:15:12.196 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Cumulative compaction: 0.00 GB write, 0.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T03:15:12.196 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Interval compaction: 0.00 GB write, 0.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T03:15:12.196 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-13T03:15:12.196 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: ** File Read Latency Histogram By Level [default] ** 2024-02-13T03:15:12.196 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: ** Compaction Stats [default] ** 2024-02-13T03:15:12.196 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-13T03:15:12.196 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T03:15:12.196 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: L0 1/0 1.37 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:12.196 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Sum 1/0 1.37 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: ** Compaction Stats [default] ** 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.00 0.00 1 0.001 0 0 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: AddFile(Total Files): cumulative 0, interval 0 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: AddFile(Keys): cumulative 0, interval 0 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Cumulative compaction: 0.00 GB write, 0.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T03:15:12.197 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: ** File Read Latency Histogram By Level [default] ** 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.959+0000 7fdfa8810880 0 mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.959+0000 7fdfa8810880 0 using public_addrv v1:172.21.15.161:6790/0 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.959+0000 7fdfa8810880 0 starting mon.c rank -1 at public addrs v1:172.21.15.161:6790/0 at bind addrs v1:172.21.15.161:6790/0 mon_data /var/lib/ceph/mon/ceph-c fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.963+0000 7fdfa8810880 1 mon.c@-1(???) e0 preinit fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.983+0000 7fdf91e06700 0 mon.c@-1(synchronizing).mds e1 new map 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.983+0000 7fdf91e06700 0 mon.c@-1(synchronizing).mds e1 print_map 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: e1 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: legacy client fscid: -1 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: No filesystems configured 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.983+0000 7fdf91e06700 1 mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-13T03:15:12.198 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.987+0000 7fdf91e06700 1 mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-13T03:15:12.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.987+0000 7fdf91e06700 1 mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-13T03:15:12.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.987+0000 7fdf91e06700 1 mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-13T03:15:12.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.987+0000 7fdf91e06700 1 mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-13T03:15:12.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.987+0000 7fdf91e06700 1 mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-13T03:15:12.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.987+0000 7fdf91e06700 0 mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-13T03:15:12.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.987+0000 7fdf91e06700 0 mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T03:15:12.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: cluster 2024-02-13T03:15:06.934811+0000 mgr.y (mgr.14146) 38 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:12.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: audit 2024-02-13T03:15:07.241028+0000 mon.a (mon.0) 126 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:12.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: audit 2024-02-13T03:15:07.242889+0000 mon.a (mon.0) 127 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T03:15:12.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: audit 2024-02-13T03:15:07.244556+0000 mon.a (mon.0) 128 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:12.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: cephadm 2024-02-13T03:15:07.246075+0000 mgr.y (mgr.14146) 39 : cephadm [INF] Deploying daemon mon.c on smithi161 2024-02-13T03:15:12.199 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:11 smithi161 bash[27879]: debug 2024-02-13T03:15:11.987+0000 7fdf91e06700 1 mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-13T03:15:13.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.481+0000 7f9f01392700 1 -- 172.21.15.183:0/3535752015 >> v1:172.21.15.161:6789/0 conn(0x7f9efc0fd5f0 legacy=0x7f9efc0fd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:13.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.481+0000 7f9f01392700 1 -- 172.21.15.183:0/3535752015 shutdown_connections 2024-02-13T03:15:13.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.481+0000 7f9f01392700 1 -- 172.21.15.183:0/3535752015 >> 172.21.15.183:0/3535752015 conn(0x7f9efc0f3190 msgr2=0x7f9efc0f55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:15:13.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.481+0000 7f9f01392700 1 -- 172.21.15.183:0/3535752015 shutdown_connections 2024-02-13T03:15:13.485 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.481+0000 7f9f01392700 1 -- 172.21.15.183:0/3535752015 wait complete. 2024-02-13T03:15:13.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.485+0000 7f9f01392700 1 Processor -- start 2024-02-13T03:15:13.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.485+0000 7f9f01392700 1 -- start start 2024-02-13T03:15:13.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.485+0000 7f9f01392700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9efc1130e0 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.485+0000 7f9f01392700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9efc114280 con 0x7f9efc10d270 2024-02-13T03:15:13.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.485+0000 7f9efaffd700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f9efc0fd5f0 0x7f9efc10cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.183:59422/0 (socket says 172.21.15.183:59422) 2024-02-13T03:15:13.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.485+0000 7f9efaffd700 1 -- 172.21.15.183:0/4261901021 learned_addr learned my addr 172.21.15.183:0/4261901021 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:15:13.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.485+0000 7f9efa7fc700 1 --1- 172.21.15.183:0/4261901021 >> v1:172.21.15.183:6789/0 conn(0x7f9efc10d270 0x7f9efc1119c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:52714/0 (socket says 172.21.15.183:52714) 2024-02-13T03:15:13.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.485+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561841749 0 0) 0x7f9efc1130e0 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.485+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ee00053b0 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.489 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.485+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3455783392 0 0) 0x7f9ee00053b0 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.489 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.485+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9efc1130e0 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.489 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.485+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 595990527 0 0) 0x7f9eec001ff0 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.489+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 679587981 0 0) 0x7f9efc1130e0 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.489+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 >> v1:172.21.15.183:6789/0 conn(0x7f9efc10d270 legacy=0x7f9efc1119c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:13.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.489+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc115420 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.489+0000 7f9f01392700 1 -- 172.21.15.183:0/4261901021 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9efc1132d0 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.489+0000 7f9f01392700 1 -- 172.21.15.183:0/4261901021 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9efc1137c0 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.489+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f9eec002cd0 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.489+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 595990527 0 0) 0x7f9eec0038c0 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.489+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7f9eec003b00 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.489+0000 7f9f01392700 1 -- 172.21.15.183:0/4261901021 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee4004f20 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.489+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 1047224232 0 0) 0x7f9eec03fc70 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.505+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f9eec0110e0 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.763 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.761+0000 7f9f01392700 1 -- 172.21.15.183:0/4261901021 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9ee40051d0 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.764 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.761+0000 7f9ef3fff700 1 -- 172.21.15.183:0/4261901021 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+893 (unknown 3557084514 0 1153993403) 0x7f9eec013180 con 0x7f9efc0fd5f0 2024-02-13T03:15:13.764 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-02-13T03:15:13.765 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"b9b3ef98-ca1d-11ee-95b9-87774f69a715","modified":"2024-02-13T03:15:08.441895Z","created":"2024-02-13T03:13:01.970936Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-13T03:15:13.766 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.765+0000 7f9f01392700 1 -- 172.21.15.183:0/4261901021 >> v1:172.21.15.161:6800/1187535987 conn(0x7f9ee00339d0 legacy=0x7f9ee0035e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:13.766 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.765+0000 7f9f01392700 1 -- 172.21.15.183:0/4261901021 >> v1:172.21.15.161:6789/0 conn(0x7f9efc0fd5f0 legacy=0x7f9efc10cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:13.767 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.765+0000 7f9f01392700 1 -- 172.21.15.183:0/4261901021 shutdown_connections 2024-02-13T03:15:13.767 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.765+0000 7f9f01392700 1 -- 172.21.15.183:0/4261901021 >> 172.21.15.183:0/4261901021 conn(0x7f9efc0f3190 msgr2=0x7f9efc0ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:15:13.767 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.765+0000 7f9f01392700 1 -- 172.21.15.183:0/4261901021 shutdown_connections 2024-02-13T03:15:13.767 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:13.765+0000 7f9f01392700 1 -- 172.21.15.183:0/4261901021 wait complete. 2024-02-13T03:15:13.767 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2024-02-13T03:15:13.819 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: audit 2024-02-13T03:15:08.445275+0000 mon.a (mon.0) 130 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T03:15:13.819 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: cluster 2024-02-13T03:15:08.445514+0000 mon.a (mon.0) 131 : cluster [INF] mon.a calling monitor election 2024-02-13T03:15:13.819 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: audit 2024-02-13T03:15:08.446901+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:13.819 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: cluster 2024-02-13T03:15:08.935218+0000 mgr.y (mgr.14146) 40 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:13.820 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: audit 2024-02-13T03:15:09.440889+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:13.820 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: audit 2024-02-13T03:15:10.440980+0000 mon.a (mon.0) 134 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:13.820 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: cluster 2024-02-13T03:15:10.444867+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-13T03:15:13.820 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: cluster 2024-02-13T03:15:10.935585+0000 mgr.y (mgr.14146) 41 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:13.820 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: audit 2024-02-13T03:15:11.441150+0000 mon.a (mon.0) 135 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:13.820 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: audit 2024-02-13T03:15:11.998147+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:13.820 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: audit 2024-02-13T03:15:12.441312+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:13.820 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: audit 2024-02-13T03:15:12.997667+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:13.820 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: audit 2024-02-13T03:15:13.441320+0000 mon.a (mon.0) 139 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:13.821 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: cluster 2024-02-13T03:15:13.467537+0000 mon.a (mon.0) 140 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-13T03:15:13.821 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: cluster 2024-02-13T03:15:13.473526+0000 mon.a (mon.0) 141 : cluster [DBG] monmap e2: 2 mons at {a=v1:172.21.15.161:6789/0,b=v1:172.21.15.183:6789/0} removed_ranks: {} 2024-02-13T03:15:13.821 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: cluster 2024-02-13T03:15:13.474754+0000 mon.a (mon.0) 142 : cluster [DBG] fsmap 2024-02-13T03:15:13.821 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: cluster 2024-02-13T03:15:13.474841+0000 mon.a (mon.0) 143 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-13T03:15:13.821 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: cluster 2024-02-13T03:15:13.475188+0000 mon.a (mon.0) 144 : cluster [DBG] mgrmap e13: y(active, since 70s) 2024-02-13T03:15:13.821 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: cluster 2024-02-13T03:15:13.482416+0000 mon.a (mon.0) 145 : cluster [INF] overall HEALTH_OK 2024-02-13T03:15:13.821 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: audit 2024-02-13T03:15:13.485752+0000 mon.a (mon.0) 146 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:13.821 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:13 smithi183 bash[20824]: audit 2024-02-13T03:15:13.487560+0000 mon.a (mon.0) 147 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:13.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: audit 2024-02-13T03:15:08.445275+0000 mon.a (mon.0) 130 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T03:15:13.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: cluster 2024-02-13T03:15:08.445514+0000 mon.a (mon.0) 131 : cluster [INF] mon.a calling monitor election 2024-02-13T03:15:13.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: audit 2024-02-13T03:15:08.446901+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:13.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: cluster 2024-02-13T03:15:08.935218+0000 mgr.y (mgr.14146) 40 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:13.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: audit 2024-02-13T03:15:09.440889+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: audit 2024-02-13T03:15:10.440980+0000 mon.a (mon.0) 134 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: cluster 2024-02-13T03:15:10.444867+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-13T03:15:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: cluster 2024-02-13T03:15:10.935585+0000 mgr.y (mgr.14146) 41 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: audit 2024-02-13T03:15:11.441150+0000 mon.a (mon.0) 135 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: audit 2024-02-13T03:15:11.998147+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: audit 2024-02-13T03:15:12.441312+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: audit 2024-02-13T03:15:12.997667+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: audit 2024-02-13T03:15:13.441320+0000 mon.a (mon.0) 139 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: cluster 2024-02-13T03:15:13.467537+0000 mon.a (mon.0) 140 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-13T03:15:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: cluster 2024-02-13T03:15:13.473526+0000 mon.a (mon.0) 141 : cluster [DBG] monmap e2: 2 mons at {a=v1:172.21.15.161:6789/0,b=v1:172.21.15.183:6789/0} removed_ranks: {} 2024-02-13T03:15:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: cluster 2024-02-13T03:15:13.474754+0000 mon.a (mon.0) 142 : cluster [DBG] fsmap 2024-02-13T03:15:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: cluster 2024-02-13T03:15:13.474841+0000 mon.a (mon.0) 143 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-13T03:15:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: cluster 2024-02-13T03:15:13.475188+0000 mon.a (mon.0) 144 : cluster [DBG] mgrmap e13: y(active, since 70s) 2024-02-13T03:15:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: cluster 2024-02-13T03:15:13.482416+0000 mon.a (mon.0) 145 : cluster [INF] overall HEALTH_OK 2024-02-13T03:15:13.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: audit 2024-02-13T03:15:13.485752+0000 mon.a (mon.0) 146 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:13.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:13 smithi161 bash[19769]: audit 2024-02-13T03:15:13.487560+0000 mon.a (mon.0) 147 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:14.925 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:15:14 smithi161 bash[20075]: debug 2024-02-13T03:15:14.439+0000 7f2e2b5fa700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-02-13T03:15:15.340 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-13T03:15:15.340 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-13T03:15:19.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.025+0000 7f6279c74700 1 -- 172.21.15.183:0/1907877635 >> v1:172.21.15.161:6789/0 conn(0x7f62740ff7e0 legacy=0x7f62740ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:19.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.025+0000 7f6279c74700 1 -- 172.21.15.183:0/1907877635 shutdown_connections 2024-02-13T03:15:19.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.025+0000 7f6279c74700 1 -- 172.21.15.183:0/1907877635 >> 172.21.15.183:0/1907877635 conn(0x7f62740f5710 msgr2=0x7f62740f7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:15:19.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.029+0000 7f6279c74700 1 -- 172.21.15.183:0/1907877635 shutdown_connections 2024-02-13T03:15:19.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.029+0000 7f6279c74700 1 -- 172.21.15.183:0/1907877635 wait complete. 2024-02-13T03:15:19.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.029+0000 7f6279c74700 1 Processor -- start 2024-02-13T03:15:19.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.029+0000 7f6279c74700 1 -- start start 2024-02-13T03:15:19.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.029+0000 7f6279c74700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f62741191d0 con 0x7f62740ff7e0 2024-02-13T03:15:19.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.029+0000 7f6279c74700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f627411a3d0 con 0x7f627410f630 2024-02-13T03:15:19.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.029+0000 7f6279c74700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f627411b5d0 con 0x7f6274115460 2024-02-13T03:15:19.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.029+0000 7f6272ffd700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f627410f630 0x7f6274113d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:52730/0 (socket says 172.21.15.183:52730) 2024-02-13T03:15:19.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.029+0000 7f6272ffd700 1 -- 172.21.15.183:0/1270585249 learned_addr learned my addr 172.21.15.183:0/1270585249 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:15:19.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.029+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3872132900 0 0) 0x7f627411b5d0 con 0x7f6274115460 2024-02-13T03:15:19.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.029+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6260003ae0 con 0x7f6274115460 2024-02-13T03:15:19.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3052016993 0 0) 0x7f6260003ae0 con 0x7f6274115460 2024-02-13T03:15:19.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f627411b5d0 con 0x7f6274115460 2024-02-13T03:15:19.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f626c001b40 con 0x7f6274115460 2024-02-13T03:15:19.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429176784 0 0) 0x7f62741191d0 con 0x7f62740ff7e0 2024-02-13T03:15:19.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6260003ae0 con 0x7f62740ff7e0 2024-02-13T03:15:19.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1284566809 0 0) 0x7f6260003ae0 con 0x7f62740ff7e0 2024-02-13T03:15:19.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62741191d0 con 0x7f62740ff7e0 2024-02-13T03:15:19.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f62640021e0 con 0x7f62740ff7e0 2024-02-13T03:15:19.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3896711866 0 0) 0x7f62741191d0 con 0x7f62740ff7e0 2024-02-13T03:15:19.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 >> v1:172.21.15.161:6790/0 conn(0x7f6274115460 legacy=0x7f62741178d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:19.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 >> v1:172.21.15.183:6789/0 conn(0x7f627410f630 legacy=0x7f6274113d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:19.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627411c7d0 con 0x7f62740ff7e0 2024-02-13T03:15:19.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6279c74700 1 -- 172.21.15.183:0/1270585249 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627411a5c0 con 0x7f62740ff7e0 2024-02-13T03:15:19.036 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6279c74700 1 -- 172.21.15.183:0/1270585249 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f627411aab0 con 0x7f62740ff7e0 2024-02-13T03:15:19.036 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f62640026f0 con 0x7f62740ff7e0 2024-02-13T03:15:19.036 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f62640037b0 con 0x7f62740ff7e0 2024-02-13T03:15:19.037 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.033+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7f6264003930 con 0x7f62740ff7e0 2024-02-13T03:15:19.037 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.037+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 1047224232 0 0) 0x7f6264002860 con 0x7f62740ff7e0 2024-02-13T03:15:19.038 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.037+0000 7f6279c74700 1 -- 172.21.15.183:0/1270585249 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6254004f20 con 0x7f62740ff7e0 2024-02-13T03:15:19.048 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.045+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f6264004290 con 0x7f62740ff7e0 2024-02-13T03:15:19.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.301+0000 7f6279c74700 1 -- 172.21.15.183:0/1270585249 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f62540051d0 con 0x7f62740ff7e0 2024-02-13T03:15:19.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.305+0000 7f6270ff9700 1 -- 172.21.15.183:0/1270585249 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1111 (unknown 409899127 0 840546026) 0x7f6264012090 con 0x7f62740ff7e0 2024-02-13T03:15:19.306 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-02-13T03:15:19.306 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":3,"fsid":"b9b3ef98-ca1d-11ee-95b9-87774f69a715","modified":"2024-02-13T03:15:13.999632Z","created":"2024-02-13T03:13:01.970936Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6790","nonce":0}]},"addr":"172.21.15.161:6790/0","public_addr":"172.21.15.161:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-13T03:15:19.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.305+0000 7f6279c74700 1 -- 172.21.15.183:0/1270585249 >> v1:172.21.15.161:6800/1187535987 conn(0x7f62600339d0 legacy=0x7f6260035e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:19.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.309+0000 7f6279c74700 1 -- 172.21.15.183:0/1270585249 >> v1:172.21.15.161:6789/0 conn(0x7f62740ff7e0 legacy=0x7f627410ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:19.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.309+0000 7f6279c74700 1 -- 172.21.15.183:0/1270585249 shutdown_connections 2024-02-13T03:15:19.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.309+0000 7f6279c74700 1 -- 172.21.15.183:0/1270585249 >> 172.21.15.183:0/1270585249 conn(0x7f62740f5710 msgr2=0x7f62740f6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:15:19.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.309+0000 7f6279c74700 1 -- 172.21.15.183:0/1270585249 shutdown_connections 2024-02-13T03:15:19.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:19.309+0000 7f6279c74700 1 -- 172.21.15.183:0/1270585249 wait complete. 2024-02-13T03:15:19.310 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 3 2024-02-13T03:15:19.363 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: audit 2024-02-13T03:15:14.005789+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T03:15:19.363 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: cluster 2024-02-13T03:15:14.006250+0000 mon.a (mon.0) 151 : cluster [INF] mon.a calling monitor election 2024-02-13T03:15:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: audit 2024-02-13T03:15:14.007678+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: audit 2024-02-13T03:15:14.007885+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: cluster 2024-02-13T03:15:14.011252+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-13T03:15:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: cluster 2024-02-13T03:15:14.936399+0000 mgr.y (mgr.14146) 43 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: audit 2024-02-13T03:15:14.998056+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: audit 2024-02-13T03:15:15.998159+0000 mon.a (mon.0) 155 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: cluster 2024-02-13T03:15:16.001169+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-13T03:15:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: cluster 2024-02-13T03:15:16.936861+0000 mgr.y (mgr.14146) 44 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: audit 2024-02-13T03:15:16.998267+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: audit 2024-02-13T03:15:17.998483+0000 mon.a (mon.0) 157 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: audit 2024-02-13T03:15:18.998594+0000 mon.a (mon.0) 158 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: cluster 2024-02-13T03:15:19.011489+0000 mon.a (mon.0) 159 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-13T03:15:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: cluster 2024-02-13T03:15:19.018013+0000 mon.a (mon.0) 160 : cluster [DBG] monmap e3: 3 mons at {a=v1:172.21.15.161:6789/0,b=v1:172.21.15.183:6789/0,c=v1:172.21.15.161:6790/0} removed_ranks: {} 2024-02-13T03:15:19.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: cluster 2024-02-13T03:15:19.018840+0000 mon.a (mon.0) 161 : cluster [DBG] fsmap 2024-02-13T03:15:19.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: cluster 2024-02-13T03:15:19.018885+0000 mon.a (mon.0) 162 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-13T03:15:19.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: cluster 2024-02-13T03:15:19.019146+0000 mon.a (mon.0) 163 : cluster [DBG] mgrmap e13: y(active, since 76s) 2024-02-13T03:15:19.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: cluster 2024-02-13T03:15:19.026613+0000 mon.a (mon.0) 164 : cluster [INF] overall HEALTH_OK 2024-02-13T03:15:19.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: audit 2024-02-13T03:15:19.031968+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:19.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: audit 2024-02-13T03:15:19.039962+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:19.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: audit 2024-02-13T03:15:19.051605+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:19.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: audit 2024-02-13T03:15:19.052958+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:19.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:19 smithi183 bash[20824]: audit 2024-02-13T03:15:19.054418+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:19.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:08.445275+0000 mon.a (mon.0) 130 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T03:15:19.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:08.445514+0000 mon.a (mon.0) 131 : cluster [INF] mon.a calling monitor election 2024-02-13T03:15:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:08.446901+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:08.935218+0000 mgr.y (mgr.14146) 40 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:09.440889+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:10.440980+0000 mon.a (mon.0) 134 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:10.444867+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-13T03:15:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:10.935585+0000 mgr.y (mgr.14146) 41 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:11.441150+0000 mon.a (mon.0) 135 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:11.998147+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:12.441312+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:12.997667+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:13.441320+0000 mon.a (mon.0) 139 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:13.467537+0000 mon.a (mon.0) 140 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-13T03:15:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:13.473526+0000 mon.a (mon.0) 141 : cluster [DBG] monmap e2: 2 mons at {a=v1:172.21.15.161:6789/0,b=v1:172.21.15.183:6789/0} removed_ranks: {} 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:13.474754+0000 mon.a (mon.0) 142 : cluster [DBG] fsmap 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:13.474841+0000 mon.a (mon.0) 143 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:13.475188+0000 mon.a (mon.0) 144 : cluster [DBG] mgrmap e13: y(active, since 70s) 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:13.482416+0000 mon.a (mon.0) 145 : cluster [INF] overall HEALTH_OK 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:13.485752+0000 mon.a (mon.0) 146 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:13.487560+0000 mon.a (mon.0) 147 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:14.005789+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:14.006250+0000 mon.a (mon.0) 151 : cluster [INF] mon.a calling monitor election 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:14.007678+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:14.007885+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:14.011252+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:14.936399+0000 mgr.y (mgr.14146) 43 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:14.998056+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:15.998159+0000 mon.a (mon.0) 155 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:16.001169+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-13T03:15:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:16.936861+0000 mgr.y (mgr.14146) 44 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:16.998267+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:17.998483+0000 mon.a (mon.0) 157 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:18.998594+0000 mon.a (mon.0) 158 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:19.011489+0000 mon.a (mon.0) 159 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-13T03:15:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:19.018013+0000 mon.a (mon.0) 160 : cluster [DBG] monmap e3: 3 mons at {a=v1:172.21.15.161:6789/0,b=v1:172.21.15.183:6789/0,c=v1:172.21.15.161:6790/0} removed_ranks: {} 2024-02-13T03:15:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:19.018840+0000 mon.a (mon.0) 161 : cluster [DBG] fsmap 2024-02-13T03:15:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:19.018885+0000 mon.a (mon.0) 162 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-13T03:15:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:19.019146+0000 mon.a (mon.0) 163 : cluster [DBG] mgrmap e13: y(active, since 76s) 2024-02-13T03:15:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: cluster 2024-02-13T03:15:19.026613+0000 mon.a (mon.0) 164 : cluster [INF] overall HEALTH_OK 2024-02-13T03:15:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:19.031968+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:19.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:19.039962+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:19.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:19.051605+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:19.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:19.052958+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:19.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[27879]: audit 2024-02-13T03:15:19.054418+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:19.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: audit 2024-02-13T03:15:14.005789+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T03:15:19.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: cluster 2024-02-13T03:15:14.006250+0000 mon.a (mon.0) 151 : cluster [INF] mon.a calling monitor election 2024-02-13T03:15:19.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: audit 2024-02-13T03:15:14.007678+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:15:19.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: audit 2024-02-13T03:15:14.007885+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: cluster 2024-02-13T03:15:14.011252+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-13T03:15:19.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: cluster 2024-02-13T03:15:14.936399+0000 mgr.y (mgr.14146) 43 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:19.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: audit 2024-02-13T03:15:14.998056+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: audit 2024-02-13T03:15:15.998159+0000 mon.a (mon.0) 155 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: cluster 2024-02-13T03:15:16.001169+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-13T03:15:19.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: cluster 2024-02-13T03:15:16.936861+0000 mgr.y (mgr.14146) 44 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:19.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: audit 2024-02-13T03:15:16.998267+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: audit 2024-02-13T03:15:17.998483+0000 mon.a (mon.0) 157 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: audit 2024-02-13T03:15:18.998594+0000 mon.a (mon.0) 158 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:19.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: cluster 2024-02-13T03:15:19.011489+0000 mon.a (mon.0) 159 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-13T03:15:19.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: cluster 2024-02-13T03:15:19.018013+0000 mon.a (mon.0) 160 : cluster [DBG] monmap e3: 3 mons at {a=v1:172.21.15.161:6789/0,b=v1:172.21.15.183:6789/0,c=v1:172.21.15.161:6790/0} removed_ranks: {} 2024-02-13T03:15:19.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: cluster 2024-02-13T03:15:19.018840+0000 mon.a (mon.0) 161 : cluster [DBG] fsmap 2024-02-13T03:15:19.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: cluster 2024-02-13T03:15:19.018885+0000 mon.a (mon.0) 162 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-13T03:15:19.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: cluster 2024-02-13T03:15:19.019146+0000 mon.a (mon.0) 163 : cluster [DBG] mgrmap e13: y(active, since 76s) 2024-02-13T03:15:19.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: cluster 2024-02-13T03:15:19.026613+0000 mon.a (mon.0) 164 : cluster [INF] overall HEALTH_OK 2024-02-13T03:15:19.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: audit 2024-02-13T03:15:19.031968+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:19.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: audit 2024-02-13T03:15:19.039962+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:19.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: audit 2024-02-13T03:15:19.051605+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:19.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: audit 2024-02-13T03:15:19.052958+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:19.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:19 smithi161 bash[19769]: audit 2024-02-13T03:15:19.054418+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:19.908 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-13T03:15:19.908 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph config generate-minimal-conf 2024-02-13T03:15:20.199 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[19769]: cluster 2024-02-13T03:15:18.937237+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:20.199 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[19769]: cephadm 2024-02-13T03:15:19.056514+0000 mgr.y (mgr.14146) 46 : cephadm [INF] Updating smithi161:/etc/ceph/ceph.conf 2024-02-13T03:15:20.200 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[19769]: cephadm 2024-02-13T03:15:19.057114+0000 mgr.y (mgr.14146) 47 : cephadm [INF] Updating smithi183:/etc/ceph/ceph.conf 2024-02-13T03:15:20.200 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[19769]: audit 2024-02-13T03:15:19.092257+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:20.200 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[19769]: audit 2024-02-13T03:15:19.096795+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:20.200 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[19769]: audit 2024-02-13T03:15:19.101591+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:20.200 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[19769]: cephadm 2024-02-13T03:15:19.103768+0000 mgr.y (mgr.14146) 48 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-13T03:15:20.200 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[19769]: audit 2024-02-13T03:15:19.104179+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T03:15:20.200 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[19769]: audit 2024-02-13T03:15:19.105302+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T03:15:20.200 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[19769]: audit 2024-02-13T03:15:19.106396+0000 mon.a (mon.0) 175 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:20.200 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[19769]: cephadm 2024-02-13T03:15:19.107421+0000 mgr.y (mgr.14146) 49 : cephadm [INF] Reconfiguring daemon mon.a on smithi161 2024-02-13T03:15:20.200 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[19769]: audit 2024-02-13T03:15:19.305734+0000 mon.a (mon.0) 176 : audit [DBG] from='client.? v1:172.21.15.183:0/1270585249' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T03:15:20.200 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[19769]: audit 2024-02-13T03:15:19.998599+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:20.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[27879]: cluster 2024-02-13T03:15:18.937237+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:20.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[27879]: cephadm 2024-02-13T03:15:19.056514+0000 mgr.y (mgr.14146) 46 : cephadm [INF] Updating smithi161:/etc/ceph/ceph.conf 2024-02-13T03:15:20.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[27879]: cephadm 2024-02-13T03:15:19.057114+0000 mgr.y (mgr.14146) 47 : cephadm [INF] Updating smithi183:/etc/ceph/ceph.conf 2024-02-13T03:15:20.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[27879]: audit 2024-02-13T03:15:19.092257+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:20.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[27879]: audit 2024-02-13T03:15:19.096795+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:20.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[27879]: audit 2024-02-13T03:15:19.101591+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:20.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[27879]: cephadm 2024-02-13T03:15:19.103768+0000 mgr.y (mgr.14146) 48 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-13T03:15:20.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[27879]: audit 2024-02-13T03:15:19.104179+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T03:15:20.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[27879]: audit 2024-02-13T03:15:19.105302+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T03:15:20.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[27879]: audit 2024-02-13T03:15:19.106396+0000 mon.a (mon.0) 175 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:20.201 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[27879]: cephadm 2024-02-13T03:15:19.107421+0000 mgr.y (mgr.14146) 49 : cephadm [INF] Reconfiguring daemon mon.a on smithi161 2024-02-13T03:15:20.202 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[27879]: audit 2024-02-13T03:15:19.305734+0000 mon.a (mon.0) 176 : audit [DBG] from='client.? v1:172.21.15.183:0/1270585249' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T03:15:20.202 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:20 smithi161 bash[27879]: audit 2024-02-13T03:15:19.998599+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:20 smithi183 bash[20824]: cluster 2024-02-13T03:15:18.937237+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:20 smithi183 bash[20824]: cephadm 2024-02-13T03:15:19.056514+0000 mgr.y (mgr.14146) 46 : cephadm [INF] Updating smithi161:/etc/ceph/ceph.conf 2024-02-13T03:15:20.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:20 smithi183 bash[20824]: cephadm 2024-02-13T03:15:19.057114+0000 mgr.y (mgr.14146) 47 : cephadm [INF] Updating smithi183:/etc/ceph/ceph.conf 2024-02-13T03:15:20.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:20 smithi183 bash[20824]: audit 2024-02-13T03:15:19.092257+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:20.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:20 smithi183 bash[20824]: audit 2024-02-13T03:15:19.096795+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:20.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:20 smithi183 bash[20824]: audit 2024-02-13T03:15:19.101591+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:20.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:20 smithi183 bash[20824]: cephadm 2024-02-13T03:15:19.103768+0000 mgr.y (mgr.14146) 48 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-13T03:15:20.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:20 smithi183 bash[20824]: audit 2024-02-13T03:15:19.104179+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T03:15:20.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:20 smithi183 bash[20824]: audit 2024-02-13T03:15:19.105302+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T03:15:20.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:20 smithi183 bash[20824]: audit 2024-02-13T03:15:19.106396+0000 mon.a (mon.0) 175 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:20.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:20 smithi183 bash[20824]: cephadm 2024-02-13T03:15:19.107421+0000 mgr.y (mgr.14146) 49 : cephadm [INF] Reconfiguring daemon mon.a on smithi161 2024-02-13T03:15:20.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:20 smithi183 bash[20824]: audit 2024-02-13T03:15:19.305734+0000 mon.a (mon.0) 176 : audit [DBG] from='client.? v1:172.21.15.183:0/1270585249' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T03:15:20.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:20 smithi183 bash[20824]: audit 2024-02-13T03:15:19.998599+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:15:22.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:22 smithi161 bash[19769]: cluster 2024-02-13T03:15:20.937711+0000 mgr.y (mgr.14146) 50 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:22.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:22 smithi161 bash[19769]: audit 2024-02-13T03:15:21.626362+0000 mon.a (mon.0) 178 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:22.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:22 smithi161 bash[19769]: audit 2024-02-13T03:15:21.627878+0000 mon.a (mon.0) 179 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T03:15:22.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:22 smithi161 bash[19769]: audit 2024-02-13T03:15:21.629297+0000 mon.a (mon.0) 180 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T03:15:22.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:22 smithi161 bash[19769]: audit 2024-02-13T03:15:21.630544+0000 mon.a (mon.0) 181 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:22.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:22 smithi161 bash[27879]: cluster 2024-02-13T03:15:20.937711+0000 mgr.y (mgr.14146) 50 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:22.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:22 smithi161 bash[27879]: audit 2024-02-13T03:15:21.626362+0000 mon.a (mon.0) 178 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:22.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:22 smithi161 bash[27879]: audit 2024-02-13T03:15:21.627878+0000 mon.a (mon.0) 179 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T03:15:22.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:22 smithi161 bash[27879]: audit 2024-02-13T03:15:21.629297+0000 mon.a (mon.0) 180 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T03:15:22.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:22 smithi161 bash[27879]: audit 2024-02-13T03:15:21.630544+0000 mon.a (mon.0) 181 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:22.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:22 smithi183 bash[20824]: cluster 2024-02-13T03:15:20.937711+0000 mgr.y (mgr.14146) 50 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:22.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:22 smithi183 bash[20824]: audit 2024-02-13T03:15:21.626362+0000 mon.a (mon.0) 178 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:22.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:22 smithi183 bash[20824]: audit 2024-02-13T03:15:21.627878+0000 mon.a (mon.0) 179 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T03:15:22.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:22 smithi183 bash[20824]: audit 2024-02-13T03:15:21.629297+0000 mon.a (mon.0) 180 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T03:15:22.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:22 smithi183 bash[20824]: audit 2024-02-13T03:15:21.630544+0000 mon.a (mon.0) 181 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:23.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:23 smithi161 bash[19769]: cephadm 2024-02-13T03:15:21.627377+0000 mgr.y (mgr.14146) 51 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-13T03:15:23.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:23 smithi161 bash[19769]: cephadm 2024-02-13T03:15:21.631583+0000 mgr.y (mgr.14146) 52 : cephadm [INF] Reconfiguring daemon mon.c on smithi161 2024-02-13T03:15:23.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:23 smithi161 bash[27879]: cephadm 2024-02-13T03:15:21.627377+0000 mgr.y (mgr.14146) 51 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-13T03:15:23.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:23 smithi161 bash[27879]: cephadm 2024-02-13T03:15:21.631583+0000 mgr.y (mgr.14146) 52 : cephadm [INF] Reconfiguring daemon mon.c on smithi161 2024-02-13T03:15:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:23 smithi183 bash[20824]: cephadm 2024-02-13T03:15:21.627377+0000 mgr.y (mgr.14146) 51 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-13T03:15:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:23 smithi183 bash[20824]: cephadm 2024-02-13T03:15:21.631583+0000 mgr.y (mgr.14146) 52 : cephadm [INF] Reconfiguring daemon mon.c on smithi161 2024-02-13T03:15:23.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2601894755 >> v1:172.21.15.161:6790/0 conn(0x7f4c64100710 legacy=0x7f4c64102b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:23.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2601894755 shutdown_connections 2024-02-13T03:15:23.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2601894755 >> 172.21.15.161:0/2601894755 conn(0x7f4c6406cc10 msgr2=0x7f4c640f4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:15:23.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2601894755 shutdown_connections 2024-02-13T03:15:23.687 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2601894755 wait complete. 2024-02-13T03:15:23.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c6b5e1700 1 Processor -- start 2024-02-13T03:15:23.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c6b5e1700 1 -- start start 2024-02-13T03:15:23.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c6b5e1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4c640f8b20 con 0x7f4c64104220 2024-02-13T03:15:23.688 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c6b5e1700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4c640f8cf0 con 0x7f4c640fd8c0 2024-02-13T03:15:23.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c6b5e1700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4c640f8ee0 con 0x7f4c64100710 2024-02-13T03:15:23.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c69b7e700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f4c64104220 0x7f4c641183b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:38796/0 (socket says 172.21.15.161:38796) 2024-02-13T03:15:23.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c6937d700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f4c640fd8c0 0x7f4c640fc9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.161:53084/0 (socket says 172.21.15.161:53084) 2024-02-13T03:15:23.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c69b7e700 1 -- 172.21.15.161:0/2867453759 learned_addr learned my addr 172.21.15.161:0/2867453759 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:15:23.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c68b7c700 1 --1- 172.21.15.161:0/2867453759 >> v1:172.21.15.161:6790/0 conn(0x7f4c64100710 0x7f4c64114c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:58814/0 (socket says 172.21.15.161:58814) 2024-02-13T03:15:23.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895923918 0 0) 0x7f4c640f8b20 con 0x7f4c64104220 2024-02-13T03:15:23.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c4c0033f0 con 0x7f4c64104220 2024-02-13T03:15:23.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798589851 0 0) 0x7f4c640f8ee0 con 0x7f4c64100710 2024-02-13T03:15:23.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c640f8b20 con 0x7f4c64100710 2024-02-13T03:15:23.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807153671 0 0) 0x7f4c640f8cf0 con 0x7f4c640fd8c0 2024-02-13T03:15:23.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c640f8ee0 con 0x7f4c640fd8c0 2024-02-13T03:15:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1045523583 0 0) 0x7f4c4c0033f0 con 0x7f4c64104220 2024-02-13T03:15:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4c640f8cf0 con 0x7f4c64104220 2024-02-13T03:15:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f4c5c002cd0 con 0x7f4c64104220 2024-02-13T03:15:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 116239234 0 0) 0x7f4c640f8b20 con 0x7f4c64100710 2024-02-13T03:15:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4c4c0033f0 con 0x7f4c64100710 2024-02-13T03:15:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f4c58002d20 con 0x7f4c64100710 2024-02-13T03:15:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 681627906 0 0) 0x7f4c640f8ee0 con 0x7f4c640fd8c0 2024-02-13T03:15:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4c640f8b20 con 0x7f4c640fd8c0 2024-02-13T03:15:23.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f4c540019d0 con 0x7f4c640fd8c0 2024-02-13T03:15:23.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1565444237 0 0) 0x7f4c640f8cf0 con 0x7f4c64104220 2024-02-13T03:15:23.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 >> v1:172.21.15.161:6790/0 conn(0x7f4c64100710 legacy=0x7f4c64114c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:23.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 >> v1:172.21.15.183:6789/0 conn(0x7f4c640fd8c0 legacy=0x7f4c640fc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:23.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6411cb40 con 0x7f4c64104220 2024-02-13T03:15:23.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.683+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2867453759 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c64119b30 con 0x7f4c64104220 2024-02-13T03:15:23.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.687+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2867453759 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c64119fa0 con 0x7f4c64104220 2024-02-13T03:15:23.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.687+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f4c5c0030a0 con 0x7f4c64104220 2024-02-13T03:15:23.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.687+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f4c5c005ed0 con 0x7f4c64104220 2024-02-13T03:15:23.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.687+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7f4c5c00f7b0 con 0x7f4c64104220 2024-02-13T03:15:23.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.687+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 1047224232 0 0) 0x7f4c5c03eda0 con 0x7f4c64104220 2024-02-13T03:15:23.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.687+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2867453759 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c50004f20 con 0x7f4c64104220 2024-02-13T03:15:23.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.691+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f4c5c011200 con 0x7f4c64104220 2024-02-13T03:15:23.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.871+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2867453759 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f4c500051d0 con 0x7f4c64104220 2024-02-13T03:15:23.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.875+0000 7f4c627fc700 1 -- 172.21.15.161:0/2867453759 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+193 (unknown 2189217731 0 2055147285) 0x7f4c5c013130 con 0x7f4c64104220 2024-02-13T03:15:23.878 INFO:teuthology.orchestra.run.smithi161.stdout:# minimal ceph.conf for b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:15:23.878 INFO:teuthology.orchestra.run.smithi161.stdout:[global] 2024-02-13T03:15:23.878 INFO:teuthology.orchestra.run.smithi161.stdout: fsid = b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:15:23.878 INFO:teuthology.orchestra.run.smithi161.stdout: mon_host = 172.21.15.161:6789/0 172.21.15.183:6789/0 v1:172.21.15.161:6790/0 2024-02-13T03:15:23.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.879+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2867453759 >> v1:172.21.15.161:6800/1187535987 conn(0x7f4c4c033a00 legacy=0x7f4c4c035ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:23.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.879+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2867453759 >> v1:172.21.15.161:6789/0 conn(0x7f4c64104220 legacy=0x7f4c641183b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:23.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.879+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2867453759 shutdown_connections 2024-02-13T03:15:23.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.879+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2867453759 >> 172.21.15.161:0/2867453759 conn(0x7f4c6406cc10 msgr2=0x7f4c64107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:15:23.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.879+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2867453759 shutdown_connections 2024-02-13T03:15:23.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:23.879+0000 7f4c6b5e1700 1 -- 172.21.15.161:0/2867453759 wait complete. 2024-02-13T03:15:24.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:24 smithi161 bash[19769]: cluster 2024-02-13T03:15:22.938153+0000 mgr.y (mgr.14146) 53 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:24.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:24 smithi161 bash[19769]: audit 2024-02-13T03:15:23.877853+0000 mon.a (mon.0) 182 : audit [DBG] from='client.? v1:172.21.15.161:0/2867453759' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:24 smithi161 bash[27879]: cluster 2024-02-13T03:15:22.938153+0000 mgr.y (mgr.14146) 53 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:24 smithi161 bash[27879]: audit 2024-02-13T03:15:23.877853+0000 mon.a (mon.0) 182 : audit [DBG] from='client.? v1:172.21.15.161:0/2867453759' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:24 smithi183 bash[20824]: cluster 2024-02-13T03:15:22.938153+0000 mgr.y (mgr.14146) 53 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:24 smithi183 bash[20824]: audit 2024-02-13T03:15:23.877853+0000 mon.a (mon.0) 182 : audit [DBG] from='client.? v1:172.21.15.161:0/2867453759' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:24.626 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-13T03:15:24.626 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:15:24.626 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-02-13T03:15:24.641 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:15:24.641 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-13T03:15:24.700 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-02-13T03:15:24.700 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-02-13T03:15:24.714 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-02-13T03:15:24.714 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-13T03:15:24.771 INFO:tasks.cephadm:Adding mgr.y on smithi161 2024-02-13T03:15:24.771 INFO:tasks.cephadm:Adding mgr.x on smithi183 2024-02-13T03:15:24.771 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch apply mgr '2;smithi161=y;smithi183=x' 2024-02-13T03:15:25.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:25 smithi161 bash[19769]: audit 2024-02-13T03:15:24.243823+0000 mon.a (mon.0) 183 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:25.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:25 smithi161 bash[19769]: cephadm 2024-02-13T03:15:24.244821+0000 mgr.y (mgr.14146) 54 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-13T03:15:25.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:25 smithi161 bash[19769]: audit 2024-02-13T03:15:24.245404+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T03:15:25.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:25 smithi161 bash[19769]: audit 2024-02-13T03:15:24.246937+0000 mon.a (mon.0) 185 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T03:15:25.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:25 smithi161 bash[19769]: audit 2024-02-13T03:15:24.248370+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:25.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:25 smithi161 bash[19769]: cephadm 2024-02-13T03:15:24.249658+0000 mgr.y (mgr.14146) 55 : cephadm [INF] Reconfiguring daemon mon.b on smithi183 2024-02-13T03:15:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:25 smithi161 bash[27879]: audit 2024-02-13T03:15:24.243823+0000 mon.a (mon.0) 183 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:25 smithi161 bash[27879]: cephadm 2024-02-13T03:15:24.244821+0000 mgr.y (mgr.14146) 54 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-13T03:15:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:25 smithi161 bash[27879]: audit 2024-02-13T03:15:24.245404+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T03:15:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:25 smithi161 bash[27879]: audit 2024-02-13T03:15:24.246937+0000 mon.a (mon.0) 185 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T03:15:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:25 smithi161 bash[27879]: audit 2024-02-13T03:15:24.248370+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:25 smithi161 bash[27879]: cephadm 2024-02-13T03:15:24.249658+0000 mgr.y (mgr.14146) 55 : cephadm [INF] Reconfiguring daemon mon.b on smithi183 2024-02-13T03:15:25.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:25 smithi183 bash[20824]: audit 2024-02-13T03:15:24.243823+0000 mon.a (mon.0) 183 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:25.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:25 smithi183 bash[20824]: cephadm 2024-02-13T03:15:24.244821+0000 mgr.y (mgr.14146) 54 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-13T03:15:25.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:25 smithi183 bash[20824]: audit 2024-02-13T03:15:24.245404+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T03:15:25.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:25 smithi183 bash[20824]: audit 2024-02-13T03:15:24.246937+0000 mon.a (mon.0) 185 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T03:15:25.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:25 smithi183 bash[20824]: audit 2024-02-13T03:15:24.248370+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:25.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:25 smithi183 bash[20824]: cephadm 2024-02-13T03:15:24.249658+0000 mgr.y (mgr.14146) 55 : cephadm [INF] Reconfiguring daemon mon.b on smithi183 2024-02-13T03:15:26.298 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:26 smithi183 bash[20824]: cluster 2024-02-13T03:15:24.938605+0000 mgr.y (mgr.14146) 56 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:26.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:26 smithi161 bash[27879]: cluster 2024-02-13T03:15:24.938605+0000 mgr.y (mgr.14146) 56 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:26.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:26 smithi161 bash[19769]: cluster 2024-02-13T03:15:24.938605+0000 mgr.y (mgr.14146) 56 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:27.737 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.733+0000 7f5f62762700 1 -- 172.21.15.183:0/443120505 >> v1:172.21.15.161:6789/0 conn(0x7f5f5c0f78d0 legacy=0x7f5f5c0f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:27.737 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.733+0000 7f5f62762700 1 -- 172.21.15.183:0/443120505 shutdown_connections 2024-02-13T03:15:27.737 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.733+0000 7f5f62762700 1 -- 172.21.15.183:0/443120505 >> 172.21.15.183:0/443120505 conn(0x7f5f5c0f3050 msgr2=0x7f5f5c0f5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:15:27.737 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.733+0000 7f5f62762700 1 -- 172.21.15.183:0/443120505 shutdown_connections 2024-02-13T03:15:27.738 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.733+0000 7f5f62762700 1 -- 172.21.15.183:0/443120505 wait complete. 2024-02-13T03:15:27.738 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.737+0000 7f5f62762700 1 Processor -- start 2024-02-13T03:15:27.738 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.737+0000 7f5f62762700 1 -- start start 2024-02-13T03:15:27.738 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.737+0000 7f5f62762700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5f5c114b70 con 0x7f5f5c0ff9d0 2024-02-13T03:15:27.738 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.737+0000 7f5f62762700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5f5c115d50 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.739 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.737+0000 7f5f62762700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5f5c116f50 con 0x7f5f5c0fbda0 2024-02-13T03:15:27.739 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.737+0000 7f5f5bfff700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f5f5c0f78d0 0x7f5f5c06ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:60280/0 (socket says 172.21.15.183:60280) 2024-02-13T03:15:27.739 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.737+0000 7f5f5bfff700 1 -- 172.21.15.183:0/849754161 learned_addr learned my addr 172.21.15.183:0/849754161 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:15:27.743 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749582432 0 0) 0x7f5f5c115d50 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.743 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f48003110 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.744 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795642867 0 0) 0x7f5f5c114b70 con 0x7f5f5c0ff9d0 2024-02-13T03:15:27.744 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f5c115d50 con 0x7f5f5c0ff9d0 2024-02-13T03:15:27.745 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3604212856 0 0) 0x7f5f48003110 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.745 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5f5c114b70 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.745 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5f54001930 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.745 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3590587819 0 0) 0x7f5f5c115d50 con 0x7f5f5c0ff9d0 2024-02-13T03:15:27.745 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5f48003110 con 0x7f5f5c0ff9d0 2024-02-13T03:15:27.745 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5f4c0018e0 con 0x7f5f5c0ff9d0 2024-02-13T03:15:27.745 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 59973106 0 0) 0x7f5f5c114b70 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.745 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 >> v1:172.21.15.161:6790/0 conn(0x7f5f5c0fbda0 legacy=0x7f5f5c10f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:27.746 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 >> v1:172.21.15.161:6789/0 conn(0x7f5f5c0ff9d0 legacy=0x7f5f5c113230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:27.746 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c118150 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.746 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f62762700 1 -- 172.21.15.183:0/849754161 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5c114d60 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.746 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f62762700 1 -- 172.21.15.183:0/849754161 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c115250 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.746 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f5f54002820 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.746 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.741+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5f54003dd0 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.746 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.745+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7f5f54004010 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.747 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.745+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 1047224232 0 0) 0x7f5f5403f1b0 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.747 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.745+0000 7f5f62762700 1 -- 172.21.15.183:0/849754161 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f3c004f20 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.753+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f5f54015140 con 0x7f5f5c0f78d0 2024-02-13T03:15:27.955 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.953+0000 7f5f62762700 1 -- 172.21.15.183:0/849754161 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi161=y;smithi183=x", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f3c000b90 con 0x7f5f48033a30 2024-02-13T03:15:27.967 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.965+0000 7f5f597fa700 1 -- 172.21.15.183:0/849754161 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7f5f3c000b90 con 0x7f5f48033a30 2024-02-13T03:15:27.968 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mgr update... 2024-02-13T03:15:27.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.969+0000 7f5f62762700 1 -- 172.21.15.183:0/849754161 >> v1:172.21.15.161:6800/1187535987 conn(0x7f5f48033a30 legacy=0x7f5f48035ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:27.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.969+0000 7f5f62762700 1 -- 172.21.15.183:0/849754161 >> v1:172.21.15.183:6789/0 conn(0x7f5f5c0f78d0 legacy=0x7f5f5c06ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:27.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.969+0000 7f5f62762700 1 -- 172.21.15.183:0/849754161 shutdown_connections 2024-02-13T03:15:27.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.969+0000 7f5f62762700 1 -- 172.21.15.183:0/849754161 >> 172.21.15.183:0/849754161 conn(0x7f5f5c0f3050 msgr2=0x7f5f5c101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:15:27.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.969+0000 7f5f62762700 1 -- 172.21.15.183:0/849754161 shutdown_connections 2024-02-13T03:15:27.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:15:27.969+0000 7f5f62762700 1 -- 172.21.15.183:0/849754161 wait complete. 2024-02-13T03:15:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:27 smithi183 bash[20824]: audit 2024-02-13T03:15:26.733166+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:27 smithi183 bash[20824]: audit 2024-02-13T03:15:26.735463+0000 mon.a (mon.0) 188 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:27 smithi183 bash[20824]: cluster 2024-02-13T03:15:26.939058+0000 mgr.y (mgr.14146) 57 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:27 smithi183 bash[20824]: audit 2024-02-13T03:15:27.057222+0000 mon.a (mon.0) 189 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:27 smithi183 bash[20824]: audit 2024-02-13T03:15:27.058447+0000 mon.a (mon.0) 190 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:27.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:27 smithi183 bash[20824]: audit 2024-02-13T03:15:27.067720+0000 mon.a (mon.0) 191 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:28.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:27 smithi161 bash[19769]: audit 2024-02-13T03:15:26.733166+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:28.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:27 smithi161 bash[19769]: audit 2024-02-13T03:15:26.735463+0000 mon.a (mon.0) 188 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:28.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:27 smithi161 bash[19769]: cluster 2024-02-13T03:15:26.939058+0000 mgr.y (mgr.14146) 57 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:28.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:27 smithi161 bash[19769]: audit 2024-02-13T03:15:27.057222+0000 mon.a (mon.0) 189 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:28.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:27 smithi161 bash[19769]: audit 2024-02-13T03:15:27.058447+0000 mon.a (mon.0) 190 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:28.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:27 smithi161 bash[19769]: audit 2024-02-13T03:15:27.067720+0000 mon.a (mon.0) 191 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:28.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:27 smithi161 bash[27879]: audit 2024-02-13T03:15:26.733166+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:28.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:27 smithi161 bash[27879]: audit 2024-02-13T03:15:26.735463+0000 mon.a (mon.0) 188 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:28.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:27 smithi161 bash[27879]: cluster 2024-02-13T03:15:26.939058+0000 mgr.y (mgr.14146) 57 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:28.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:27 smithi161 bash[27879]: audit 2024-02-13T03:15:27.057222+0000 mon.a (mon.0) 189 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:28.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:27 smithi161 bash[27879]: audit 2024-02-13T03:15:27.058447+0000 mon.a (mon.0) 190 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:28.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:27 smithi161 bash[27879]: audit 2024-02-13T03:15:27.067720+0000 mon.a (mon.0) 191 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:28.663 DEBUG:teuthology.orchestra.run.smithi183:mgr.x> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mgr.x.service 2024-02-13T03:15:28.666 INFO:tasks.cephadm:Deploying OSDs... 2024-02-13T03:15:28.666 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:15:28.666 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-02-13T03:15:28.673 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-13T03:15:28.673 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-02-13T03:15:28.724 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-13T03:15:28.724 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T03:15:28.724 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 340 Links: 1 2024-02-13T03:15:28.724 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T03:15:28.724 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-02-13 03:09:01.075230309 +0000 2024-02-13T03:15:28.724 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-02-13 03:09:00.911234213 +0000 2024-02-13T03:15:28.725 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-02-13 03:09:00.911234213 +0000 2024-02-13T03:15:28.725 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-02-13T03:15:28.725 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-13T03:15:28.777 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-02-13T03:15:28.777 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-02-13T03:15:28.777 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000418293 s, 1.2 MB/s 2024-02-13T03:15:28.778 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-13T03:15:28.829 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-02-13T03:15:28.880 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-13T03:15:28.880 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T03:15:28.880 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 349 Links: 1 2024-02-13T03:15:28.880 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T03:15:28.880 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-02-13 03:09:01.227226691 +0000 2024-02-13T03:15:28.880 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-02-13 03:09:01.223226786 +0000 2024-02-13T03:15:28.880 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-02-13 03:09:01.223226786 +0000 2024-02-13T03:15:28.880 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-02-13T03:15:28.880 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-13T03:15:28.938 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-02-13T03:15:28.939 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-02-13T03:15:28.939 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000390104 s, 1.3 MB/s 2024-02-13T03:15:28.940 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-13T03:15:28.993 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-02-13T03:15:28.995 INFO:journalctl@ceph.mgr.x.smithi183.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:15:29.044 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-13T03:15:29.044 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T03:15:29.044 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 355 Links: 1 2024-02-13T03:15:29.044 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T03:15:29.045 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-02-13 03:09:01.683215836 +0000 2024-02-13T03:15:29.045 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-02-13 03:09:01.515219835 +0000 2024-02-13T03:15:29.045 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-02-13 03:09:01.515219835 +0000 2024-02-13T03:15:29.045 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-02-13T03:15:29.045 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-13T03:15:29.097 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[19769]: audit 2024-02-13T03:15:27.956831+0000 mgr.y (mgr.14146) 58 : audit [DBG] from='client.24109 v1:172.21.15.183:0/849754161' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi161=y;smithi183=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:15:29.097 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[19769]: cephadm 2024-02-13T03:15:27.960394+0000 mgr.y (mgr.14146) 59 : cephadm [INF] Saving service mgr spec with placement smithi161=y;smithi183=x;count:2 2024-02-13T03:15:29.097 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[19769]: audit 2024-02-13T03:15:27.966516+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:29.097 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[19769]: audit 2024-02-13T03:15:27.986410+0000 mon.a (mon.0) 193 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:29.097 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[19769]: audit 2024-02-13T03:15:27.989723+0000 mon.a (mon.0) 194 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:29.097 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[19769]: audit 2024-02-13T03:15:27.991388+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:29.097 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[19769]: audit 2024-02-13T03:15:27.998966+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:29.097 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[19769]: audit 2024-02-13T03:15:28.001265+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T03:15:29.097 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[19769]: audit 2024-02-13T03:15:28.006720+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-13T03:15:29.097 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[19769]: audit 2024-02-13T03:15:28.008195+0000 mon.a (mon.0) 199 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T03:15:29.097 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[19769]: audit 2024-02-13T03:15:28.009909+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:29.098 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[19769]: cephadm 2024-02-13T03:15:28.011145+0000 mgr.y (mgr.14146) 60 : cephadm [INF] Deploying daemon mgr.x on smithi183 2024-02-13T03:15:29.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[27879]: audit 2024-02-13T03:15:27.956831+0000 mgr.y (mgr.14146) 58 : audit [DBG] from='client.24109 v1:172.21.15.183:0/849754161' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi161=y;smithi183=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:15:29.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[27879]: cephadm 2024-02-13T03:15:27.960394+0000 mgr.y (mgr.14146) 59 : cephadm [INF] Saving service mgr spec with placement smithi161=y;smithi183=x;count:2 2024-02-13T03:15:29.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[27879]: audit 2024-02-13T03:15:27.966516+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:29.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[27879]: audit 2024-02-13T03:15:27.986410+0000 mon.a (mon.0) 193 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:29.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[27879]: audit 2024-02-13T03:15:27.989723+0000 mon.a (mon.0) 194 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:29.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[27879]: audit 2024-02-13T03:15:27.991388+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:29.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[27879]: audit 2024-02-13T03:15:27.998966+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:29.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[27879]: audit 2024-02-13T03:15:28.001265+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T03:15:29.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[27879]: audit 2024-02-13T03:15:28.006720+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-13T03:15:29.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[27879]: audit 2024-02-13T03:15:28.008195+0000 mon.a (mon.0) 199 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T03:15:29.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[27879]: audit 2024-02-13T03:15:28.009909+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:29.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:28 smithi161 bash[27879]: cephadm 2024-02-13T03:15:28.011145+0000 mgr.y (mgr.14146) 60 : cephadm [INF] Deploying daemon mgr.x on smithi183 2024-02-13T03:15:29.100 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-02-13T03:15:29.100 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-02-13T03:15:29.100 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.0004129 s, 1.2 MB/s 2024-02-13T03:15:29.100 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-13T03:15:29.149 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-02-13T03:15:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-13T03:15:29.200 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T03:15:29.200 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 361 Links: 1 2024-02-13T03:15:29.200 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T03:15:29.200 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-02-13 03:09:01.979208790 +0000 2024-02-13T03:15:29.200 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-02-13 03:09:01.807212884 +0000 2024-02-13T03:15:29.201 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-02-13 03:09:01.807212884 +0000 2024-02-13T03:15:29.201 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-02-13T03:15:29.201 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-13T03:15:29.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:28 smithi183 bash[20824]: audit 2024-02-13T03:15:27.956831+0000 mgr.y (mgr.14146) 58 : audit [DBG] from='client.24109 v1:172.21.15.183:0/849754161' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi161=y;smithi183=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:15:29.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:28 smithi183 bash[20824]: cephadm 2024-02-13T03:15:27.960394+0000 mgr.y (mgr.14146) 59 : cephadm [INF] Saving service mgr spec with placement smithi161=y;smithi183=x;count:2 2024-02-13T03:15:29.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:28 smithi183 bash[20824]: audit 2024-02-13T03:15:27.966516+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:29.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:28 smithi183 bash[20824]: audit 2024-02-13T03:15:27.986410+0000 mon.a (mon.0) 193 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:29.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:28 smithi183 bash[20824]: audit 2024-02-13T03:15:27.989723+0000 mon.a (mon.0) 194 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:29.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:28 smithi183 bash[20824]: audit 2024-02-13T03:15:27.991388+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:29.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:28 smithi183 bash[20824]: audit 2024-02-13T03:15:27.998966+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:29.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:28 smithi183 bash[20824]: audit 2024-02-13T03:15:28.001265+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T03:15:29.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:28 smithi183 bash[20824]: audit 2024-02-13T03:15:28.006720+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-13T03:15:29.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:28 smithi183 bash[20824]: audit 2024-02-13T03:15:28.008195+0000 mon.a (mon.0) 199 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T03:15:29.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:28 smithi183 bash[20824]: audit 2024-02-13T03:15:28.009909+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:29.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:28 smithi183 bash[20824]: cephadm 2024-02-13T03:15:28.011145+0000 mgr.y (mgr.14146) 60 : cephadm [INF] Deploying daemon mgr.x on smithi183 2024-02-13T03:15:29.254 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-02-13T03:15:29.254 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-02-13T03:15:29.254 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000322119 s, 1.6 MB/s 2024-02-13T03:15:29.255 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-13T03:15:29.305 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-02-13T03:15:29.305 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-02-13T03:15:29.310 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-13T03:15:29.310 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-02-13T03:15:29.360 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-13T03:15:29.360 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T03:15:29.360 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 340 Links: 1 2024-02-13T03:15:29.360 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T03:15:29.360 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-02-13 03:08:53.327099435 +0000 2024-02-13T03:15:29.360 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-02-13 03:08:53.171103356 +0000 2024-02-13T03:15:29.360 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-02-13 03:08:53.171103356 +0000 2024-02-13T03:15:29.360 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2024-02-13T03:15:29.361 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-13T03:15:29.413 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-02-13T03:15:29.413 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-02-13T03:15:29.413 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000544527 s, 940 kB/s 2024-02-13T03:15:29.414 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-13T03:15:29.464 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-02-13T03:15:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-13T03:15:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T03:15:29.512 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 347 Links: 1 2024-02-13T03:15:29.512 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T03:15:29.512 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-02-13 03:08:53.479095614 +0000 2024-02-13T03:15:29.512 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-02-13 03:08:53.475095716 +0000 2024-02-13T03:15:29.512 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-02-13 03:08:53.475095716 +0000 2024-02-13T03:15:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2024-02-13T03:15:29.513 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-13T03:15:29.567 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-02-13T03:15:29.567 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-02-13T03:15:29.567 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000410562 s, 1.2 MB/s 2024-02-13T03:15:29.568 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-13T03:15:29.621 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-02-13T03:15:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-13T03:15:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T03:15:29.672 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 353 Links: 1 2024-02-13T03:15:29.672 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T03:15:29.672 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-02-13 03:08:53.907084857 +0000 2024-02-13T03:15:29.672 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-02-13 03:08:53.743088980 +0000 2024-02-13T03:15:29.673 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-02-13 03:08:53.743088980 +0000 2024-02-13T03:15:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2024-02-13T03:15:29.673 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-13T03:15:29.726 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-02-13T03:15:29.727 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-02-13T03:15:29.727 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000396088 s, 1.3 MB/s 2024-02-13T03:15:29.729 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-13T03:15:29.781 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-02-13T03:15:29.831 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-13T03:15:29.831 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T03:15:29.831 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-02-13T03:15:29.831 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T03:15:29.831 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-02-13 03:08:54.183077922 +0000 2024-02-13T03:15:29.831 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-02-13 03:08:54.035081641 +0000 2024-02-13T03:15:29.831 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-02-13 03:08:54.035081641 +0000 2024-02-13T03:15:29.832 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2024-02-13T03:15:29.832 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-13T03:15:29.885 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-02-13T03:15:29.885 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-02-13T03:15:29.885 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000362935 s, 1.4 MB/s 2024-02-13T03:15:29.886 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-13T03:15:29.937 INFO:tasks.cephadm:Deploying osd.0 on smithi161 with /dev/vg_nvme/lv_4... 2024-02-13T03:15:29.937 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-13T03:15:30.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:30 smithi183 bash[20824]: cluster 2024-02-13T03:15:28.939471+0000 mgr.y (mgr.14146) 61 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:30.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:30 smithi161 bash[19769]: cluster 2024-02-13T03:15:28.939471+0000 mgr.y (mgr.14146) 61 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:30.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:30 smithi161 bash[27879]: cluster 2024-02-13T03:15:28.939471+0000 mgr.y (mgr.14146) 61 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:30.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:30 smithi183 systemd[1]: Started Ceph mgr.x for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:15:31.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:31 smithi183 bash[22114]: debug 2024-02-13T03:15:31.373+0000 7f403e56f700 1 -- 172.21.15.183:0/931592718 <== mon.2 v1:172.21.15.161:6790/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 1547003632 0 0) 0x55ad60b349c0 con 0x55ad60c28800 2024-02-13T03:15:31.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:31 smithi183 bash[20824]: audit 2024-02-13T03:15:30.315051+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:31.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:31 smithi183 bash[20824]: audit 2024-02-13T03:15:30.319629+0000 mon.a (mon.0) 202 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:31.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:31 smithi161 bash[19769]: audit 2024-02-13T03:15:30.315051+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:31.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:31 smithi161 bash[19769]: audit 2024-02-13T03:15:30.319629+0000 mon.a (mon.0) 202 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:31 smithi161 bash[27879]: audit 2024-02-13T03:15:30.315051+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:31 smithi161 bash[27879]: audit 2024-02-13T03:15:30.319629+0000 mon.a (mon.0) 202 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:31.987 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:31 smithi183 bash[22114]: debug 2024-02-13T03:15:31.525+0000 7f404eb6b680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T03:15:31.987 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:31 smithi183 bash[22114]: debug 2024-02-13T03:15:31.629+0000 7f404eb6b680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T03:15:32.324 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:32 smithi183 bash[22114]: debug 2024-02-13T03:15:32.061+0000 7f404eb6b680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T03:15:32.638 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:32 smithi161 bash[19769]: cluster 2024-02-13T03:15:30.939896+0000 mgr.y (mgr.14146) 62 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:32.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:32 smithi161 bash[27879]: cluster 2024-02-13T03:15:30.939896+0000 mgr.y (mgr.14146) 62 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:32 smithi183 bash[20824]: cluster 2024-02-13T03:15:30.939896+0000 mgr.y (mgr.14146) 62 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:33.135 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-13T03:15:33.136 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-13T03:15:33.136 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-02-13T03:15:33.136 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-02-13T03:15:33.136 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0137107 s, 765 MB/s 2024-02-13T03:15:33.136 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 2024-02-13T03:15:33.136 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-02-13T03:15:33.237 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:32 smithi183 bash[22114]: debug 2024-02-13T03:15:32.953+0000 7f404eb6b680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T03:15:33.237 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:33 smithi183 bash[22114]: debug 2024-02-13T03:15:33.045+0000 7f404eb6b680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T03:15:33.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:33 smithi183 bash[22114]: debug 2024-02-13T03:15:33.285+0000 7f404eb6b680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T03:15:33.746 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-02-13T03:15:34.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:34 smithi183 bash[20824]: cluster 2024-02-13T03:15:32.940226+0000 mgr.y (mgr.14146) 63 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:34.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:34 smithi183 bash[20824]: audit 2024-02-13T03:15:34.040425+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:34.651 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:34 smithi183 bash[22114]: debug 2024-02-13T03:15:34.393+0000 7f404eb6b680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T03:15:34.652 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:34 smithi183 bash[22114]: debug 2024-02-13T03:15:34.553+0000 7f404eb6b680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T03:15:34.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:34 smithi161 bash[19769]: cluster 2024-02-13T03:15:32.940226+0000 mgr.y (mgr.14146) 63 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:34.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:34 smithi161 bash[19769]: audit 2024-02-13T03:15:34.040425+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:34.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:34 smithi161 bash[27879]: cluster 2024-02-13T03:15:32.940226+0000 mgr.y (mgr.14146) 63 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:34.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:34 smithi161 bash[27879]: audit 2024-02-13T03:15:34.040425+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:34.975 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:34 smithi183 bash[22114]: debug 2024-02-13T03:15:34.649+0000 7f404eb6b680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T03:15:34.976 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:34 smithi183 bash[22114]: debug 2024-02-13T03:15:34.757+0000 7f404eb6b680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T03:15:35.237 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:34 smithi183 bash[22114]: debug 2024-02-13T03:15:34.973+0000 7f404eb6b680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T03:15:35.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:35 smithi183 bash[22114]: debug 2024-02-13T03:15:35.365+0000 7f404eb6b680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T03:15:35.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:35 smithi183 bash[22114]: debug 2024-02-13T03:15:35.485+0000 7f404eb6b680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T03:15:36.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[27879]: cluster 2024-02-13T03:15:34.940646+0000 mgr.y (mgr.14146) 64 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:36.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[27879]: audit 2024-02-13T03:15:35.387445+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:36.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[27879]: audit 2024-02-13T03:15:35.388647+0000 mon.a (mon.0) 205 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:36.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[27879]: audit 2024-02-13T03:15:35.389675+0000 mon.a (mon.0) 206 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:36.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[27879]: audit 2024-02-13T03:15:35.396150+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:36.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[27879]: cephadm 2024-02-13T03:15:35.398373+0000 mgr.y (mgr.14146) 65 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-13T03:15:36.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[27879]: audit 2024-02-13T03:15:35.398709+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T03:15:36.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[27879]: audit 2024-02-13T03:15:35.399816+0000 mon.a (mon.0) 209 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T03:15:36.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[27879]: audit 2024-02-13T03:15:35.400635+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:36.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[27879]: cephadm 2024-02-13T03:15:35.401423+0000 mgr.y (mgr.14146) 66 : cephadm [INF] Reconfiguring daemon mgr.y on smithi161 2024-02-13T03:15:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[19769]: cluster 2024-02-13T03:15:34.940646+0000 mgr.y (mgr.14146) 64 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:36.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[19769]: audit 2024-02-13T03:15:35.387445+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:36.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[19769]: audit 2024-02-13T03:15:35.388647+0000 mon.a (mon.0) 205 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:36.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[19769]: audit 2024-02-13T03:15:35.389675+0000 mon.a (mon.0) 206 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:36.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[19769]: audit 2024-02-13T03:15:35.396150+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:36.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[19769]: cephadm 2024-02-13T03:15:35.398373+0000 mgr.y (mgr.14146) 65 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-13T03:15:36.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[19769]: audit 2024-02-13T03:15:35.398709+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T03:15:36.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[19769]: audit 2024-02-13T03:15:35.399816+0000 mon.a (mon.0) 209 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T03:15:36.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[19769]: audit 2024-02-13T03:15:35.400635+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:36.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:36 smithi161 bash[19769]: cephadm 2024-02-13T03:15:35.401423+0000 mgr.y (mgr.14146) 66 : cephadm [INF] Reconfiguring daemon mgr.y on smithi161 2024-02-13T03:15:36.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[22114]: debug 2024-02-13T03:15:36.361+0000 7f404eb6b680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T03:15:36.738 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[22114]: debug 2024-02-13T03:15:36.465+0000 7f404eb6b680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T03:15:36.738 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[22114]: debug 2024-02-13T03:15:36.577+0000 7f404eb6b680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T03:15:36.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[20824]: cluster 2024-02-13T03:15:34.940646+0000 mgr.y (mgr.14146) 64 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:36.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[20824]: audit 2024-02-13T03:15:35.387445+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:36.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[20824]: audit 2024-02-13T03:15:35.388647+0000 mon.a (mon.0) 205 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:36.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[20824]: audit 2024-02-13T03:15:35.389675+0000 mon.a (mon.0) 206 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:36.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[20824]: audit 2024-02-13T03:15:35.396150+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:36.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[20824]: cephadm 2024-02-13T03:15:35.398373+0000 mgr.y (mgr.14146) 65 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-13T03:15:36.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[20824]: audit 2024-02-13T03:15:35.398709+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T03:15:36.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[20824]: audit 2024-02-13T03:15:35.399816+0000 mon.a (mon.0) 209 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T03:15:36.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[20824]: audit 2024-02-13T03:15:35.400635+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:36.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[20824]: cephadm 2024-02-13T03:15:35.401423+0000 mgr.y (mgr.14146) 66 : cephadm [INF] Reconfiguring daemon mgr.y on smithi161 2024-02-13T03:15:36.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb27b72700 1 -- 172.21.15.161:0/1065215261 >> v1:172.21.15.161:6790/0 conn(0x7feb200ff9b0 legacy=0x7feb20101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:36.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb27b72700 1 -- 172.21.15.161:0/1065215261 shutdown_connections 2024-02-13T03:15:36.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb27b72700 1 -- 172.21.15.161:0/1065215261 >> 172.21.15.161:0/1065215261 conn(0x7feb200f1f20 msgr2=0x7feb200f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:15:36.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb27b72700 1 -- 172.21.15.161:0/1065215261 shutdown_connections 2024-02-13T03:15:36.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb27b72700 1 -- 172.21.15.161:0/1065215261 wait complete. 2024-02-13T03:15:36.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb27b72700 1 Processor -- start 2024-02-13T03:15:36.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb27b72700 1 -- start start 2024-02-13T03:15:36.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb27b72700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7feb201082f0 con 0x7feb200617e0 2024-02-13T03:15:36.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb27b72700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7feb20113b50 con 0x7feb200ff9b0 2024-02-13T03:15:36.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb27b72700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7feb20114d30 con 0x7feb200fbc50 2024-02-13T03:15:36.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb2590e700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7feb200617e0 0x7feb20107720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:34666/0 (socket says 172.21.15.161:34666) 2024-02-13T03:15:36.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb2590e700 1 -- 172.21.15.161:0/845059475 learned_addr learned my addr 172.21.15.161:0/845059475 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:15:36.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224559754 0 0) 0x7feb201082f0 con 0x7feb200617e0 2024-02-13T03:15:36.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb08003030 con 0x7feb200617e0 2024-02-13T03:15:36.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539477450 0 0) 0x7feb20113b50 con 0x7feb200ff9b0 2024-02-13T03:15:36.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb201082f0 con 0x7feb200ff9b0 2024-02-13T03:15:36.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518917712 0 0) 0x7feb20114d30 con 0x7feb200fbc50 2024-02-13T03:15:36.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb20113b50 con 0x7feb200fbc50 2024-02-13T03:15:36.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1113407333 0 0) 0x7feb201082f0 con 0x7feb200ff9b0 2024-02-13T03:15:36.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb20114d30 con 0x7feb200ff9b0 2024-02-13T03:15:36.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7feb100031e0 con 0x7feb200ff9b0 2024-02-13T03:15:36.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4096645014 0 0) 0x7feb08003030 con 0x7feb200617e0 2024-02-13T03:15:36.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb201082f0 con 0x7feb200617e0 2024-02-13T03:15:36.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3247293576 0 0) 0x7feb20113b50 con 0x7feb200fbc50 2024-02-13T03:15:36.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feb08003030 con 0x7feb200fbc50 2024-02-13T03:15:36.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.735+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7feb18001900 con 0x7feb200617e0 2024-02-13T03:15:36.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.739+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7feb140018d0 con 0x7feb200fbc50 2024-02-13T03:15:36.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.739+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4076276801 0 0) 0x7feb20114d30 con 0x7feb200ff9b0 2024-02-13T03:15:36.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.739+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 >> v1:172.21.15.161:6790/0 conn(0x7feb200fbc50 legacy=0x7feb2010dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:36.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.739+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 >> v1:172.21.15.161:6789/0 conn(0x7feb200617e0 legacy=0x7feb20107720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:15:36.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.739+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb20115f10 con 0x7feb200ff9b0 2024-02-13T03:15:36.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.739+0000 7feb27b72700 1 -- 172.21.15.161:0/845059475 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb20114f20 con 0x7feb200ff9b0 2024-02-13T03:15:36.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.739+0000 7feb27b72700 1 -- 172.21.15.161:0/845059475 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7feb20115410 con 0x7feb200ff9b0 2024-02-13T03:15:36.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.739+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7feb10001ed0 con 0x7feb200ff9b0 2024-02-13T03:15:36.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.739+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7feb10003ad0 con 0x7feb200ff9b0 2024-02-13T03:15:36.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.739+0000 7feb27b72700 1 -- 172.21.15.161:0/845059475 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c004f20 con 0x7feb200ff9b0 2024-02-13T03:15:36.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.739+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2433666048 0 0) 0x7feb1000d3f0 con 0x7feb200ff9b0 2024-02-13T03:15:36.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.739+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 1047224232 0 0) 0x7feb1003c8d0 con 0x7feb200ff9b0 2024-02-13T03:15:36.752 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.747+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7feb1000ec30 con 0x7feb200ff9b0 2024-02-13T03:15:36.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:36.951+0000 7feb27b72700 1 -- 172.21.15.161:0/845059475 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7feb0c000b90 con 0x7feb080339e0 2024-02-13T03:15:37.077 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:36 smithi183 bash[22114]: debug 2024-02-13T03:15:36.789+0000 7f404eb6b680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T03:15:37.395 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:37 smithi183 bash[22114]: debug 2024-02-13T03:15:37.077+0000 7f404eb6b680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T03:15:37.396 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:37 smithi183 bash[22114]: debug 2024-02-13T03:15:37.233+0000 7f404eb6b680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T03:15:37.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:37 smithi161 bash[19769]: audit 2024-02-13T03:15:36.961909+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:15:37.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:37 smithi161 bash[19769]: audit 2024-02-13T03:15:36.965542+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:15:37.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:37 smithi161 bash[19769]: audit 2024-02-13T03:15:36.966524+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:37.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:37 smithi161 bash[27879]: audit 2024-02-13T03:15:36.961909+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:15:37.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:37 smithi161 bash[27879]: audit 2024-02-13T03:15:36.965542+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:15:37.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:37 smithi161 bash[27879]: audit 2024-02-13T03:15:36.966524+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:37.718 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:37 smithi183 bash[22114]: debug 2024-02-13T03:15:37.401+0000 7f404eb6b680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T03:15:37.718 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:37 smithi183 bash[22114]: debug 2024-02-13T03:15:37.609+0000 7f404eb6b680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T03:15:37.718 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:37 smithi183 bash[20824]: audit 2024-02-13T03:15:36.961909+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:15:37.718 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:37 smithi183 bash[20824]: audit 2024-02-13T03:15:36.965542+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:15:37.718 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:37 smithi183 bash[20824]: audit 2024-02-13T03:15:36.966524+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:37.987 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:15:37 smithi183 bash[22114]: debug 2024-02-13T03:15:37.713+0000 7f404eb6b680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T03:15:38.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:15:38.463+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mon.1 v1:172.21.15.183:6789/0 10 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 1965425884 0 0) 0x7feb1003bdc0 con 0x7feb200ff9b0 2024-02-13T03:15:38.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:38 smithi183 bash[20824]: cluster 2024-02-13T03:15:36.941038+0000 mgr.y (mgr.14146) 67 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:38.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:38 smithi183 bash[20824]: audit 2024-02-13T03:15:36.955316+0000 mgr.y (mgr.14146) 68 : audit [DBG] from='client.24121 v1:172.21.15.161:0/845059475' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:15:38.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:38 smithi183 bash[20824]: audit 2024-02-13T03:15:37.451522+0000 mon.a (mon.0) 214 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:38.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:38 smithi183 bash[20824]: audit 2024-02-13T03:15:37.453204+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:38.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:38 smithi183 bash[20824]: audit 2024-02-13T03:15:37.455613+0000 mon.a (mon.0) 216 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:38.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:38 smithi183 bash[20824]: audit 2024-02-13T03:15:37.456741+0000 mon.a (mon.0) 217 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:38.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:38 smithi183 bash[20824]: audit 2024-02-13T03:15:37.462870+0000 mon.a (mon.0) 218 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:38.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:38 smithi183 bash[20824]: cluster 2024-02-13T03:15:37.734808+0000 mon.a (mon.0) 219 : cluster [DBG] Standby manager daemon x started 2024-02-13T03:15:38.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:38 smithi183 bash[20824]: audit 2024-02-13T03:15:37.736780+0000 mon.a (mon.0) 220 : audit [DBG] from='mgr.? v1:172.21.15.183:0/651573785' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T03:15:38.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:38 smithi183 bash[20824]: audit 2024-02-13T03:15:37.737743+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.? v1:172.21.15.183:0/651573785' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T03:15:38.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:38 smithi183 bash[20824]: audit 2024-02-13T03:15:37.739837+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.? v1:172.21.15.183:0/651573785' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T03:15:38.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:38 smithi183 bash[20824]: audit 2024-02-13T03:15:37.740782+0000 mon.a (mon.0) 223 : audit [DBG] from='mgr.? v1:172.21.15.183:0/651573785' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T03:15:38.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[19769]: cluster 2024-02-13T03:15:36.941038+0000 mgr.y (mgr.14146) 67 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[19769]: audit 2024-02-13T03:15:36.955316+0000 mgr.y (mgr.14146) 68 : audit [DBG] from='client.24121 v1:172.21.15.161:0/845059475' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:15:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[19769]: audit 2024-02-13T03:15:37.451522+0000 mon.a (mon.0) 214 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[19769]: audit 2024-02-13T03:15:37.453204+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[19769]: audit 2024-02-13T03:15:37.455613+0000 mon.a (mon.0) 216 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[19769]: audit 2024-02-13T03:15:37.456741+0000 mon.a (mon.0) 217 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[19769]: audit 2024-02-13T03:15:37.462870+0000 mon.a (mon.0) 218 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[19769]: cluster 2024-02-13T03:15:37.734808+0000 mon.a (mon.0) 219 : cluster [DBG] Standby manager daemon x started 2024-02-13T03:15:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[19769]: audit 2024-02-13T03:15:37.736780+0000 mon.a (mon.0) 220 : audit [DBG] from='mgr.? v1:172.21.15.183:0/651573785' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T03:15:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[19769]: audit 2024-02-13T03:15:37.737743+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.? v1:172.21.15.183:0/651573785' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T03:15:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[19769]: audit 2024-02-13T03:15:37.739837+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.? v1:172.21.15.183:0/651573785' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T03:15:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[19769]: audit 2024-02-13T03:15:37.740782+0000 mon.a (mon.0) 223 : audit [DBG] from='mgr.? v1:172.21.15.183:0/651573785' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T03:15:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[27879]: cluster 2024-02-13T03:15:36.941038+0000 mgr.y (mgr.14146) 67 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[27879]: audit 2024-02-13T03:15:36.955316+0000 mgr.y (mgr.14146) 68 : audit [DBG] from='client.24121 v1:172.21.15.161:0/845059475' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:15:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[27879]: audit 2024-02-13T03:15:37.451522+0000 mon.a (mon.0) 214 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[27879]: audit 2024-02-13T03:15:37.453204+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:15:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[27879]: audit 2024-02-13T03:15:37.455613+0000 mon.a (mon.0) 216 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[27879]: audit 2024-02-13T03:15:37.456741+0000 mon.a (mon.0) 217 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:15:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[27879]: audit 2024-02-13T03:15:37.462870+0000 mon.a (mon.0) 218 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:15:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[27879]: cluster 2024-02-13T03:15:37.734808+0000 mon.a (mon.0) 219 : cluster [DBG] Standby manager daemon x started 2024-02-13T03:15:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[27879]: audit 2024-02-13T03:15:37.736780+0000 mon.a (mon.0) 220 : audit [DBG] from='mgr.? v1:172.21.15.183:0/651573785' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T03:15:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[27879]: audit 2024-02-13T03:15:37.737743+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.? v1:172.21.15.183:0/651573785' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T03:15:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[27879]: audit 2024-02-13T03:15:37.739837+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.? v1:172.21.15.183:0/651573785' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T03:15:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:38 smithi161 bash[27879]: audit 2024-02-13T03:15:37.740782+0000 mon.a (mon.0) 223 : audit [DBG] from='mgr.? v1:172.21.15.183:0/651573785' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T03:15:39.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:39 smithi183 bash[20824]: cluster 2024-02-13T03:15:38.471980+0000 mon.a (mon.0) 224 : cluster [DBG] mgrmap e14: y(active, since 95s), standbys: x 2024-02-13T03:15:39.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:39 smithi183 bash[20824]: audit 2024-02-13T03:15:38.472207+0000 mon.a (mon.0) 225 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T03:15:39.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:39 smithi161 bash[19769]: cluster 2024-02-13T03:15:38.471980+0000 mon.a (mon.0) 224 : cluster [DBG] mgrmap e14: y(active, since 95s), standbys: x 2024-02-13T03:15:39.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:39 smithi161 bash[19769]: audit 2024-02-13T03:15:38.472207+0000 mon.a (mon.0) 225 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T03:15:39.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:39 smithi161 bash[27879]: cluster 2024-02-13T03:15:38.471980+0000 mon.a (mon.0) 224 : cluster [DBG] mgrmap e14: y(active, since 95s), standbys: x 2024-02-13T03:15:39.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:39 smithi161 bash[27879]: audit 2024-02-13T03:15:38.472207+0000 mon.a (mon.0) 225 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T03:15:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:40 smithi183 bash[20824]: cluster 2024-02-13T03:15:38.941432+0000 mgr.y (mgr.14146) 69 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:40.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:40 smithi161 bash[19769]: cluster 2024-02-13T03:15:38.941432+0000 mgr.y (mgr.14146) 69 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:40.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:40 smithi161 bash[27879]: cluster 2024-02-13T03:15:38.941432+0000 mgr.y (mgr.14146) 69 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:42.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:42 smithi183 bash[20824]: cluster 2024-02-13T03:15:40.941844+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:42.875 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:42 smithi161 bash[19769]: cluster 2024-02-13T03:15:40.941844+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:42.875 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:42 smithi161 bash[27879]: cluster 2024-02-13T03:15:40.941844+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:44.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:44 smithi161 bash[19769]: cluster 2024-02-13T03:15:42.942151+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:44.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:44 smithi161 bash[27879]: cluster 2024-02-13T03:15:42.942151+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:44.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:44 smithi183 bash[20824]: cluster 2024-02-13T03:15:42.942151+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:45.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:45 smithi161 bash[19769]: audit 2024-02-13T03:15:44.997870+0000 mon.a (mon.0) 226 : audit [INF] from='client.? v1:172.21.15.161:0/2881129297' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf603e39-02f0-4508-b426-6fdd8402f1b6"}]: dispatch 2024-02-13T03:15:45.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:45 smithi161 bash[19769]: audit 2024-02-13T03:15:45.004597+0000 mon.a (mon.0) 227 : audit [INF] from='client.? v1:172.21.15.161:0/2881129297' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf603e39-02f0-4508-b426-6fdd8402f1b6"}]': finished 2024-02-13T03:15:45.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:45 smithi161 bash[19769]: cluster 2024-02-13T03:15:45.004776+0000 mon.a (mon.0) 228 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-13T03:15:45.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:45 smithi161 bash[19769]: audit 2024-02-13T03:15:45.004932+0000 mon.a (mon.0) 229 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:15:45.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:45 smithi161 bash[27879]: audit 2024-02-13T03:15:44.997870+0000 mon.a (mon.0) 226 : audit [INF] from='client.? v1:172.21.15.161:0/2881129297' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf603e39-02f0-4508-b426-6fdd8402f1b6"}]: dispatch 2024-02-13T03:15:45.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:45 smithi161 bash[27879]: audit 2024-02-13T03:15:45.004597+0000 mon.a (mon.0) 227 : audit [INF] from='client.? v1:172.21.15.161:0/2881129297' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf603e39-02f0-4508-b426-6fdd8402f1b6"}]': finished 2024-02-13T03:15:45.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:45 smithi161 bash[27879]: cluster 2024-02-13T03:15:45.004776+0000 mon.a (mon.0) 228 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-13T03:15:45.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:45 smithi161 bash[27879]: audit 2024-02-13T03:15:45.004932+0000 mon.a (mon.0) 229 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:15:45.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:45 smithi183 bash[20824]: audit 2024-02-13T03:15:44.997870+0000 mon.a (mon.0) 226 : audit [INF] from='client.? v1:172.21.15.161:0/2881129297' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf603e39-02f0-4508-b426-6fdd8402f1b6"}]: dispatch 2024-02-13T03:15:45.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:45 smithi183 bash[20824]: audit 2024-02-13T03:15:45.004597+0000 mon.a (mon.0) 227 : audit [INF] from='client.? v1:172.21.15.161:0/2881129297' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf603e39-02f0-4508-b426-6fdd8402f1b6"}]': finished 2024-02-13T03:15:45.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:45 smithi183 bash[20824]: cluster 2024-02-13T03:15:45.004776+0000 mon.a (mon.0) 228 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-13T03:15:45.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:45 smithi183 bash[20824]: audit 2024-02-13T03:15:45.004932+0000 mon.a (mon.0) 229 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:15:46.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:46 smithi161 bash[19769]: cluster 2024-02-13T03:15:44.942430+0000 mgr.y (mgr.14146) 72 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:46.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:46 smithi161 bash[19769]: audit 2024-02-13T03:15:45.606362+0000 mon.a (mon.0) 230 : audit [DBG] from='client.? v1:172.21.15.161:0/4286122875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:15:46.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:46 smithi161 bash[27879]: cluster 2024-02-13T03:15:44.942430+0000 mgr.y (mgr.14146) 72 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:46.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:46 smithi161 bash[27879]: audit 2024-02-13T03:15:45.606362+0000 mon.a (mon.0) 230 : audit [DBG] from='client.? v1:172.21.15.161:0/4286122875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:15:46.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:46 smithi183 bash[20824]: cluster 2024-02-13T03:15:44.942430+0000 mgr.y (mgr.14146) 72 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:46.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:46 smithi183 bash[20824]: audit 2024-02-13T03:15:45.606362+0000 mon.a (mon.0) 230 : audit [DBG] from='client.? v1:172.21.15.161:0/4286122875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:15:48.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:48 smithi161 bash[19769]: cluster 2024-02-13T03:15:46.942880+0000 mgr.y (mgr.14146) 73 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:48.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:48 smithi161 bash[27879]: cluster 2024-02-13T03:15:46.942880+0000 mgr.y (mgr.14146) 73 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:48.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:48 smithi183 bash[20824]: cluster 2024-02-13T03:15:46.942880+0000 mgr.y (mgr.14146) 73 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:50.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:50 smithi161 bash[19769]: cluster 2024-02-13T03:15:48.943203+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:50.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:50 smithi161 bash[27879]: cluster 2024-02-13T03:15:48.943203+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:50.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:50 smithi183 bash[20824]: cluster 2024-02-13T03:15:48.943203+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:52.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:52 smithi161 bash[19769]: cluster 2024-02-13T03:15:50.943580+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:52.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:52 smithi161 bash[27879]: cluster 2024-02-13T03:15:50.943580+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:52 smithi183 bash[20824]: cluster 2024-02-13T03:15:50.943580+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:54.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:54 smithi161 bash[19769]: cluster 2024-02-13T03:15:52.944017+0000 mgr.y (mgr.14146) 76 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:54.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:54 smithi161 bash[27879]: cluster 2024-02-13T03:15:52.944017+0000 mgr.y (mgr.14146) 76 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:54 smithi183 bash[20824]: cluster 2024-02-13T03:15:52.944017+0000 mgr.y (mgr.14146) 76 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:56.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:56 smithi161 bash[19769]: cluster 2024-02-13T03:15:54.944510+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:56.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:56 smithi161 bash[27879]: cluster 2024-02-13T03:15:54.944510+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:56 smithi183 bash[20824]: cluster 2024-02-13T03:15:54.944510+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:57.786 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:57 smithi161 bash[27879]: audit 2024-02-13T03:15:56.635403+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-13T03:15:57.786 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:57 smithi161 bash[27879]: audit 2024-02-13T03:15:56.636861+0000 mon.a (mon.0) 232 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:57.786 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:57 smithi161 bash[19769]: audit 2024-02-13T03:15:56.635403+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-13T03:15:57.787 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:57 smithi161 bash[19769]: audit 2024-02-13T03:15:56.636861+0000 mon.a (mon.0) 232 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:57 smithi183 bash[20824]: audit 2024-02-13T03:15:56.635403+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-13T03:15:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:57 smithi183 bash[20824]: audit 2024-02-13T03:15:56.636861+0000 mon.a (mon.0) 232 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:15:58.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:58 smithi161 bash[19769]: cephadm 2024-02-13T03:15:56.638051+0000 mgr.y (mgr.14146) 78 : cephadm [INF] Deploying daemon osd.0 on smithi161 2024-02-13T03:15:58.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:15:58 smithi161 bash[19769]: cluster 2024-02-13T03:15:56.944964+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:58.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:58 smithi161 bash[27879]: cephadm 2024-02-13T03:15:56.638051+0000 mgr.y (mgr.14146) 78 : cephadm [INF] Deploying daemon osd.0 on smithi161 2024-02-13T03:15:58.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:15:58 smithi161 bash[27879]: cluster 2024-02-13T03:15:56.944964+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:15:58.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:58 smithi183 bash[20824]: cephadm 2024-02-13T03:15:56.638051+0000 mgr.y (mgr.14146) 78 : cephadm [INF] Deploying daemon osd.0 on smithi161 2024-02-13T03:15:58.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:15:58 smithi183 bash[20824]: cluster 2024-02-13T03:15:56.944964+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:00.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:00 smithi161 bash[19769]: cluster 2024-02-13T03:15:58.945380+0000 mgr.y (mgr.14146) 80 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:00.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:00 smithi161 bash[19769]: audit 2024-02-13T03:16:00.419295+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:00.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:00 smithi161 bash[19769]: audit 2024-02-13T03:16:00.437642+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:16:00.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:00 smithi161 bash[27879]: cluster 2024-02-13T03:15:58.945380+0000 mgr.y (mgr.14146) 80 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:00.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:00 smithi161 bash[27879]: audit 2024-02-13T03:16:00.419295+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:00.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:00 smithi161 bash[27879]: audit 2024-02-13T03:16:00.437642+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:16:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:00 smithi183 bash[20824]: cluster 2024-02-13T03:15:58.945380+0000 mgr.y (mgr.14146) 80 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:00 smithi183 bash[20824]: audit 2024-02-13T03:16:00.419295+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:00 smithi183 bash[20824]: audit 2024-02-13T03:16:00.437642+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:16:02.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:02 smithi161 bash[19769]: cluster 2024-02-13T03:16:00.945847+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:02.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:02 smithi161 bash[27879]: cluster 2024-02-13T03:16:00.945847+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:02.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:02 smithi183 bash[20824]: cluster 2024-02-13T03:16:00.945847+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:04.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:04 smithi161 bash[19769]: cluster 2024-02-13T03:16:02.946268+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:04.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:04 smithi161 bash[27879]: cluster 2024-02-13T03:16:02.946268+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:04.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:04 smithi183 bash[20824]: cluster 2024-02-13T03:16:02.946268+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:06.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:06 smithi161 bash[19769]: cluster 2024-02-13T03:16:04.946813+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:06 smithi161 bash[19769]: audit 2024-02-13T03:16:05.774546+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:06 smithi161 bash[19769]: audit 2024-02-13T03:16:05.775985+0000 mon.a (mon.0) 236 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:06 smithi161 bash[19769]: audit 2024-02-13T03:16:05.777454+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:16:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:06 smithi161 bash[19769]: audit 2024-02-13T03:16:05.786792+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:06 smithi161 bash[27879]: cluster 2024-02-13T03:16:04.946813+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:06 smithi161 bash[27879]: audit 2024-02-13T03:16:05.774546+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:06 smithi161 bash[27879]: audit 2024-02-13T03:16:05.775985+0000 mon.a (mon.0) 236 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:06 smithi161 bash[27879]: audit 2024-02-13T03:16:05.777454+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:16:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:06 smithi161 bash[27879]: audit 2024-02-13T03:16:05.786792+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:06.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:06 smithi183 bash[20824]: cluster 2024-02-13T03:16:04.946813+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:06.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:06 smithi183 bash[20824]: audit 2024-02-13T03:16:05.774546+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:06.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:06 smithi183 bash[20824]: audit 2024-02-13T03:16:05.775985+0000 mon.a (mon.0) 236 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:06.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:06 smithi183 bash[20824]: audit 2024-02-13T03:16:05.777454+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:16:06.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:06 smithi183 bash[20824]: audit 2024-02-13T03:16:05.786792+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:07.807 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:07 smithi161 bash[19769]: audit 2024-02-13T03:16:06.909463+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 v1:172.21.15.161:6801/2012328581' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-13T03:16:07.807 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:07 smithi161 bash[19769]: audit 2024-02-13T03:16:06.909795+0000 mon.a (mon.0) 239 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-13T03:16:07.808 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:07 smithi161 bash[27879]: audit 2024-02-13T03:16:06.909463+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 v1:172.21.15.161:6801/2012328581' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-13T03:16:07.808 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:07 smithi161 bash[27879]: audit 2024-02-13T03:16:06.909795+0000 mon.a (mon.0) 239 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-13T03:16:07.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:07 smithi183 bash[20824]: audit 2024-02-13T03:16:06.909463+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 v1:172.21.15.161:6801/2012328581' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-13T03:16:07.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:07 smithi183 bash[20824]: audit 2024-02-13T03:16:06.909795+0000 mon.a (mon.0) 239 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-13T03:16:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:08.814+0000 7feb1effd700 1 -- 172.21.15.161:0/845059475 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1166280913) 0x7feb0c000b90 con 0x7feb080339e0 2024-02-13T03:16:08.820 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 0 on host 'smithi161' 2024-02-13T03:16:08.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:08.818+0000 7feb27b72700 1 -- 172.21.15.161:0/845059475 >> v1:172.21.15.161:6800/1187535987 conn(0x7feb080339e0 legacy=0x7feb08035ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:16:08.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:08.818+0000 7feb27b72700 1 -- 172.21.15.161:0/845059475 >> v1:172.21.15.183:6789/0 conn(0x7feb200ff9b0 legacy=0x7feb20111420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:16:08.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:08.818+0000 7feb27b72700 1 -- 172.21.15.161:0/845059475 shutdown_connections 2024-02-13T03:16:08.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:08.818+0000 7feb27b72700 1 -- 172.21.15.161:0/845059475 >> 172.21.15.161:0/845059475 conn(0x7feb200f1f20 msgr2=0x7feb200f4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:16:08.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:08.818+0000 7feb27b72700 1 -- 172.21.15.161:0/845059475 shutdown_connections 2024-02-13T03:16:08.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:08.818+0000 7feb27b72700 1 -- 172.21.15.161:0/845059475 wait complete. 2024-02-13T03:16:08.885 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:08 smithi161 bash[19769]: cluster 2024-02-13T03:16:06.947349+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:08.885 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:08 smithi161 bash[19769]: audit 2024-02-13T03:16:07.567409+0000 mon.a (mon.0) 240 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-13T03:16:08.885 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:08 smithi161 bash[19769]: cluster 2024-02-13T03:16:07.567585+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-13T03:16:08.886 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:08 smithi161 bash[19769]: audit 2024-02-13T03:16:07.567774+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:16:08.886 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:08 smithi161 bash[19769]: audit 2024-02-13T03:16:07.568463+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 v1:172.21.15.161:6801/2012328581' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:16:08.886 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:08 smithi161 bash[19769]: audit 2024-02-13T03:16:07.568963+0000 mon.a (mon.0) 243 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:16:08.886 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:08 smithi161 bash[27879]: cluster 2024-02-13T03:16:06.947349+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:08.886 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:08 smithi161 bash[27879]: audit 2024-02-13T03:16:07.567409+0000 mon.a (mon.0) 240 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-13T03:16:08.886 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:08 smithi161 bash[27879]: cluster 2024-02-13T03:16:07.567585+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-13T03:16:08.886 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:08 smithi161 bash[27879]: audit 2024-02-13T03:16:07.567774+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:16:08.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:08 smithi161 bash[27879]: audit 2024-02-13T03:16:07.568463+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 v1:172.21.15.161:6801/2012328581' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:16:08.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:08 smithi161 bash[27879]: audit 2024-02-13T03:16:07.568963+0000 mon.a (mon.0) 243 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:16:08.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:08 smithi183 bash[20824]: cluster 2024-02-13T03:16:06.947349+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:08.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:08 smithi183 bash[20824]: audit 2024-02-13T03:16:07.567409+0000 mon.a (mon.0) 240 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-13T03:16:08.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:08 smithi183 bash[20824]: cluster 2024-02-13T03:16:07.567585+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-13T03:16:08.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:08 smithi183 bash[20824]: audit 2024-02-13T03:16:07.567774+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:16:08.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:08 smithi183 bash[20824]: audit 2024-02-13T03:16:07.568463+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 v1:172.21.15.161:6801/2012328581' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:16:08.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:08 smithi183 bash[20824]: audit 2024-02-13T03:16:07.568963+0000 mon.a (mon.0) 243 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:16:09.461 DEBUG:teuthology.orchestra.run.smithi161:osd.0> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.0.service 2024-02-13T03:16:09.463 INFO:tasks.cephadm:Deploying osd.1 on smithi161 with /dev/vg_nvme/lv_3... 2024-02-13T03:16:09.464 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-13T03:16:09.634 INFO:journalctl@ceph.osd.0.smithi161.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:16:09.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:09 smithi161 bash[19769]: audit 2024-02-13T03:16:08.569267+0000 mon.a (mon.0) 244 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-02-13T03:16:09.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:09 smithi161 bash[19769]: cluster 2024-02-13T03:16:08.569565+0000 mon.a (mon.0) 245 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-13T03:16:09.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:09 smithi161 bash[19769]: audit 2024-02-13T03:16:08.570341+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:16:09.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:09 smithi161 bash[19769]: audit 2024-02-13T03:16:08.578490+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:16:09.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:09 smithi161 bash[19769]: audit 2024-02-13T03:16:08.810833+0000 mon.a (mon.0) 248 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:16:09.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:09 smithi161 bash[19769]: audit 2024-02-13T03:16:08.815520+0000 mon.a (mon.0) 249 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:09.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:09 smithi161 bash[27879]: audit 2024-02-13T03:16:08.569267+0000 mon.a (mon.0) 244 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-02-13T03:16:09.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:09 smithi161 bash[27879]: cluster 2024-02-13T03:16:08.569565+0000 mon.a (mon.0) 245 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-13T03:16:09.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:09 smithi161 bash[27879]: audit 2024-02-13T03:16:08.570341+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:16:09.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:09 smithi161 bash[27879]: audit 2024-02-13T03:16:08.578490+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:16:09.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:09 smithi161 bash[27879]: audit 2024-02-13T03:16:08.810833+0000 mon.a (mon.0) 248 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:16:09.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:09 smithi161 bash[27879]: audit 2024-02-13T03:16:08.815520+0000 mon.a (mon.0) 249 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:09.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:09 smithi183 bash[20824]: audit 2024-02-13T03:16:08.569267+0000 mon.a (mon.0) 244 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-02-13T03:16:09.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:09 smithi183 bash[20824]: cluster 2024-02-13T03:16:08.569565+0000 mon.a (mon.0) 245 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-13T03:16:09.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:09 smithi183 bash[20824]: audit 2024-02-13T03:16:08.570341+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:16:09.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:09 smithi183 bash[20824]: audit 2024-02-13T03:16:08.578490+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:16:09.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:09 smithi183 bash[20824]: audit 2024-02-13T03:16:08.810833+0000 mon.a (mon.0) 248 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:16:09.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:09 smithi183 bash[20824]: audit 2024-02-13T03:16:08.815520+0000 mon.a (mon.0) 249 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:10.753 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:10 smithi161 bash[19769]: cluster 2024-02-13T03:16:07.873867+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:16:10.753 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:10 smithi161 bash[19769]: cluster 2024-02-13T03:16:07.873966+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:16:10.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:10 smithi161 bash[19769]: cluster 2024-02-13T03:16:08.947727+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:10.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:10 smithi161 bash[19769]: cluster 2024-02-13T03:16:09.579044+0000 mon.a (mon.0) 250 : cluster [INF] osd.0 v1:172.21.15.161:6801/2012328581 boot 2024-02-13T03:16:10.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:10 smithi161 bash[19769]: cluster 2024-02-13T03:16:09.579141+0000 mon.a (mon.0) 251 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-13T03:16:10.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:10 smithi161 bash[19769]: audit 2024-02-13T03:16:09.579905+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:16:10.754 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:10 smithi161 bash[27879]: cluster 2024-02-13T03:16:07.873867+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:16:10.754 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:10 smithi161 bash[27879]: cluster 2024-02-13T03:16:07.873966+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:16:10.754 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:10 smithi161 bash[27879]: cluster 2024-02-13T03:16:08.947727+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:10.754 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:10 smithi161 bash[27879]: cluster 2024-02-13T03:16:09.579044+0000 mon.a (mon.0) 250 : cluster [INF] osd.0 v1:172.21.15.161:6801/2012328581 boot 2024-02-13T03:16:10.754 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:10 smithi161 bash[27879]: cluster 2024-02-13T03:16:09.579141+0000 mon.a (mon.0) 251 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-13T03:16:10.754 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:10 smithi161 bash[27879]: audit 2024-02-13T03:16:09.579905+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:16:10.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:10 smithi183 bash[20824]: cluster 2024-02-13T03:16:07.873867+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:16:10.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:10 smithi183 bash[20824]: cluster 2024-02-13T03:16:07.873966+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:16:10.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:10 smithi183 bash[20824]: cluster 2024-02-13T03:16:08.947727+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T03:16:10.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:10 smithi183 bash[20824]: cluster 2024-02-13T03:16:09.579044+0000 mon.a (mon.0) 250 : cluster [INF] osd.0 v1:172.21.15.161:6801/2012328581 boot 2024-02-13T03:16:10.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:10 smithi183 bash[20824]: cluster 2024-02-13T03:16:09.579141+0000 mon.a (mon.0) 251 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-13T03:16:10.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:10 smithi183 bash[20824]: audit 2024-02-13T03:16:09.579905+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:16:12.074 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:11 smithi161 bash[19769]: cluster 2024-02-13T03:16:10.819853+0000 mon.a (mon.0) 253 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-13T03:16:12.074 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:11 smithi161 bash[19769]: cluster 2024-02-13T03:16:10.948272+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:12.074 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:11 smithi161 bash[27879]: cluster 2024-02-13T03:16:10.819853+0000 mon.a (mon.0) 253 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-13T03:16:12.075 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:11 smithi161 bash[27879]: cluster 2024-02-13T03:16:10.948272+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:12.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:11 smithi183 bash[20824]: cluster 2024-02-13T03:16:10.819853+0000 mon.a (mon.0) 253 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-13T03:16:12.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:11 smithi183 bash[20824]: cluster 2024-02-13T03:16:10.948272+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:12.579 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-13T03:16:12.579 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-13T03:16:12.579 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-02-13T03:16:12.579 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-02-13T03:16:12.580 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0158628 s, 661 MB/s 2024-02-13T03:16:12.580 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-02-13T03:16:13.199 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-02-13T03:16:14.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:14 smithi161 bash[27879]: cluster 2024-02-13T03:16:12.948822+0000 mgr.y (mgr.14146) 87 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:14.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:14 smithi161 bash[19769]: cluster 2024-02-13T03:16:12.948822+0000 mgr.y (mgr.14146) 87 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:14 smithi183 bash[20824]: cluster 2024-02-13T03:16:12.948822+0000 mgr.y (mgr.14146) 87 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:16.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:16 smithi161 bash[19769]: cluster 2024-02-13T03:16:14.949364+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:16.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:16 smithi161 bash[27879]: cluster 2024-02-13T03:16:14.949364+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:16 smithi183 bash[20824]: cluster 2024-02-13T03:16:14.949364+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:17.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd720b85700 1 -- 172.21.15.161:0/4101448960 >> v1:172.21.15.161:6790/0 conn(0x7fd71c0f67c0 legacy=0x7fd71c0f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:16:17.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd720b85700 1 -- 172.21.15.161:0/4101448960 shutdown_connections 2024-02-13T03:16:17.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd720b85700 1 -- 172.21.15.161:0/4101448960 >> 172.21.15.161:0/4101448960 conn(0x7fd71c0f1f00 msgr2=0x7fd71c0f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:16:17.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd720b85700 1 -- 172.21.15.161:0/4101448960 shutdown_connections 2024-02-13T03:16:17.076 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd720b85700 1 -- 172.21.15.161:0/4101448960 wait complete. 2024-02-13T03:16:17.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd720b85700 1 Processor -- start 2024-02-13T03:16:17.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd720b85700 1 -- start start 2024-02-13T03:16:17.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd720b85700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd71c108380 con 0x7fd71c0fac90 2024-02-13T03:16:17.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd720b85700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd71c113be0 con 0x7fd71c0fe8c0 2024-02-13T03:16:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd720b85700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd71c114dc0 con 0x7fd71c0f67c0 2024-02-13T03:16:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd719d9b700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd71c0fac90 0x7fd71c10dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:45638/0 (socket says 172.21.15.161:45638) 2024-02-13T03:16:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd719d9b700 1 -- 172.21.15.161:0/3335480451 learned_addr learned my addr 172.21.15.161:0/3335480451 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:16:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd71a59c700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7fd71c0f67c0 0x7fd71c1077b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:46772/0 (socket says 172.21.15.161:46772) 2024-02-13T03:16:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3968561716 0 0) 0x7fd71c114dc0 con 0x7fd71c0f67c0 2024-02-13T03:16:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7040033f0 con 0x7fd71c0f67c0 2024-02-13T03:16:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069362450 0 0) 0x7fd71c108380 con 0x7fd71c0fac90 2024-02-13T03:16:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd71c114dc0 con 0x7fd71c0fac90 2024-02-13T03:16:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850742646 0 0) 0x7fd71c113be0 con 0x7fd71c0fe8c0 2024-02-13T03:16:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd71c108380 con 0x7fd71c0fe8c0 2024-02-13T03:16:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 195237876 0 0) 0x7fd7040033f0 con 0x7fd71c0f67c0 2024-02-13T03:16:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd71c113be0 con 0x7fd71c0f67c0 2024-02-13T03:16:17.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.070+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fd70c0015d0 con 0x7fd71c0f67c0 2024-02-13T03:16:17.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 445054667 0 0) 0x7fd71c114dc0 con 0x7fd71c0fac90 2024-02-13T03:16:17.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd7040033f0 con 0x7fd71c0fac90 2024-02-13T03:16:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fd710001a70 con 0x7fd71c0fac90 2024-02-13T03:16:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3940734562 0 0) 0x7fd71c108380 con 0x7fd71c0fe8c0 2024-02-13T03:16:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd71c114dc0 con 0x7fd71c0fe8c0 2024-02-13T03:16:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fd714001b00 con 0x7fd71c0fe8c0 2024-02-13T03:16:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3870373360 0 0) 0x7fd71c113be0 con 0x7fd71c0f67c0 2024-02-13T03:16:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 >> v1:172.21.15.183:6789/0 conn(0x7fd71c0fe8c0 legacy=0x7fd71c1114b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:16:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 >> v1:172.21.15.161:6789/0 conn(0x7fd71c0fac90 legacy=0x7fd71c10dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:16:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c115fa0 con 0x7fd71c0f67c0 2024-02-13T03:16:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd720b85700 1 -- 172.21.15.161:0/3335480451 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71c112c30 con 0x7fd71c0f67c0 2024-02-13T03:16:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd720b85700 1 -- 172.21.15.161:0/3335480451 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c113120 con 0x7fd71c0f67c0 2024-02-13T03:16:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fd70c0011d0 con 0x7fd71c0f67c0 2024-02-13T03:16:17.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fd70c003800 con 0x7fd71c0f67c0 2024-02-13T03:16:17.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd720b85700 1 -- 172.21.15.161:0/3335480451 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc004f20 con 0x7fd71c0f67c0 2024-02-13T03:16:17.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 1965425884 0 0) 0x7fd70c0039e0 con 0x7fd71c0f67c0 2024-02-13T03:16:17.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.074+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(9..9 src has 1..9) v4 ==== 1484+0+0 (unknown 2100674288 0 0) 0x7fd70c077660 con 0x7fd71c0f67c0 2024-02-13T03:16:17.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.082+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fd70c049800 con 0x7fd71c0f67c0 2024-02-13T03:16:17.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:17.270+0000 7fd720b85700 1 -- 172.21.15.161:0/3335480451 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6fc000b90 con 0x7fd704062270 2024-02-13T03:16:18.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:18 smithi161 bash[19769]: cluster 2024-02-13T03:16:16.949796+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:18.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:18 smithi161 bash[19769]: audit 2024-02-13T03:16:17.276331+0000 mgr.y (mgr.14146) 90 : audit [DBG] from='client.24149 v1:172.21.15.161:0/3335480451' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:16:18.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:18 smithi161 bash[19769]: audit 2024-02-13T03:16:17.283064+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:16:18.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:18 smithi161 bash[19769]: audit 2024-02-13T03:16:17.286420+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:16:18.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:18 smithi161 bash[19769]: audit 2024-02-13T03:16:17.287338+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:18 smithi161 bash[27879]: cluster 2024-02-13T03:16:16.949796+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:18 smithi161 bash[27879]: audit 2024-02-13T03:16:17.276331+0000 mgr.y (mgr.14146) 90 : audit [DBG] from='client.24149 v1:172.21.15.161:0/3335480451' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:16:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:18 smithi161 bash[27879]: audit 2024-02-13T03:16:17.283064+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:16:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:18 smithi161 bash[27879]: audit 2024-02-13T03:16:17.286420+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:16:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:18 smithi161 bash[27879]: audit 2024-02-13T03:16:17.287338+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:18.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:18 smithi183 bash[20824]: cluster 2024-02-13T03:16:16.949796+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:18.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:18 smithi183 bash[20824]: audit 2024-02-13T03:16:17.276331+0000 mgr.y (mgr.14146) 90 : audit [DBG] from='client.24149 v1:172.21.15.161:0/3335480451' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:16:18.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:18 smithi183 bash[20824]: audit 2024-02-13T03:16:17.283064+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:16:18.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:18 smithi183 bash[20824]: audit 2024-02-13T03:16:17.286420+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:16:18.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:18 smithi183 bash[20824]: audit 2024-02-13T03:16:17.287338+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:19.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[19769]: cephadm 2024-02-13T03:16:18.568020+0000 mgr.y (mgr.14146) 91 : cephadm [INF] Detected new or changed devices on smithi161 2024-02-13T03:16:19.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[19769]: audit 2024-02-13T03:16:18.578264+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:19.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[19769]: audit 2024-02-13T03:16:18.579618+0000 mon.a (mon.0) 258 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:16:19.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[19769]: audit 2024-02-13T03:16:18.588696+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:19.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[19769]: audit 2024-02-13T03:16:18.590365+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:19.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[19769]: audit 2024-02-13T03:16:18.591446+0000 mon.a (mon.0) 261 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:16:19.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[19769]: audit 2024-02-13T03:16:18.600118+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:19.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[27879]: cephadm 2024-02-13T03:16:18.568020+0000 mgr.y (mgr.14146) 91 : cephadm [INF] Detected new or changed devices on smithi161 2024-02-13T03:16:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[27879]: audit 2024-02-13T03:16:18.578264+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[27879]: audit 2024-02-13T03:16:18.579618+0000 mon.a (mon.0) 258 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:16:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[27879]: audit 2024-02-13T03:16:18.588696+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[27879]: audit 2024-02-13T03:16:18.590365+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[27879]: audit 2024-02-13T03:16:18.591446+0000 mon.a (mon.0) 261 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:16:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:19 smithi161 bash[27879]: audit 2024-02-13T03:16:18.600118+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:19.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:19 smithi183 bash[20824]: cephadm 2024-02-13T03:16:18.568020+0000 mgr.y (mgr.14146) 91 : cephadm [INF] Detected new or changed devices on smithi161 2024-02-13T03:16:19.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:19 smithi183 bash[20824]: audit 2024-02-13T03:16:18.578264+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:19.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:19 smithi183 bash[20824]: audit 2024-02-13T03:16:18.579618+0000 mon.a (mon.0) 258 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:16:19.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:19 smithi183 bash[20824]: audit 2024-02-13T03:16:18.588696+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:19.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:19 smithi183 bash[20824]: audit 2024-02-13T03:16:18.590365+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:19.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:19 smithi183 bash[20824]: audit 2024-02-13T03:16:18.591446+0000 mon.a (mon.0) 261 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:16:19.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:19 smithi183 bash[20824]: audit 2024-02-13T03:16:18.600118+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:20.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:20 smithi161 bash[19769]: cephadm 2024-02-13T03:16:18.580389+0000 mgr.y (mgr.14146) 92 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 16294M 2024-02-13T03:16:20.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:20 smithi161 bash[19769]: cluster 2024-02-13T03:16:18.950324+0000 mgr.y (mgr.14146) 93 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:20 smithi161 bash[27879]: cephadm 2024-02-13T03:16:18.580389+0000 mgr.y (mgr.14146) 92 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 16294M 2024-02-13T03:16:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:20 smithi161 bash[27879]: cluster 2024-02-13T03:16:18.950324+0000 mgr.y (mgr.14146) 93 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:20 smithi183 bash[20824]: cephadm 2024-02-13T03:16:18.580389+0000 mgr.y (mgr.14146) 92 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 16294M 2024-02-13T03:16:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:20 smithi183 bash[20824]: cluster 2024-02-13T03:16:18.950324+0000 mgr.y (mgr.14146) 93 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:21.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:21 smithi161 bash[19769]: cluster 2024-02-13T03:16:20.950925+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:21.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:21 smithi161 bash[27879]: cluster 2024-02-13T03:16:20.950925+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:21 smithi183 bash[20824]: cluster 2024-02-13T03:16:20.950925+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:24.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:24 smithi161 bash[27879]: cluster 2024-02-13T03:16:22.951484+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:24.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:24 smithi161 bash[19769]: cluster 2024-02-13T03:16:22.951484+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:24 smithi183 bash[20824]: cluster 2024-02-13T03:16:22.951484+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[19769]: cluster 2024-02-13T03:16:24.951835+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[19769]: audit 2024-02-13T03:16:25.268018+0000 mon.c (mon.2) 4 : audit [INF] from='client.? v1:172.21.15.161:0/961728109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bda8b4b-0bd5-47f1-a699-93a180d0a278"}]: dispatch 2024-02-13T03:16:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[19769]: audit 2024-02-13T03:16:25.268695+0000 mon.a (mon.0) 263 : audit [INF] from='client.24155 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bda8b4b-0bd5-47f1-a699-93a180d0a278"}]: dispatch 2024-02-13T03:16:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[19769]: audit 2024-02-13T03:16:25.277162+0000 mon.a (mon.0) 264 : audit [INF] from='client.24155 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3bda8b4b-0bd5-47f1-a699-93a180d0a278"}]': finished 2024-02-13T03:16:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[19769]: cluster 2024-02-13T03:16:25.277280+0000 mon.a (mon.0) 265 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-13T03:16:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[19769]: audit 2024-02-13T03:16:25.277470+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[19769]: audit 2024-02-13T03:16:25.850085+0000 mon.a (mon.0) 267 : audit [DBG] from='client.? v1:172.21.15.161:0/39195660' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:16:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[27879]: cluster 2024-02-13T03:16:24.951835+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[27879]: audit 2024-02-13T03:16:25.268018+0000 mon.c (mon.2) 4 : audit [INF] from='client.? v1:172.21.15.161:0/961728109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bda8b4b-0bd5-47f1-a699-93a180d0a278"}]: dispatch 2024-02-13T03:16:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[27879]: audit 2024-02-13T03:16:25.268695+0000 mon.a (mon.0) 263 : audit [INF] from='client.24155 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bda8b4b-0bd5-47f1-a699-93a180d0a278"}]: dispatch 2024-02-13T03:16:26.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[27879]: audit 2024-02-13T03:16:25.277162+0000 mon.a (mon.0) 264 : audit [INF] from='client.24155 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3bda8b4b-0bd5-47f1-a699-93a180d0a278"}]': finished 2024-02-13T03:16:26.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[27879]: cluster 2024-02-13T03:16:25.277280+0000 mon.a (mon.0) 265 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-13T03:16:26.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[27879]: audit 2024-02-13T03:16:25.277470+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:26.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:26 smithi161 bash[27879]: audit 2024-02-13T03:16:25.850085+0000 mon.a (mon.0) 267 : audit [DBG] from='client.? v1:172.21.15.161:0/39195660' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:16:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:26 smithi183 bash[20824]: cluster 2024-02-13T03:16:24.951835+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:26 smithi183 bash[20824]: audit 2024-02-13T03:16:25.268018+0000 mon.c (mon.2) 4 : audit [INF] from='client.? v1:172.21.15.161:0/961728109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bda8b4b-0bd5-47f1-a699-93a180d0a278"}]: dispatch 2024-02-13T03:16:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:26 smithi183 bash[20824]: audit 2024-02-13T03:16:25.268695+0000 mon.a (mon.0) 263 : audit [INF] from='client.24155 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3bda8b4b-0bd5-47f1-a699-93a180d0a278"}]: dispatch 2024-02-13T03:16:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:26 smithi183 bash[20824]: audit 2024-02-13T03:16:25.277162+0000 mon.a (mon.0) 264 : audit [INF] from='client.24155 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3bda8b4b-0bd5-47f1-a699-93a180d0a278"}]': finished 2024-02-13T03:16:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:26 smithi183 bash[20824]: cluster 2024-02-13T03:16:25.277280+0000 mon.a (mon.0) 265 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-13T03:16:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:26 smithi183 bash[20824]: audit 2024-02-13T03:16:25.277470+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:26.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:26 smithi183 bash[20824]: audit 2024-02-13T03:16:25.850085+0000 mon.a (mon.0) 267 : audit [DBG] from='client.? v1:172.21.15.161:0/39195660' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:16:28.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:28 smithi161 bash[27879]: cluster 2024-02-13T03:16:26.952262+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:28.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:28 smithi161 bash[19769]: cluster 2024-02-13T03:16:26.952262+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:28 smithi183 bash[20824]: cluster 2024-02-13T03:16:26.952262+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:30.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:30 smithi161 bash[19769]: cluster 2024-02-13T03:16:28.952751+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:30.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:30 smithi161 bash[27879]: cluster 2024-02-13T03:16:28.952751+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:30 smithi183 bash[20824]: cluster 2024-02-13T03:16:28.952751+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:32.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:32 smithi161 bash[19769]: cluster 2024-02-13T03:16:30.953290+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:32.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:32 smithi161 bash[27879]: cluster 2024-02-13T03:16:30.953290+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:32 smithi183 bash[20824]: cluster 2024-02-13T03:16:30.953290+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:34.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:34 smithi161 bash[19769]: cluster 2024-02-13T03:16:32.953786+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:34.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:34 smithi161 bash[27879]: cluster 2024-02-13T03:16:32.953786+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:34 smithi183 bash[20824]: cluster 2024-02-13T03:16:32.953786+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:36.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:36 smithi161 bash[19769]: cluster 2024-02-13T03:16:34.954301+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:36.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:36 smithi161 bash[27879]: cluster 2024-02-13T03:16:34.954301+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:36.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:36 smithi183 bash[20824]: cluster 2024-02-13T03:16:34.954301+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:37.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:37 smithi183 bash[20824]: audit 2024-02-13T03:16:37.029825+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-13T03:16:37.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:37 smithi183 bash[20824]: audit 2024-02-13T03:16:37.031121+0000 mon.a (mon.0) 269 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:37.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:37 smithi161 bash[19769]: audit 2024-02-13T03:16:37.029825+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-13T03:16:37.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:37 smithi161 bash[19769]: audit 2024-02-13T03:16:37.031121+0000 mon.a (mon.0) 269 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:37.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:37 smithi161 bash[27879]: audit 2024-02-13T03:16:37.029825+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-13T03:16:37.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:37 smithi161 bash[27879]: audit 2024-02-13T03:16:37.031121+0000 mon.a (mon.0) 269 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:38.352 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:38 smithi161 bash[27879]: cluster 2024-02-13T03:16:36.954834+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:38.352 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:38 smithi161 bash[27879]: cephadm 2024-02-13T03:16:37.032358+0000 mgr.y (mgr.14146) 103 : cephadm [INF] Deploying daemon osd.1 on smithi161 2024-02-13T03:16:38.353 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:38 smithi161 bash[19769]: cluster 2024-02-13T03:16:36.954834+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:38.353 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:38 smithi161 bash[19769]: cephadm 2024-02-13T03:16:37.032358+0000 mgr.y (mgr.14146) 103 : cephadm [INF] Deploying daemon osd.1 on smithi161 2024-02-13T03:16:38.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:38 smithi183 bash[20824]: cluster 2024-02-13T03:16:36.954834+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:38.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:38 smithi183 bash[20824]: cephadm 2024-02-13T03:16:37.032358+0000 mgr.y (mgr.14146) 103 : cephadm [INF] Deploying daemon osd.1 on smithi161 2024-02-13T03:16:40.195 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:40 smithi161 bash[19769]: cluster 2024-02-13T03:16:38.955345+0000 mgr.y (mgr.14146) 104 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:40.467 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:40 smithi161 bash[27879]: cluster 2024-02-13T03:16:38.955345+0000 mgr.y (mgr.14146) 104 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:40 smithi183 bash[20824]: cluster 2024-02-13T03:16:38.955345+0000 mgr.y (mgr.14146) 104 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:41 smithi183 bash[20824]: audit 2024-02-13T03:16:40.772357+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:16:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:41 smithi183 bash[20824]: audit 2024-02-13T03:16:40.778067+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:41.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:41 smithi161 bash[27879]: audit 2024-02-13T03:16:40.772357+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:16:41.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:41 smithi161 bash[27879]: audit 2024-02-13T03:16:40.778067+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:41.666 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:41 smithi161 bash[19769]: audit 2024-02-13T03:16:40.772357+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:16:41.667 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:41 smithi161 bash[19769]: audit 2024-02-13T03:16:40.778067+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:42 smithi183 bash[20824]: cluster 2024-02-13T03:16:40.955795+0000 mgr.y (mgr.14146) 105 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:42.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:42 smithi161 bash[27879]: cluster 2024-02-13T03:16:40.955795+0000 mgr.y (mgr.14146) 105 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:42.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:42 smithi161 bash[19769]: cluster 2024-02-13T03:16:40.955795+0000 mgr.y (mgr.14146) 105 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:44.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:44 smithi161 bash[19769]: cluster 2024-02-13T03:16:42.956398+0000 mgr.y (mgr.14146) 106 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:44.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:44 smithi161 bash[27879]: cluster 2024-02-13T03:16:42.956398+0000 mgr.y (mgr.14146) 106 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:44 smithi183 bash[20824]: cluster 2024-02-13T03:16:42.956398+0000 mgr.y (mgr.14146) 106 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:46.469 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:46 smithi161 bash[19769]: cluster 2024-02-13T03:16:44.956940+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:46 smithi161 bash[27879]: cluster 2024-02-13T03:16:44.956940+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:46 smithi183 bash[20824]: cluster 2024-02-13T03:16:44.956940+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:47 smithi183 bash[20824]: audit 2024-02-13T03:16:46.230775+0000 mon.a (mon.0) 272 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:47 smithi183 bash[20824]: audit 2024-02-13T03:16:46.522134+0000 mon.a (mon.0) 273 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:47 smithi183 bash[20824]: audit 2024-02-13T03:16:46.523804+0000 mon.a (mon.0) 274 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:16:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:47 smithi183 bash[20824]: audit 2024-02-13T03:16:46.531837+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:47 smithi183 bash[20824]: audit 2024-02-13T03:16:46.941562+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 v1:172.21.15.161:6805/72291598' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-13T03:16:47.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:47 smithi161 bash[19769]: audit 2024-02-13T03:16:46.230775+0000 mon.a (mon.0) 272 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:47.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:47 smithi161 bash[19769]: audit 2024-02-13T03:16:46.522134+0000 mon.a (mon.0) 273 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:47 smithi161 bash[19769]: audit 2024-02-13T03:16:46.523804+0000 mon.a (mon.0) 274 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:16:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:47 smithi161 bash[19769]: audit 2024-02-13T03:16:46.531837+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:47 smithi161 bash[19769]: audit 2024-02-13T03:16:46.941562+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 v1:172.21.15.161:6805/72291598' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-13T03:16:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:47 smithi161 bash[27879]: audit 2024-02-13T03:16:46.230775+0000 mon.a (mon.0) 272 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:47.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:47 smithi161 bash[27879]: audit 2024-02-13T03:16:46.522134+0000 mon.a (mon.0) 273 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:47.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:47 smithi161 bash[27879]: audit 2024-02-13T03:16:46.523804+0000 mon.a (mon.0) 274 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:16:47.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:47 smithi161 bash[27879]: audit 2024-02-13T03:16:46.531837+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:47.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:47 smithi161 bash[27879]: audit 2024-02-13T03:16:46.941562+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 v1:172.21.15.161:6805/72291598' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-13T03:16:48.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:48 smithi161 bash[27879]: cluster 2024-02-13T03:16:46.957481+0000 mgr.y (mgr.14146) 108 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:48.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:48 smithi161 bash[27879]: audit 2024-02-13T03:16:47.541111+0000 mon.a (mon.0) 277 : audit [INF] from='osd.1 v1:172.21.15.161:6805/72291598' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-13T03:16:48.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:48 smithi161 bash[27879]: cluster 2024-02-13T03:16:47.541246+0000 mon.a (mon.0) 278 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-13T03:16:48.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:48 smithi161 bash[27879]: audit 2024-02-13T03:16:47.541400+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:48.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:48 smithi161 bash[27879]: audit 2024-02-13T03:16:47.541944+0000 mon.a (mon.0) 280 : audit [INF] from='osd.1 v1:172.21.15.161:6805/72291598' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:16:48.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:48 smithi161 bash[19769]: cluster 2024-02-13T03:16:46.957481+0000 mgr.y (mgr.14146) 108 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:48.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:48 smithi161 bash[19769]: audit 2024-02-13T03:16:47.541111+0000 mon.a (mon.0) 277 : audit [INF] from='osd.1 v1:172.21.15.161:6805/72291598' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-13T03:16:48.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:48 smithi161 bash[19769]: cluster 2024-02-13T03:16:47.541246+0000 mon.a (mon.0) 278 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-13T03:16:48.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:48 smithi161 bash[19769]: audit 2024-02-13T03:16:47.541400+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:48.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:48 smithi161 bash[19769]: audit 2024-02-13T03:16:47.541944+0000 mon.a (mon.0) 280 : audit [INF] from='osd.1 v1:172.21.15.161:6805/72291598' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:16:48.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:48 smithi183 bash[20824]: cluster 2024-02-13T03:16:46.957481+0000 mgr.y (mgr.14146) 108 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:48.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:48 smithi183 bash[20824]: audit 2024-02-13T03:16:47.541111+0000 mon.a (mon.0) 277 : audit [INF] from='osd.1 v1:172.21.15.161:6805/72291598' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-13T03:16:48.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:48 smithi183 bash[20824]: cluster 2024-02-13T03:16:47.541246+0000 mon.a (mon.0) 278 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-13T03:16:48.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:48 smithi183 bash[20824]: audit 2024-02-13T03:16:47.541400+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:48.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:48 smithi183 bash[20824]: audit 2024-02-13T03:16:47.541944+0000 mon.a (mon.0) 280 : audit [INF] from='osd.1 v1:172.21.15.161:6805/72291598' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:16:49.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:49.177+0000 7fd70b7fe700 1 -- 172.21.15.161:0/3335480451 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 428992101) 0x7fd6fc000b90 con 0x7fd704062270 2024-02-13T03:16:49.181 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 1 on host 'smithi161' 2024-02-13T03:16:49.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:49.181+0000 7fd720b85700 1 -- 172.21.15.161:0/3335480451 >> v1:172.21.15.161:6800/1187535987 conn(0x7fd704062270 legacy=0x7fd704064730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:16:49.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:49.181+0000 7fd720b85700 1 -- 172.21.15.161:0/3335480451 >> v1:172.21.15.161:6790/0 conn(0x7fd71c0f67c0 legacy=0x7fd71c1077b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:16:49.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:49.181+0000 7fd720b85700 1 -- 172.21.15.161:0/3335480451 shutdown_connections 2024-02-13T03:16:49.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:49.181+0000 7fd720b85700 1 -- 172.21.15.161:0/3335480451 >> 172.21.15.161:0/3335480451 conn(0x7fd71c0f1f00 msgr2=0x7fd71c060e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:16:49.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:49.181+0000 7fd720b85700 1 -- 172.21.15.161:0/3335480451 shutdown_connections 2024-02-13T03:16:49.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:49.181+0000 7fd720b85700 1 -- 172.21.15.161:0/3335480451 wait complete. 2024-02-13T03:16:49.789 DEBUG:teuthology.orchestra.run.smithi161:osd.1> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.1.service 2024-02-13T03:16:49.792 INFO:tasks.cephadm:Deploying osd.2 on smithi161 with /dev/vg_nvme/lv_2... 2024-02-13T03:16:49.792 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-13T03:16:49.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:49 smithi161 bash[27879]: audit 2024-02-13T03:16:48.544498+0000 mon.a (mon.0) 281 : audit [INF] from='osd.1 v1:172.21.15.161:6805/72291598' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-02-13T03:16:49.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:49 smithi161 bash[27879]: cluster 2024-02-13T03:16:48.544766+0000 mon.a (mon.0) 282 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-13T03:16:49.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:49 smithi161 bash[27879]: audit 2024-02-13T03:16:48.545478+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:49.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:49 smithi161 bash[27879]: audit 2024-02-13T03:16:48.551856+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:49.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:49 smithi161 bash[27879]: audit 2024-02-13T03:16:49.178315+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:49.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:49 smithi161 bash[27879]: audit 2024-02-13T03:16:49.204480+0000 mon.a (mon.0) 286 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:16:49.845 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:49 smithi161 bash[19769]: audit 2024-02-13T03:16:48.544498+0000 mon.a (mon.0) 281 : audit [INF] from='osd.1 v1:172.21.15.161:6805/72291598' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-02-13T03:16:49.845 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:49 smithi161 bash[19769]: cluster 2024-02-13T03:16:48.544766+0000 mon.a (mon.0) 282 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-13T03:16:49.845 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:49 smithi161 bash[19769]: audit 2024-02-13T03:16:48.545478+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:49.845 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:49 smithi161 bash[19769]: audit 2024-02-13T03:16:48.551856+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:49.846 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:49 smithi161 bash[19769]: audit 2024-02-13T03:16:49.178315+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:49.846 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:49 smithi161 bash[19769]: audit 2024-02-13T03:16:49.204480+0000 mon.a (mon.0) 286 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:16:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:49 smithi183 bash[20824]: audit 2024-02-13T03:16:48.544498+0000 mon.a (mon.0) 281 : audit [INF] from='osd.1 v1:172.21.15.161:6805/72291598' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-02-13T03:16:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:49 smithi183 bash[20824]: cluster 2024-02-13T03:16:48.544766+0000 mon.a (mon.0) 282 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-13T03:16:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:49 smithi183 bash[20824]: audit 2024-02-13T03:16:48.545478+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:49 smithi183 bash[20824]: audit 2024-02-13T03:16:48.551856+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:49.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:49 smithi183 bash[20824]: audit 2024-02-13T03:16:49.178315+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:16:49.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:49 smithi183 bash[20824]: audit 2024-02-13T03:16:49.204480+0000 mon.a (mon.0) 286 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:16:50.007 INFO:journalctl@ceph.osd.1.smithi161.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:16:50.905 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:50 smithi161 bash[27879]: cluster 2024-02-13T03:16:47.940559+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:16:50.906 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:50 smithi161 bash[27879]: cluster 2024-02-13T03:16:47.940667+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:16:50.906 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:50 smithi161 bash[27879]: cluster 2024-02-13T03:16:48.957924+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:50.906 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:50 smithi161 bash[27879]: cluster 2024-02-13T03:16:49.556708+0000 mon.a (mon.0) 287 : cluster [INF] osd.1 v1:172.21.15.161:6805/72291598 boot 2024-02-13T03:16:50.906 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:50 smithi161 bash[27879]: cluster 2024-02-13T03:16:49.556748+0000 mon.a (mon.0) 288 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-13T03:16:50.906 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:50 smithi161 bash[27879]: audit 2024-02-13T03:16:49.557311+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:50.906 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:50 smithi161 bash[19769]: cluster 2024-02-13T03:16:47.940559+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:16:50.906 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:50 smithi161 bash[19769]: cluster 2024-02-13T03:16:47.940667+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:16:50.906 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:50 smithi161 bash[19769]: cluster 2024-02-13T03:16:48.957924+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:50.907 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:50 smithi161 bash[19769]: cluster 2024-02-13T03:16:49.556708+0000 mon.a (mon.0) 287 : cluster [INF] osd.1 v1:172.21.15.161:6805/72291598 boot 2024-02-13T03:16:50.907 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:50 smithi161 bash[19769]: cluster 2024-02-13T03:16:49.556748+0000 mon.a (mon.0) 288 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-13T03:16:50.907 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:50 smithi161 bash[19769]: audit 2024-02-13T03:16:49.557311+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:50.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:50 smithi183 bash[20824]: cluster 2024-02-13T03:16:47.940559+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:16:50.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:50 smithi183 bash[20824]: cluster 2024-02-13T03:16:47.940667+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:16:50.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:50 smithi183 bash[20824]: cluster 2024-02-13T03:16:48.957924+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-13T03:16:50.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:50 smithi183 bash[20824]: cluster 2024-02-13T03:16:49.556708+0000 mon.a (mon.0) 287 : cluster [INF] osd.1 v1:172.21.15.161:6805/72291598 boot 2024-02-13T03:16:50.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:50 smithi183 bash[20824]: cluster 2024-02-13T03:16:49.556748+0000 mon.a (mon.0) 288 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-13T03:16:50.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:50 smithi183 bash[20824]: audit 2024-02-13T03:16:49.557311+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:16:51.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:51 smithi161 bash[27879]: cluster 2024-02-13T03:16:50.575942+0000 mon.a (mon.0) 290 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-13T03:16:51.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:51 smithi161 bash[19769]: cluster 2024-02-13T03:16:50.575942+0000 mon.a (mon.0) 290 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-13T03:16:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:51 smithi183 bash[20824]: cluster 2024-02-13T03:16:50.575942+0000 mon.a (mon.0) 290 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-13T03:16:52.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:52 smithi161 bash[19769]: cluster 2024-02-13T03:16:50.958542+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:16:52.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:52 smithi161 bash[27879]: cluster 2024-02-13T03:16:50.958542+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:16:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:52 smithi183 bash[20824]: cluster 2024-02-13T03:16:50.958542+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:16:53.142 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-13T03:16:53.142 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-13T03:16:53.142 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-02-13T03:16:53.142 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-02-13T03:16:53.143 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0211654 s, 495 MB/s 2024-02-13T03:16:53.143 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-02-13T03:16:53.777 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-02-13T03:16:53.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:53 smithi161 bash[27879]: cluster 2024-02-13T03:16:52.959105+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:16:53.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:53 smithi161 bash[19769]: cluster 2024-02-13T03:16:52.959105+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:16:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:53 smithi183 bash[20824]: cluster 2024-02-13T03:16:52.959105+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:16:56.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:56 smithi161 bash[19769]: cluster 2024-02-13T03:16:54.959658+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:16:56.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:56 smithi161 bash[27879]: cluster 2024-02-13T03:16:54.959658+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:16:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:56 smithi183 bash[20824]: cluster 2024-02-13T03:16:54.959658+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:16:57.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.653+0000 7fb951c54700 1 -- 172.21.15.161:0/2431809903 >> v1:172.21.15.161:6789/0 conn(0x7fb94c0fe730 legacy=0x7fb94c100bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:16:57.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.653+0000 7fb951c54700 1 -- 172.21.15.161:0/2431809903 shutdown_connections 2024-02-13T03:16:57.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.653+0000 7fb951c54700 1 -- 172.21.15.161:0/2431809903 >> 172.21.15.161:0/2431809903 conn(0x7fb94c0f1e10 msgr2=0x7fb94c0f4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:16:57.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.653+0000 7fb951c54700 1 -- 172.21.15.161:0/2431809903 shutdown_connections 2024-02-13T03:16:57.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.653+0000 7fb951c54700 1 -- 172.21.15.161:0/2431809903 wait complete. 2024-02-13T03:16:57.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb951c54700 1 Processor -- start 2024-02-13T03:16:57.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb951c54700 1 -- start start 2024-02-13T03:16:57.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb951c54700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb94c111b70 con 0x7fb94c0fab60 2024-02-13T03:16:57.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb951c54700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb94c112d10 con 0x7fb94c0f65f0 2024-02-13T03:16:57.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb951c54700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb94c113eb0 con 0x7fb94c0fe730 2024-02-13T03:16:57.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb94bfff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb94c0fab60 0x7fb94c10cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:49418/0 (socket says 172.21.15.161:49418) 2024-02-13T03:16:57.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb94bfff700 1 -- 172.21.15.161:0/3784245983 learned_addr learned my addr 172.21.15.161:0/3784245983 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:16:57.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2946255048 0 0) 0x7fb94c111b70 con 0x7fb94c0fab60 2024-02-13T03:16:57.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb934003630 con 0x7fb94c0fab60 2024-02-13T03:16:57.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 53412931 0 0) 0x7fb94c113eb0 con 0x7fb94c0fe730 2024-02-13T03:16:57.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb94c111b70 con 0x7fb94c0fe730 2024-02-13T03:16:57.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648544629 0 0) 0x7fb94c112d10 con 0x7fb94c0f65f0 2024-02-13T03:16:57.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb94c113eb0 con 0x7fb94c0f65f0 2024-02-13T03:16:57.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2331445066 0 0) 0x7fb934003630 con 0x7fb94c0fab60 2024-02-13T03:16:57.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb94c112d10 con 0x7fb94c0fab60 2024-02-13T03:16:57.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4117393142 0 0) 0x7fb94c111b70 con 0x7fb94c0fe730 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb934003630 con 0x7fb94c0fe730 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb940001980 con 0x7fb94c0fab60 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb93c001ec0 con 0x7fb94c0fe730 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4286383380 0 0) 0x7fb94c113eb0 con 0x7fb94c0f65f0 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb94c111b70 con 0x7fb94c0f65f0 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb944001d90 con 0x7fb94c0f65f0 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 469456188 0 0) 0x7fb94c112d10 con 0x7fb94c0fab60 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 >> v1:172.21.15.161:6790/0 conn(0x7fb94c0fe730 legacy=0x7fb94c110450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 >> v1:172.21.15.183:6789/0 conn(0x7fb94c0f65f0 legacy=0x7fb94c106700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c115050 con 0x7fb94c0fab60 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb951c54700 1 -- 172.21.15.161:0/3784245983 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c1140a0 con 0x7fb94c0fab60 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb951c54700 1 -- 172.21.15.161:0/3784245983 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c114590 con 0x7fb94c0fab60 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fb940001b70 con 0x7fb94c0fab60 2024-02-13T03:16:57.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb940005b00 con 0x7fb94c0fab60 2024-02-13T03:16:57.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb951c54700 1 -- 172.21.15.161:0/3784245983 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb938004f20 con 0x7fb94c0fab60 2024-02-13T03:16:57.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.657+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 1965425884 0 0) 0x7fb9400021b0 con 0x7fb94c0fab60 2024-02-13T03:16:57.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.661+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(14..14 src has 1..14) v4 ==== 1776+0+0 (unknown 2677439825 0 0) 0x7fb940077250 con 0x7fb94c0fab60 2024-02-13T03:16:57.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.665+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fb940048410 con 0x7fb94c0fab60 2024-02-13T03:16:57.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:16:57.865+0000 7fb951c54700 1 -- 172.21.15.161:0/3784245983 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb938000b90 con 0x7fb9340643b0 2024-02-13T03:16:58.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:58 smithi161 bash[19769]: cluster 2024-02-13T03:16:56.960189+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:16:58.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:58 smithi161 bash[19769]: audit 2024-02-13T03:16:57.871868+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:16:58.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:58 smithi161 bash[19769]: audit 2024-02-13T03:16:57.876966+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:16:58.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:58 smithi161 bash[19769]: audit 2024-02-13T03:16:57.878386+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:58.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:58 smithi161 bash[27879]: cluster 2024-02-13T03:16:56.960189+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:16:58.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:58 smithi161 bash[27879]: audit 2024-02-13T03:16:57.871868+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:16:58.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:58 smithi161 bash[27879]: audit 2024-02-13T03:16:57.876966+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:16:58.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:58 smithi161 bash[27879]: audit 2024-02-13T03:16:57.878386+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:58 smithi183 bash[20824]: cluster 2024-02-13T03:16:56.960189+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:16:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:58 smithi183 bash[20824]: audit 2024-02-13T03:16:57.871868+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:16:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:58 smithi183 bash[20824]: audit 2024-02-13T03:16:57.876966+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:16:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:58 smithi183 bash[20824]: audit 2024-02-13T03:16:57.878386+0000 mon.a (mon.0) 293 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:16:59.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:16:59 smithi161 bash[19769]: audit 2024-02-13T03:16:57.867972+0000 mgr.y (mgr.14146) 114 : audit [DBG] from='client.14265 v1:172.21.15.161:0/3784245983' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:16:59.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:16:59 smithi161 bash[27879]: audit 2024-02-13T03:16:57.867972+0000 mgr.y (mgr.14146) 114 : audit [DBG] from='client.14265 v1:172.21.15.161:0/3784245983' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:16:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:16:59 smithi183 bash[20824]: audit 2024-02-13T03:16:57.867972+0000 mgr.y (mgr.14146) 114 : audit [DBG] from='client.14265 v1:172.21.15.161:0/3784245983' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:17:00.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[19769]: cluster 2024-02-13T03:16:58.960635+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:00.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[19769]: cephadm 2024-02-13T03:16:59.318322+0000 mgr.y (mgr.14146) 116 : cephadm [INF] Detected new or changed devices on smithi161 2024-02-13T03:17:00.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[19769]: audit 2024-02-13T03:16:59.325807+0000 mon.a (mon.0) 294 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:00.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[19769]: audit 2024-02-13T03:16:59.327531+0000 mon.a (mon.0) 295 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:00.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[19769]: audit 2024-02-13T03:16:59.329005+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:00.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[19769]: cephadm 2024-02-13T03:16:59.330022+0000 mgr.y (mgr.14146) 117 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 8147M 2024-02-13T03:17:00.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[19769]: audit 2024-02-13T03:16:59.336369+0000 mon.a (mon.0) 297 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:00.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[19769]: audit 2024-02-13T03:16:59.338380+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:00.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[19769]: audit 2024-02-13T03:16:59.339505+0000 mon.a (mon.0) 299 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:17:00.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[19769]: audit 2024-02-13T03:16:59.345980+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[27879]: cluster 2024-02-13T03:16:58.960635+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[27879]: cephadm 2024-02-13T03:16:59.318322+0000 mgr.y (mgr.14146) 116 : cephadm [INF] Detected new or changed devices on smithi161 2024-02-13T03:17:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[27879]: audit 2024-02-13T03:16:59.325807+0000 mon.a (mon.0) 294 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[27879]: audit 2024-02-13T03:16:59.327531+0000 mon.a (mon.0) 295 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[27879]: audit 2024-02-13T03:16:59.329005+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[27879]: cephadm 2024-02-13T03:16:59.330022+0000 mgr.y (mgr.14146) 117 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 8147M 2024-02-13T03:17:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[27879]: audit 2024-02-13T03:16:59.336369+0000 mon.a (mon.0) 297 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[27879]: audit 2024-02-13T03:16:59.338380+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[27879]: audit 2024-02-13T03:16:59.339505+0000 mon.a (mon.0) 299 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:17:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:00 smithi161 bash[27879]: audit 2024-02-13T03:16:59.345980+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:00 smithi183 bash[20824]: cluster 2024-02-13T03:16:58.960635+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:00 smithi183 bash[20824]: cephadm 2024-02-13T03:16:59.318322+0000 mgr.y (mgr.14146) 116 : cephadm [INF] Detected new or changed devices on smithi161 2024-02-13T03:17:00.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:00 smithi183 bash[20824]: audit 2024-02-13T03:16:59.325807+0000 mon.a (mon.0) 294 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:00.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:00 smithi183 bash[20824]: audit 2024-02-13T03:16:59.327531+0000 mon.a (mon.0) 295 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:00.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:00 smithi183 bash[20824]: audit 2024-02-13T03:16:59.329005+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:00.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:00 smithi183 bash[20824]: cephadm 2024-02-13T03:16:59.330022+0000 mgr.y (mgr.14146) 117 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 8147M 2024-02-13T03:17:00.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:00 smithi183 bash[20824]: audit 2024-02-13T03:16:59.336369+0000 mon.a (mon.0) 297 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:00.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:00 smithi183 bash[20824]: audit 2024-02-13T03:16:59.338380+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:00.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:00 smithi183 bash[20824]: audit 2024-02-13T03:16:59.339505+0000 mon.a (mon.0) 299 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:17:00.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:00 smithi183 bash[20824]: audit 2024-02-13T03:16:59.345980+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:02 smithi161 bash[19769]: cluster 2024-02-13T03:17:00.961146+0000 mgr.y (mgr.14146) 118 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:02.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:02 smithi161 bash[27879]: cluster 2024-02-13T03:17:00.961146+0000 mgr.y (mgr.14146) 118 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:02 smithi183 bash[20824]: cluster 2024-02-13T03:17:00.961146+0000 mgr.y (mgr.14146) 118 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:04.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:04 smithi161 bash[19769]: cluster 2024-02-13T03:17:02.961636+0000 mgr.y (mgr.14146) 119 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:04.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:04 smithi161 bash[27879]: cluster 2024-02-13T03:17:02.961636+0000 mgr.y (mgr.14146) 119 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:04 smithi183 bash[20824]: cluster 2024-02-13T03:17:02.961636+0000 mgr.y (mgr.14146) 119 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:06 smithi161 bash[19769]: cluster 2024-02-13T03:17:04.962128+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:06 smithi161 bash[19769]: audit 2024-02-13T03:17:05.885155+0000 mon.a (mon.0) 301 : audit [INF] from='client.? v1:172.21.15.161:0/377457022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c521de51-8f7c-4ec2-94b7-96ca43b52d3a"}]: dispatch 2024-02-13T03:17:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:06 smithi161 bash[19769]: audit 2024-02-13T03:17:05.896468+0000 mon.a (mon.0) 302 : audit [INF] from='client.? v1:172.21.15.161:0/377457022' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c521de51-8f7c-4ec2-94b7-96ca43b52d3a"}]': finished 2024-02-13T03:17:06.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:06 smithi161 bash[19769]: cluster 2024-02-13T03:17:05.896511+0000 mon.a (mon.0) 303 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-13T03:17:06.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:06 smithi161 bash[19769]: audit 2024-02-13T03:17:05.896614+0000 mon.a (mon.0) 304 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:06.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:06 smithi161 bash[27879]: cluster 2024-02-13T03:17:04.962128+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:06.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:06 smithi161 bash[27879]: audit 2024-02-13T03:17:05.885155+0000 mon.a (mon.0) 301 : audit [INF] from='client.? v1:172.21.15.161:0/377457022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c521de51-8f7c-4ec2-94b7-96ca43b52d3a"}]: dispatch 2024-02-13T03:17:06.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:06 smithi161 bash[27879]: audit 2024-02-13T03:17:05.896468+0000 mon.a (mon.0) 302 : audit [INF] from='client.? v1:172.21.15.161:0/377457022' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c521de51-8f7c-4ec2-94b7-96ca43b52d3a"}]': finished 2024-02-13T03:17:06.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:06 smithi161 bash[27879]: cluster 2024-02-13T03:17:05.896511+0000 mon.a (mon.0) 303 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-13T03:17:06.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:06 smithi161 bash[27879]: audit 2024-02-13T03:17:05.896614+0000 mon.a (mon.0) 304 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:06.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:06 smithi183 bash[20824]: cluster 2024-02-13T03:17:04.962128+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:06.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:06 smithi183 bash[20824]: audit 2024-02-13T03:17:05.885155+0000 mon.a (mon.0) 301 : audit [INF] from='client.? v1:172.21.15.161:0/377457022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c521de51-8f7c-4ec2-94b7-96ca43b52d3a"}]: dispatch 2024-02-13T03:17:06.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:06 smithi183 bash[20824]: audit 2024-02-13T03:17:05.896468+0000 mon.a (mon.0) 302 : audit [INF] from='client.? v1:172.21.15.161:0/377457022' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c521de51-8f7c-4ec2-94b7-96ca43b52d3a"}]': finished 2024-02-13T03:17:06.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:06 smithi183 bash[20824]: cluster 2024-02-13T03:17:05.896511+0000 mon.a (mon.0) 303 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-13T03:17:06.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:06 smithi183 bash[20824]: audit 2024-02-13T03:17:05.896614+0000 mon.a (mon.0) 304 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:07 smithi161 bash[19769]: audit 2024-02-13T03:17:06.502905+0000 mon.a (mon.0) 305 : audit [DBG] from='client.? v1:172.21.15.161:0/3809959302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:17:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:07 smithi161 bash[27879]: audit 2024-02-13T03:17:06.502905+0000 mon.a (mon.0) 305 : audit [DBG] from='client.? v1:172.21.15.161:0/3809959302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:17:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:07 smithi183 bash[20824]: audit 2024-02-13T03:17:06.502905+0000 mon.a (mon.0) 305 : audit [DBG] from='client.? v1:172.21.15.161:0/3809959302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:17:08.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:08 smithi161 bash[19769]: cluster 2024-02-13T03:17:06.962606+0000 mgr.y (mgr.14146) 121 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:08 smithi161 bash[27879]: cluster 2024-02-13T03:17:06.962606+0000 mgr.y (mgr.14146) 121 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:08 smithi183 bash[20824]: cluster 2024-02-13T03:17:06.962606+0000 mgr.y (mgr.14146) 121 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:10.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:10 smithi161 bash[19769]: cluster 2024-02-13T03:17:08.963004+0000 mgr.y (mgr.14146) 122 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:10 smithi161 bash[27879]: cluster 2024-02-13T03:17:08.963004+0000 mgr.y (mgr.14146) 122 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:10 smithi183 bash[20824]: cluster 2024-02-13T03:17:08.963004+0000 mgr.y (mgr.14146) 122 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:12.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:12 smithi161 bash[19769]: cluster 2024-02-13T03:17:10.963529+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:12.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:12 smithi161 bash[27879]: cluster 2024-02-13T03:17:10.963529+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:12 smithi183 bash[20824]: cluster 2024-02-13T03:17:10.963529+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:14.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:14 smithi161 bash[19769]: cluster 2024-02-13T03:17:12.964026+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:14.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:14 smithi161 bash[27879]: cluster 2024-02-13T03:17:12.964026+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:14 smithi183 bash[20824]: cluster 2024-02-13T03:17:12.964026+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:16.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:16 smithi161 bash[27879]: cluster 2024-02-13T03:17:14.964567+0000 mgr.y (mgr.14146) 125 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:16.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:16 smithi161 bash[19769]: cluster 2024-02-13T03:17:14.964567+0000 mgr.y (mgr.14146) 125 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:16 smithi183 bash[20824]: cluster 2024-02-13T03:17:14.964567+0000 mgr.y (mgr.14146) 125 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:18.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:18 smithi161 bash[19769]: cluster 2024-02-13T03:17:16.965152+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:18.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:18 smithi161 bash[19769]: audit 2024-02-13T03:17:17.770922+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-13T03:17:18.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:18 smithi161 bash[19769]: audit 2024-02-13T03:17:17.772215+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:18.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:18 smithi161 bash[27879]: cluster 2024-02-13T03:17:16.965152+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:18.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:18 smithi161 bash[27879]: audit 2024-02-13T03:17:17.770922+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-13T03:17:18.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:18 smithi161 bash[27879]: audit 2024-02-13T03:17:17.772215+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:18 smithi183 bash[20824]: cluster 2024-02-13T03:17:16.965152+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:18.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:18 smithi183 bash[20824]: audit 2024-02-13T03:17:17.770922+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-13T03:17:18.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:18 smithi183 bash[20824]: audit 2024-02-13T03:17:17.772215+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:19.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:19 smithi161 bash[19769]: cephadm 2024-02-13T03:17:17.773312+0000 mgr.y (mgr.14146) 127 : cephadm [INF] Deploying daemon osd.2 on smithi161 2024-02-13T03:17:19.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:19 smithi161 bash[27879]: cephadm 2024-02-13T03:17:17.773312+0000 mgr.y (mgr.14146) 127 : cephadm [INF] Deploying daemon osd.2 on smithi161 2024-02-13T03:17:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:19 smithi183 bash[20824]: cephadm 2024-02-13T03:17:17.773312+0000 mgr.y (mgr.14146) 127 : cephadm [INF] Deploying daemon osd.2 on smithi161 2024-02-13T03:17:20.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:20 smithi161 bash[19769]: cluster 2024-02-13T03:17:18.965739+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v103: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:20.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:20 smithi161 bash[27879]: cluster 2024-02-13T03:17:18.965739+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v103: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:20 smithi183 bash[20824]: cluster 2024-02-13T03:17:18.965739+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v103: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:22 smithi161 bash[27879]: cluster 2024-02-13T03:17:20.966369+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:22 smithi161 bash[27879]: audit 2024-02-13T03:17:21.551112+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:22 smithi161 bash[27879]: audit 2024-02-13T03:17:21.567387+0000 mon.a (mon.0) 309 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:17:22.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:22 smithi161 bash[19769]: cluster 2024-02-13T03:17:20.966369+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:22.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:22 smithi161 bash[19769]: audit 2024-02-13T03:17:21.551112+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:22.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:22 smithi161 bash[19769]: audit 2024-02-13T03:17:21.567387+0000 mon.a (mon.0) 309 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:17:22.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:22 smithi183 bash[20824]: cluster 2024-02-13T03:17:20.966369+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:22.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:22 smithi183 bash[20824]: audit 2024-02-13T03:17:21.551112+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:22.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:22 smithi183 bash[20824]: audit 2024-02-13T03:17:21.567387+0000 mon.a (mon.0) 309 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:17:24.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:24 smithi161 bash[19769]: cluster 2024-02-13T03:17:22.966841+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:24 smithi161 bash[27879]: cluster 2024-02-13T03:17:22.966841+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:24.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:24 smithi183 bash[20824]: cluster 2024-02-13T03:17:22.966841+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:26.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:26 smithi161 bash[19769]: cluster 2024-02-13T03:17:24.967346+0000 mgr.y (mgr.14146) 131 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:26.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:26 smithi161 bash[27879]: cluster 2024-02-13T03:17:24.967346+0000 mgr.y (mgr.14146) 131 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:26 smithi183 bash[20824]: cluster 2024-02-13T03:17:24.967346+0000 mgr.y (mgr.14146) 131 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:28.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:27 smithi183 bash[20824]: cluster 2024-02-13T03:17:26.967716+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:28.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:27 smithi183 bash[20824]: audit 2024-02-13T03:17:26.967873+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:28.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:27 smithi183 bash[20824]: audit 2024-02-13T03:17:26.969254+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:28.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:27 smithi183 bash[20824]: audit 2024-02-13T03:17:26.970300+0000 mon.a (mon.0) 312 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:17:28.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:27 smithi183 bash[20824]: audit 2024-02-13T03:17:26.977264+0000 mon.a (mon.0) 313 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:28.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:27 smithi161 bash[19769]: cluster 2024-02-13T03:17:26.967716+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:28.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:27 smithi161 bash[19769]: audit 2024-02-13T03:17:26.967873+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:28.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:27 smithi161 bash[19769]: audit 2024-02-13T03:17:26.969254+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:28.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:27 smithi161 bash[19769]: audit 2024-02-13T03:17:26.970300+0000 mon.a (mon.0) 312 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:17:28.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:27 smithi161 bash[19769]: audit 2024-02-13T03:17:26.977264+0000 mon.a (mon.0) 313 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:28.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:27 smithi161 bash[27879]: cluster 2024-02-13T03:17:26.967716+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:28.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:27 smithi161 bash[27879]: audit 2024-02-13T03:17:26.967873+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:28.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:27 smithi161 bash[27879]: audit 2024-02-13T03:17:26.969254+0000 mon.a (mon.0) 311 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:28.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:27 smithi161 bash[27879]: audit 2024-02-13T03:17:26.970300+0000 mon.a (mon.0) 312 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:17:28.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:27 smithi161 bash[27879]: audit 2024-02-13T03:17:26.977264+0000 mon.a (mon.0) 313 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:29.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:28 smithi161 bash[19769]: audit 2024-02-13T03:17:27.972935+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 v1:172.21.15.161:6809/1657207182' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-13T03:17:29.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:28 smithi161 bash[27879]: audit 2024-02-13T03:17:27.972935+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 v1:172.21.15.161:6809/1657207182' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-13T03:17:29.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:28 smithi183 bash[20824]: audit 2024-02-13T03:17:27.972935+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 v1:172.21.15.161:6809/1657207182' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-13T03:17:30.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:30.000+0000 7fb949ffb700 1 -- 172.21.15.161:0/3784245983 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 4256159161) 0x7fb938000b90 con 0x7fb9340643b0 2024-02-13T03:17:30.005 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 2 on host 'smithi161' 2024-02-13T03:17:30.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:30.004+0000 7fb951c54700 1 -- 172.21.15.161:0/3784245983 >> v1:172.21.15.161:6800/1187535987 conn(0x7fb9340643b0 legacy=0x7fb934066870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:17:30.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:30.004+0000 7fb951c54700 1 -- 172.21.15.161:0/3784245983 >> v1:172.21.15.161:6789/0 conn(0x7fb94c0fab60 legacy=0x7fb94c10cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:17:30.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:30.004+0000 7fb951c54700 1 -- 172.21.15.161:0/3784245983 shutdown_connections 2024-02-13T03:17:30.006 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:30.004+0000 7fb951c54700 1 -- 172.21.15.161:0/3784245983 >> 172.21.15.161:0/3784245983 conn(0x7fb94c0f1e10 msgr2=0x7fb94c0f21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:17:30.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:30.004+0000 7fb951c54700 1 -- 172.21.15.161:0/3784245983 shutdown_connections 2024-02-13T03:17:30.007 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:30.004+0000 7fb951c54700 1 -- 172.21.15.161:0/3784245983 wait complete. 2024-02-13T03:17:30.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:29 smithi183 bash[20824]: cluster 2024-02-13T03:17:28.968191+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:30.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:29 smithi183 bash[20824]: audit 2024-02-13T03:17:28.979516+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 v1:172.21.15.161:6809/1657207182' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-13T03:17:30.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:29 smithi183 bash[20824]: cluster 2024-02-13T03:17:28.979617+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-13T03:17:30.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:29 smithi183 bash[20824]: audit 2024-02-13T03:17:28.980199+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:30.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:29 smithi183 bash[20824]: audit 2024-02-13T03:17:28.980874+0000 mon.a (mon.0) 318 : audit [INF] from='osd.2 v1:172.21.15.161:6809/1657207182' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:17:30.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:29 smithi161 bash[19769]: cluster 2024-02-13T03:17:28.968191+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:30.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:29 smithi161 bash[19769]: audit 2024-02-13T03:17:28.979516+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 v1:172.21.15.161:6809/1657207182' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-13T03:17:30.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:29 smithi161 bash[19769]: cluster 2024-02-13T03:17:28.979617+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-13T03:17:30.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:29 smithi161 bash[19769]: audit 2024-02-13T03:17:28.980199+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:30.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:29 smithi161 bash[19769]: audit 2024-02-13T03:17:28.980874+0000 mon.a (mon.0) 318 : audit [INF] from='osd.2 v1:172.21.15.161:6809/1657207182' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:17:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:29 smithi161 bash[27879]: cluster 2024-02-13T03:17:28.968191+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:29 smithi161 bash[27879]: audit 2024-02-13T03:17:28.979516+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 v1:172.21.15.161:6809/1657207182' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-13T03:17:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:29 smithi161 bash[27879]: cluster 2024-02-13T03:17:28.979617+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-13T03:17:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:29 smithi161 bash[27879]: audit 2024-02-13T03:17:28.980199+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:29 smithi161 bash[27879]: audit 2024-02-13T03:17:28.980874+0000 mon.a (mon.0) 318 : audit [INF] from='osd.2 v1:172.21.15.161:6809/1657207182' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:17:30.635 DEBUG:teuthology.orchestra.run.smithi161:osd.2> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.2.service 2024-02-13T03:17:30.637 INFO:tasks.cephadm:Deploying osd.3 on smithi161 with /dev/vg_nvme/lv_1... 2024-02-13T03:17:30.638 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-13T03:17:30.872 INFO:journalctl@ceph.osd.2.smithi161.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:17:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:31 smithi183 bash[20824]: cluster 2024-02-13T03:17:28.980915+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:17:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:31 smithi183 bash[20824]: cluster 2024-02-13T03:17:28.981000+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:17:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:31 smithi183 bash[20824]: audit 2024-02-13T03:17:29.981711+0000 mon.a (mon.0) 319 : audit [INF] from='osd.2 v1:172.21.15.161:6809/1657207182' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-02-13T03:17:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:31 smithi183 bash[20824]: cluster 2024-02-13T03:17:29.981879+0000 mon.a (mon.0) 320 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-13T03:17:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:31 smithi183 bash[20824]: audit 2024-02-13T03:17:29.983985+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:31 smithi183 bash[20824]: audit 2024-02-13T03:17:29.989108+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:31 smithi183 bash[20824]: audit 2024-02-13T03:17:30.002113+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:31 smithi183 bash[20824]: audit 2024-02-13T03:17:30.002451+0000 mon.a (mon.0) 324 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:17:31.786 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[19769]: cluster 2024-02-13T03:17:28.980915+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:17:31.787 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[19769]: cluster 2024-02-13T03:17:28.981000+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:17:31.787 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[19769]: audit 2024-02-13T03:17:29.981711+0000 mon.a (mon.0) 319 : audit [INF] from='osd.2 v1:172.21.15.161:6809/1657207182' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-02-13T03:17:31.787 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[19769]: cluster 2024-02-13T03:17:29.981879+0000 mon.a (mon.0) 320 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-13T03:17:31.787 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[19769]: audit 2024-02-13T03:17:29.983985+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:31.787 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[19769]: audit 2024-02-13T03:17:29.989108+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:31.787 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[19769]: audit 2024-02-13T03:17:30.002113+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:31.787 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[19769]: audit 2024-02-13T03:17:30.002451+0000 mon.a (mon.0) 324 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:17:31.787 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[27879]: cluster 2024-02-13T03:17:28.980915+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:17:31.788 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[27879]: cluster 2024-02-13T03:17:28.981000+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:17:31.788 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[27879]: audit 2024-02-13T03:17:29.981711+0000 mon.a (mon.0) 319 : audit [INF] from='osd.2 v1:172.21.15.161:6809/1657207182' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-02-13T03:17:31.788 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[27879]: cluster 2024-02-13T03:17:29.981879+0000 mon.a (mon.0) 320 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-13T03:17:31.788 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[27879]: audit 2024-02-13T03:17:29.983985+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:31.788 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[27879]: audit 2024-02-13T03:17:29.989108+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:31.788 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[27879]: audit 2024-02-13T03:17:30.002113+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:31.788 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:31 smithi161 bash[27879]: audit 2024-02-13T03:17:30.002451+0000 mon.a (mon.0) 324 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:17:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:32 smithi183 bash[20824]: cluster 2024-02-13T03:17:30.968815+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:32 smithi183 bash[20824]: cluster 2024-02-13T03:17:30.996426+0000 mon.a (mon.0) 325 : cluster [INF] osd.2 v1:172.21.15.161:6809/1657207182 boot 2024-02-13T03:17:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:32 smithi183 bash[20824]: cluster 2024-02-13T03:17:30.996458+0000 mon.a (mon.0) 326 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-13T03:17:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:32 smithi183 bash[20824]: audit 2024-02-13T03:17:30.996814+0000 mon.a (mon.0) 327 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:32.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:32 smithi183 bash[20824]: audit 2024-02-13T03:17:30.997293+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-13T03:17:32.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:32 smithi183 bash[20824]: audit 2024-02-13T03:17:31.992771+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-13T03:17:32.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:32 smithi183 bash[20824]: cluster 2024-02-13T03:17:31.992868+0000 mon.a (mon.0) 330 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-13T03:17:32.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:32 smithi183 bash[20824]: audit 2024-02-13T03:17:31.994164+0000 mon.a (mon.0) 331 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-13T03:17:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[27879]: cluster 2024-02-13T03:17:30.968815+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[27879]: cluster 2024-02-13T03:17:30.996426+0000 mon.a (mon.0) 325 : cluster [INF] osd.2 v1:172.21.15.161:6809/1657207182 boot 2024-02-13T03:17:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[27879]: cluster 2024-02-13T03:17:30.996458+0000 mon.a (mon.0) 326 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-13T03:17:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[27879]: audit 2024-02-13T03:17:30.996814+0000 mon.a (mon.0) 327 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[27879]: audit 2024-02-13T03:17:30.997293+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-13T03:17:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[27879]: audit 2024-02-13T03:17:31.992771+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-13T03:17:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[27879]: cluster 2024-02-13T03:17:31.992868+0000 mon.a (mon.0) 330 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-13T03:17:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[27879]: audit 2024-02-13T03:17:31.994164+0000 mon.a (mon.0) 331 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-13T03:17:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[19769]: cluster 2024-02-13T03:17:30.968815+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-13T03:17:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[19769]: cluster 2024-02-13T03:17:30.996426+0000 mon.a (mon.0) 325 : cluster [INF] osd.2 v1:172.21.15.161:6809/1657207182 boot 2024-02-13T03:17:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[19769]: cluster 2024-02-13T03:17:30.996458+0000 mon.a (mon.0) 326 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-13T03:17:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[19769]: audit 2024-02-13T03:17:30.996814+0000 mon.a (mon.0) 327 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:17:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[19769]: audit 2024-02-13T03:17:30.997293+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-13T03:17:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[19769]: audit 2024-02-13T03:17:31.992771+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-13T03:17:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[19769]: cluster 2024-02-13T03:17:31.992868+0000 mon.a (mon.0) 330 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-13T03:17:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:32 smithi161 bash[19769]: audit 2024-02-13T03:17:31.994164+0000 mon.a (mon.0) 331 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-13T03:17:34.002 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-13T03:17:34.003 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-13T03:17:34.003 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-02-13T03:17:34.003 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-02-13T03:17:34.003 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0239467 s, 438 MB/s 2024-02-13T03:17:34.003 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-02-13T03:17:34.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:34 smithi161 bash[27879]: cluster 2024-02-13T03:17:32.969522+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v114: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:34.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:34 smithi161 bash[27879]: audit 2024-02-13T03:17:32.996024+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-13T03:17:34.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:34 smithi161 bash[27879]: cluster 2024-02-13T03:17:32.996161+0000 mon.a (mon.0) 333 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-13T03:17:34.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:34 smithi161 bash[19769]: cluster 2024-02-13T03:17:32.969522+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v114: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:34.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:34 smithi161 bash[19769]: audit 2024-02-13T03:17:32.996024+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-13T03:17:34.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:34 smithi161 bash[19769]: cluster 2024-02-13T03:17:32.996161+0000 mon.a (mon.0) 333 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-13T03:17:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:34 smithi183 bash[20824]: cluster 2024-02-13T03:17:32.969522+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v114: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:34 smithi183 bash[20824]: audit 2024-02-13T03:17:32.996024+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-13T03:17:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:34 smithi183 bash[20824]: cluster 2024-02-13T03:17:32.996161+0000 mon.a (mon.0) 333 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-13T03:17:34.653 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-02-13T03:17:35.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:35 smithi161 bash[27879]: cluster 2024-02-13T03:17:34.013530+0000 mon.a (mon.0) 334 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-13T03:17:35.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:35 smithi161 bash[19769]: cluster 2024-02-13T03:17:34.013530+0000 mon.a (mon.0) 334 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-13T03:17:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:35 smithi183 bash[20824]: cluster 2024-02-13T03:17:34.013530+0000 mon.a (mon.0) 334 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-13T03:17:36.313 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:36 smithi161 bash[19769]: cluster 2024-02-13T03:17:34.970121+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v117: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:36.313 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:36 smithi161 bash[27879]: cluster 2024-02-13T03:17:34.970121+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v117: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:36.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:36 smithi183 bash[20824]: cluster 2024-02-13T03:17:34.970121+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v117: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:38.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:38 smithi161 bash[27879]: cluster 2024-02-13T03:17:36.970753+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:38.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:38 smithi161 bash[19769]: cluster 2024-02-13T03:17:36.970753+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:38.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:38 smithi183 bash[20824]: cluster 2024-02-13T03:17:36.970753+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:38.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/2291435805 >> v1:172.21.15.161:6790/0 conn(0x7f1d640fe9b0 legacy=0x7f1d64100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:17:38.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/2291435805 shutdown_connections 2024-02-13T03:17:38.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/2291435805 >> 172.21.15.161:0/2291435805 conn(0x7f1d640f16b0 msgr2=0x7f1d640f3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:17:38.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/2291435805 shutdown_connections 2024-02-13T03:17:38.795 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/2291435805 wait complete. 2024-02-13T03:17:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d6b7c2700 1 Processor -- start 2024-02-13T03:17:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d6b7c2700 1 -- start start 2024-02-13T03:17:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d6b7c2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1d641082e0 con 0x7f1d640f7e30 2024-02-13T03:17:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d6b7c2700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1d64113af0 con 0x7f1d640fe9b0 2024-02-13T03:17:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d6b7c2700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1d64114cf0 con 0x7f1d640fad80 2024-02-13T03:17:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d68d5d700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f1d640fad80 0x7f1d6410dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:34868/0 (socket says 172.21.15.161:34868) 2024-02-13T03:17:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d68d5d700 1 -- 172.21.15.161:0/317367632 learned_addr learned my addr 172.21.15.161:0/317367632 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:17:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d6955e700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1d640f7e30 0x7f1d64107680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:42264/0 (socket says 172.21.15.161:42264) 2024-02-13T03:17:38.798 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021870833 0 0) 0x7f1d64114cf0 con 0x7f1d640fad80 2024-02-13T03:17:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d50003110 con 0x7f1d640fad80 2024-02-13T03:17:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191343254 0 0) 0x7f1d641082e0 con 0x7f1d640f7e30 2024-02-13T03:17:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d64114cf0 con 0x7f1d640f7e30 2024-02-13T03:17:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299896296 0 0) 0x7f1d64113af0 con 0x7f1d640fe9b0 2024-02-13T03:17:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d641082e0 con 0x7f1d640fe9b0 2024-02-13T03:17:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4134638598 0 0) 0x7f1d50003110 con 0x7f1d640fad80 2024-02-13T03:17:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1d64113af0 con 0x7f1d640fad80 2024-02-13T03:17:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f1d54001980 con 0x7f1d640fad80 2024-02-13T03:17:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 114666847 0 0) 0x7f1d64114cf0 con 0x7f1d640f7e30 2024-02-13T03:17:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1d50003110 con 0x7f1d640f7e30 2024-02-13T03:17:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f1d60001860 con 0x7f1d640f7e30 2024-02-13T03:17:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2379988047 0 0) 0x7f1d641082e0 con 0x7f1d640fe9b0 2024-02-13T03:17:38.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1d64114cf0 con 0x7f1d640fe9b0 2024-02-13T03:17:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f1d5c001730 con 0x7f1d640fe9b0 2024-02-13T03:17:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3997404049 0 0) 0x7f1d64113af0 con 0x7f1d640fad80 2024-02-13T03:17:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 >> v1:172.21.15.183:6789/0 conn(0x7f1d640fe9b0 legacy=0x7f1d64111380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:17:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 >> v1:172.21.15.161:6789/0 conn(0x7f1d640f7e30 legacy=0x7f1d64107680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:17:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d64115ef0 con 0x7f1d640fad80 2024-02-13T03:17:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/317367632 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d64112b00 con 0x7f1d640fad80 2024-02-13T03:17:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f1d54001c40 con 0x7f1d640fad80 2024-02-13T03:17:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.792+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/317367632 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d64112ff0 con 0x7f1d640fad80 2024-02-13T03:17:38.800 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.796+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f1d54005b80 con 0x7f1d640fad80 2024-02-13T03:17:38.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.796+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 1965425884 0 0) 0x7f1d54005d80 con 0x7f1d640fad80 2024-02-13T03:17:38.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.796+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/317367632 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d64060710 con 0x7f1d640fad80 2024-02-13T03:17:38.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.796+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(21..21 src has 1..21) v4 ==== 2484+0+0 (unknown 2184232411 0 0) 0x7f1d54077800 con 0x7f1d640fad80 2024-02-13T03:17:38.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:38.804+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f1d64060710 con 0x7f1d640fad80 2024-02-13T03:17:39.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:17:39.036+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/317367632 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d64060710 con 0x7f1d500623e0 2024-02-13T03:17:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:39 smithi183 bash[20824]: audit 2024-02-13T03:17:39.046462+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:17:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:39 smithi183 bash[20824]: audit 2024-02-13T03:17:39.049695+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:17:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:39 smithi183 bash[20824]: audit 2024-02-13T03:17:39.050661+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:39.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:39 smithi161 bash[19769]: audit 2024-02-13T03:17:39.046462+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:17:39.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:39 smithi161 bash[19769]: audit 2024-02-13T03:17:39.049695+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:17:39.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:39 smithi161 bash[19769]: audit 2024-02-13T03:17:39.050661+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:39.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:39 smithi161 bash[27879]: audit 2024-02-13T03:17:39.046462+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:17:39.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:39 smithi161 bash[27879]: audit 2024-02-13T03:17:39.049695+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:17:39.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:39 smithi161 bash[27879]: audit 2024-02-13T03:17:39.050661+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:40 smithi183 bash[20824]: cluster 2024-02-13T03:17:38.971316+0000 mgr.y (mgr.14146) 138 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:40 smithi183 bash[20824]: audit 2024-02-13T03:17:39.042077+0000 mgr.y (mgr.14146) 139 : audit [DBG] from='client.24200 v1:172.21.15.161:0/317367632' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:17:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:40 smithi183 bash[20824]: audit 2024-02-13T03:17:40.044090+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:40 smithi183 bash[20824]: audit 2024-02-13T03:17:40.045581+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:40.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:40 smithi183 bash[20824]: audit 2024-02-13T03:17:40.046764+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:40.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:40 smithi183 bash[20824]: audit 2024-02-13T03:17:40.048101+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:40.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:40 smithi183 bash[20824]: audit 2024-02-13T03:17:40.056928+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:40.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:40 smithi183 bash[20824]: audit 2024-02-13T03:17:40.058766+0000 mon.a (mon.0) 343 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:40.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:40 smithi183 bash[20824]: audit 2024-02-13T03:17:40.060223+0000 mon.a (mon.0) 344 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:17:40.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:40 smithi183 bash[20824]: audit 2024-02-13T03:17:40.066120+0000 mon.a (mon.0) 345 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:40.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[19769]: cluster 2024-02-13T03:17:38.971316+0000 mgr.y (mgr.14146) 138 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:40.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[19769]: audit 2024-02-13T03:17:39.042077+0000 mgr.y (mgr.14146) 139 : audit [DBG] from='client.24200 v1:172.21.15.161:0/317367632' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:17:40.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[19769]: audit 2024-02-13T03:17:40.044090+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:40.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[19769]: audit 2024-02-13T03:17:40.045581+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:40.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[19769]: audit 2024-02-13T03:17:40.046764+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:40.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[19769]: audit 2024-02-13T03:17:40.048101+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:40.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[19769]: audit 2024-02-13T03:17:40.056928+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:40.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[19769]: audit 2024-02-13T03:17:40.058766+0000 mon.a (mon.0) 343 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:40.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[19769]: audit 2024-02-13T03:17:40.060223+0000 mon.a (mon.0) 344 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:17:40.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[19769]: audit 2024-02-13T03:17:40.066120+0000 mon.a (mon.0) 345 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[27879]: cluster 2024-02-13T03:17:38.971316+0000 mgr.y (mgr.14146) 138 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[27879]: audit 2024-02-13T03:17:39.042077+0000 mgr.y (mgr.14146) 139 : audit [DBG] from='client.24200 v1:172.21.15.161:0/317367632' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:17:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[27879]: audit 2024-02-13T03:17:40.044090+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[27879]: audit 2024-02-13T03:17:40.045581+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[27879]: audit 2024-02-13T03:17:40.046764+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[27879]: audit 2024-02-13T03:17:40.048101+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:17:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[27879]: audit 2024-02-13T03:17:40.056928+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[27879]: audit 2024-02-13T03:17:40.058766+0000 mon.a (mon.0) 343 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:40.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[27879]: audit 2024-02-13T03:17:40.060223+0000 mon.a (mon.0) 344 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:17:40.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:40 smithi161 bash[27879]: audit 2024-02-13T03:17:40.066120+0000 mon.a (mon.0) 345 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:17:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:41 smithi183 bash[20824]: cephadm 2024-02-13T03:17:40.035487+0000 mgr.y (mgr.14146) 140 : cephadm [INF] Detected new or changed devices on smithi161 2024-02-13T03:17:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:41 smithi183 bash[20824]: cephadm 2024-02-13T03:17:40.049061+0000 mgr.y (mgr.14146) 141 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 5431M 2024-02-13T03:17:41.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:41 smithi161 bash[19769]: cephadm 2024-02-13T03:17:40.035487+0000 mgr.y (mgr.14146) 140 : cephadm [INF] Detected new or changed devices on smithi161 2024-02-13T03:17:41.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:41 smithi161 bash[19769]: cephadm 2024-02-13T03:17:40.049061+0000 mgr.y (mgr.14146) 141 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 5431M 2024-02-13T03:17:41.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:41 smithi161 bash[27879]: cephadm 2024-02-13T03:17:40.035487+0000 mgr.y (mgr.14146) 140 : cephadm [INF] Detected new or changed devices on smithi161 2024-02-13T03:17:41.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:41 smithi161 bash[27879]: cephadm 2024-02-13T03:17:40.049061+0000 mgr.y (mgr.14146) 141 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 5431M 2024-02-13T03:17:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:42 smithi183 bash[20824]: cluster 2024-02-13T03:17:40.971908+0000 mgr.y (mgr.14146) 142 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:42.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:42 smithi161 bash[19769]: cluster 2024-02-13T03:17:40.971908+0000 mgr.y (mgr.14146) 142 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:42.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:42 smithi161 bash[27879]: cluster 2024-02-13T03:17:40.971908+0000 mgr.y (mgr.14146) 142 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:44 smithi183 bash[20824]: cluster 2024-02-13T03:17:42.972564+0000 mgr.y (mgr.14146) 143 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:44.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:44 smithi161 bash[19769]: cluster 2024-02-13T03:17:42.972564+0000 mgr.y (mgr.14146) 143 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:44.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:44 smithi161 bash[27879]: cluster 2024-02-13T03:17:42.972564+0000 mgr.y (mgr.14146) 143 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:46 smithi183 bash[20824]: cluster 2024-02-13T03:17:44.973121+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:46.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:46 smithi161 bash[27879]: cluster 2024-02-13T03:17:44.973121+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:46.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:46 smithi161 bash[19769]: cluster 2024-02-13T03:17:44.973121+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:47 smithi183 bash[20824]: audit 2024-02-13T03:17:47.066520+0000 mon.a (mon.0) 346 : audit [INF] from='client.? v1:172.21.15.161:0/1304226816' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "871101d1-6af5-47da-9d56-085de6869077"}]: dispatch 2024-02-13T03:17:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:47 smithi183 bash[20824]: audit 2024-02-13T03:17:47.074609+0000 mon.a (mon.0) 347 : audit [INF] from='client.? v1:172.21.15.161:0/1304226816' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "871101d1-6af5-47da-9d56-085de6869077"}]': finished 2024-02-13T03:17:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:47 smithi183 bash[20824]: cluster 2024-02-13T03:17:47.074671+0000 mon.a (mon.0) 348 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-13T03:17:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:47 smithi183 bash[20824]: audit 2024-02-13T03:17:47.074819+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:17:47.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:47 smithi161 bash[19769]: audit 2024-02-13T03:17:47.066520+0000 mon.a (mon.0) 346 : audit [INF] from='client.? v1:172.21.15.161:0/1304226816' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "871101d1-6af5-47da-9d56-085de6869077"}]: dispatch 2024-02-13T03:17:47.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:47 smithi161 bash[19769]: audit 2024-02-13T03:17:47.074609+0000 mon.a (mon.0) 347 : audit [INF] from='client.? v1:172.21.15.161:0/1304226816' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "871101d1-6af5-47da-9d56-085de6869077"}]': finished 2024-02-13T03:17:47.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:47 smithi161 bash[19769]: cluster 2024-02-13T03:17:47.074671+0000 mon.a (mon.0) 348 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-13T03:17:47.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:47 smithi161 bash[19769]: audit 2024-02-13T03:17:47.074819+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:17:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:47 smithi161 bash[27879]: audit 2024-02-13T03:17:47.066520+0000 mon.a (mon.0) 346 : audit [INF] from='client.? v1:172.21.15.161:0/1304226816' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "871101d1-6af5-47da-9d56-085de6869077"}]: dispatch 2024-02-13T03:17:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:47 smithi161 bash[27879]: audit 2024-02-13T03:17:47.074609+0000 mon.a (mon.0) 347 : audit [INF] from='client.? v1:172.21.15.161:0/1304226816' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "871101d1-6af5-47da-9d56-085de6869077"}]': finished 2024-02-13T03:17:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:47 smithi161 bash[27879]: cluster 2024-02-13T03:17:47.074671+0000 mon.a (mon.0) 348 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-13T03:17:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:47 smithi161 bash[27879]: audit 2024-02-13T03:17:47.074819+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:17:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:48 smithi183 bash[20824]: cluster 2024-02-13T03:17:46.973722+0000 mgr.y (mgr.14146) 145 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:48 smithi183 bash[20824]: audit 2024-02-13T03:17:47.686676+0000 mon.a (mon.0) 350 : audit [DBG] from='client.? v1:172.21.15.161:0/2939816757' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:17:48.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:48 smithi161 bash[19769]: cluster 2024-02-13T03:17:46.973722+0000 mgr.y (mgr.14146) 145 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:48.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:48 smithi161 bash[19769]: audit 2024-02-13T03:17:47.686676+0000 mon.a (mon.0) 350 : audit [DBG] from='client.? v1:172.21.15.161:0/2939816757' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:17:48.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:48 smithi161 bash[27879]: cluster 2024-02-13T03:17:46.973722+0000 mgr.y (mgr.14146) 145 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:48.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:48 smithi161 bash[27879]: audit 2024-02-13T03:17:47.686676+0000 mon.a (mon.0) 350 : audit [DBG] from='client.? v1:172.21.15.161:0/2939816757' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:17:50.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:50 smithi161 bash[19769]: cluster 2024-02-13T03:17:48.974346+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:50.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:50 smithi161 bash[27879]: cluster 2024-02-13T03:17:48.974346+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:50 smithi183 bash[20824]: cluster 2024-02-13T03:17:48.974346+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:52 smithi183 bash[20824]: cluster 2024-02-13T03:17:50.975047+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:52.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:52 smithi161 bash[19769]: cluster 2024-02-13T03:17:50.975047+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:52.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:52 smithi161 bash[27879]: cluster 2024-02-13T03:17:50.975047+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:54.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:54 smithi183 bash[20824]: cluster 2024-02-13T03:17:52.975725+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:54 smithi161 bash[27879]: cluster 2024-02-13T03:17:52.975725+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:54.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:54 smithi161 bash[19769]: cluster 2024-02-13T03:17:52.975725+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:56.629 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:56 smithi161 bash[19769]: cluster 2024-02-13T03:17:54.976280+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:56.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:56 smithi161 bash[27879]: cluster 2024-02-13T03:17:54.976280+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:56 smithi183 bash[20824]: cluster 2024-02-13T03:17:54.976280+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:58 smithi161 bash[27879]: cluster 2024-02-13T03:17:56.976960+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:58 smithi161 bash[19769]: cluster 2024-02-13T03:17:56.976960+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:58 smithi183 bash[20824]: cluster 2024-02-13T03:17:56.976960+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:17:59.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:59 smithi161 bash[19769]: audit 2024-02-13T03:17:59.030331+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-13T03:17:59.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:17:59 smithi161 bash[19769]: audit 2024-02-13T03:17:59.031667+0000 mon.a (mon.0) 352 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:59 smithi161 bash[27879]: audit 2024-02-13T03:17:59.030331+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-13T03:17:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:17:59 smithi161 bash[27879]: audit 2024-02-13T03:17:59.031667+0000 mon.a (mon.0) 352 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:17:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:59 smithi183 bash[20824]: audit 2024-02-13T03:17:59.030331+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-13T03:17:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:17:59 smithi183 bash[20824]: audit 2024-02-13T03:17:59.031667+0000 mon.a (mon.0) 352 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:00.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:00 smithi161 bash[27879]: cluster 2024-02-13T03:17:58.977536+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:00.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:00 smithi161 bash[27879]: cephadm 2024-02-13T03:17:59.032843+0000 mgr.y (mgr.14146) 152 : cephadm [INF] Deploying daemon osd.3 on smithi161 2024-02-13T03:18:00.348 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:00 smithi161 bash[19769]: cluster 2024-02-13T03:17:58.977536+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:00.349 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:00 smithi161 bash[19769]: cephadm 2024-02-13T03:17:59.032843+0000 mgr.y (mgr.14146) 152 : cephadm [INF] Deploying daemon osd.3 on smithi161 2024-02-13T03:18:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:00 smithi183 bash[20824]: cluster 2024-02-13T03:17:58.977536+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:00 smithi183 bash[20824]: cephadm 2024-02-13T03:17:59.032843+0000 mgr.y (mgr.14146) 152 : cephadm [INF] Deploying daemon osd.3 on smithi161 2024-02-13T03:18:02.545 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:02 smithi161 bash[27879]: cluster 2024-02-13T03:18:00.978115+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:02.545 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:02 smithi161 bash[19769]: cluster 2024-02-13T03:18:00.978115+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:02 smithi183 bash[20824]: cluster 2024-02-13T03:18:00.978115+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:04.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:03 smithi161 bash[19769]: audit 2024-02-13T03:18:02.885736+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:04.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:03 smithi161 bash[19769]: audit 2024-02-13T03:18:02.889292+0000 mon.a (mon.0) 354 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:18:04.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:03 smithi161 bash[19769]: cluster 2024-02-13T03:18:02.978648+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:04.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:03 smithi161 bash[27879]: audit 2024-02-13T03:18:02.885736+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:04.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:03 smithi161 bash[27879]: audit 2024-02-13T03:18:02.889292+0000 mon.a (mon.0) 354 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:18:04.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:03 smithi161 bash[27879]: cluster 2024-02-13T03:18:02.978648+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:03 smithi183 bash[20824]: audit 2024-02-13T03:18:02.885736+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:03 smithi183 bash[20824]: audit 2024-02-13T03:18:02.889292+0000 mon.a (mon.0) 354 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:18:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:03 smithi183 bash[20824]: cluster 2024-02-13T03:18:02.978648+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:06.321 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:06 smithi161 bash[19769]: cluster 2024-02-13T03:18:04.979151+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:06.321 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:06 smithi161 bash[27879]: cluster 2024-02-13T03:18:04.979151+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:06.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:06 smithi183 bash[20824]: cluster 2024-02-13T03:18:04.979151+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:08 smithi183 bash[20824]: cluster 2024-02-13T03:18:06.979747+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:08.559 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:08 smithi161 bash[19769]: cluster 2024-02-13T03:18:06.979747+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:08.559 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:08 smithi161 bash[27879]: cluster 2024-02-13T03:18:06.979747+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:09 smithi161 bash[19769]: audit 2024-02-13T03:18:08.389640+0000 mon.a (mon.0) 355 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:09 smithi161 bash[19769]: audit 2024-02-13T03:18:08.685536+0000 mon.a (mon.0) 356 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:09 smithi161 bash[19769]: audit 2024-02-13T03:18:08.686966+0000 mon.a (mon.0) 357 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:18:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:09 smithi161 bash[19769]: audit 2024-02-13T03:18:08.696000+0000 mon.a (mon.0) 358 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:09 smithi161 bash[19769]: audit 2024-02-13T03:18:09.280610+0000 mon.a (mon.0) 359 : audit [INF] from='osd.3 v1:172.21.15.161:6813/2788336304' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-13T03:18:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:09 smithi161 bash[27879]: audit 2024-02-13T03:18:08.389640+0000 mon.a (mon.0) 355 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:09 smithi161 bash[27879]: audit 2024-02-13T03:18:08.685536+0000 mon.a (mon.0) 356 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:09 smithi161 bash[27879]: audit 2024-02-13T03:18:08.686966+0000 mon.a (mon.0) 357 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:18:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:09 smithi161 bash[27879]: audit 2024-02-13T03:18:08.696000+0000 mon.a (mon.0) 358 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:09 smithi161 bash[27879]: audit 2024-02-13T03:18:09.280610+0000 mon.a (mon.0) 359 : audit [INF] from='osd.3 v1:172.21.15.161:6813/2788336304' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-13T03:18:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:09 smithi183 bash[20824]: audit 2024-02-13T03:18:08.389640+0000 mon.a (mon.0) 355 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:09 smithi183 bash[20824]: audit 2024-02-13T03:18:08.685536+0000 mon.a (mon.0) 356 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:09 smithi183 bash[20824]: audit 2024-02-13T03:18:08.686966+0000 mon.a (mon.0) 357 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:18:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:09 smithi183 bash[20824]: audit 2024-02-13T03:18:08.696000+0000 mon.a (mon.0) 358 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:09 smithi183 bash[20824]: audit 2024-02-13T03:18:09.280610+0000 mon.a (mon.0) 359 : audit [INF] from='osd.3 v1:172.21.15.161:6813/2788336304' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-13T03:18:10.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:10 smithi161 bash[19769]: cluster 2024-02-13T03:18:08.980261+0000 mgr.y (mgr.14146) 157 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:10.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:10 smithi161 bash[19769]: audit 2024-02-13T03:18:09.707300+0000 mon.a (mon.0) 360 : audit [INF] from='osd.3 v1:172.21.15.161:6813/2788336304' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-13T03:18:10.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:10 smithi161 bash[19769]: cluster 2024-02-13T03:18:09.707461+0000 mon.a (mon.0) 361 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-13T03:18:10.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:10 smithi161 bash[19769]: audit 2024-02-13T03:18:09.707740+0000 mon.a (mon.0) 362 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:18:10.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:10 smithi161 bash[19769]: audit 2024-02-13T03:18:09.708692+0000 mon.a (mon.0) 363 : audit [INF] from='osd.3 v1:172.21.15.161:6813/2788336304' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:18:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:10 smithi161 bash[27879]: cluster 2024-02-13T03:18:08.980261+0000 mgr.y (mgr.14146) 157 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:10 smithi161 bash[27879]: audit 2024-02-13T03:18:09.707300+0000 mon.a (mon.0) 360 : audit [INF] from='osd.3 v1:172.21.15.161:6813/2788336304' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-13T03:18:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:10 smithi161 bash[27879]: cluster 2024-02-13T03:18:09.707461+0000 mon.a (mon.0) 361 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-13T03:18:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:10 smithi161 bash[27879]: audit 2024-02-13T03:18:09.707740+0000 mon.a (mon.0) 362 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:18:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:10 smithi161 bash[27879]: audit 2024-02-13T03:18:09.708692+0000 mon.a (mon.0) 363 : audit [INF] from='osd.3 v1:172.21.15.161:6813/2788336304' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:18:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:10 smithi183 bash[20824]: cluster 2024-02-13T03:18:08.980261+0000 mgr.y (mgr.14146) 157 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:10 smithi183 bash[20824]: audit 2024-02-13T03:18:09.707300+0000 mon.a (mon.0) 360 : audit [INF] from='osd.3 v1:172.21.15.161:6813/2788336304' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-13T03:18:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:10 smithi183 bash[20824]: cluster 2024-02-13T03:18:09.707461+0000 mon.a (mon.0) 361 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-13T03:18:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:10 smithi183 bash[20824]: audit 2024-02-13T03:18:09.707740+0000 mon.a (mon.0) 362 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:18:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:10 smithi183 bash[20824]: audit 2024-02-13T03:18:09.708692+0000 mon.a (mon.0) 363 : audit [INF] from='osd.3 v1:172.21.15.161:6813/2788336304' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-02-13T03:18:11.414 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:18:11.407+0000 7f1d5a7fc700 1 -- 172.21.15.161:0/317367632 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 2713336589) 0x7f1d64060710 con 0x7f1d500623e0 2024-02-13T03:18:11.415 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 3 on host 'smithi161' 2024-02-13T03:18:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:18:11.411+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/317367632 >> v1:172.21.15.161:6800/1187535987 conn(0x7f1d500623e0 legacy=0x7f1d500648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:18:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:18:11.411+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/317367632 >> v1:172.21.15.161:6790/0 conn(0x7f1d640fad80 legacy=0x7f1d6410dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:18:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:18:11.411+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/317367632 shutdown_connections 2024-02-13T03:18:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:18:11.411+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/317367632 >> 172.21.15.161:0/317367632 conn(0x7f1d640f16b0 msgr2=0x7f1d640f6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:18:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:18:11.411+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/317367632 shutdown_connections 2024-02-13T03:18:11.417 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:18:11.411+0000 7f1d6b7c2700 1 -- 172.21.15.161:0/317367632 wait complete. 2024-02-13T03:18:11.724 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[19769]: audit 2024-02-13T03:18:10.709305+0000 mon.a (mon.0) 364 : audit [INF] from='osd.3 v1:172.21.15.161:6813/2788336304' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-02-13T03:18:11.724 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[19769]: cluster 2024-02-13T03:18:10.709565+0000 mon.a (mon.0) 365 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-13T03:18:11.724 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[19769]: audit 2024-02-13T03:18:10.709782+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:18:11.724 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[19769]: audit 2024-02-13T03:18:10.715284+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:18:11.724 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[19769]: cluster 2024-02-13T03:18:10.980903+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:11.725 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[19769]: audit 2024-02-13T03:18:11.412078+0000 mon.a (mon.0) 368 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:11.725 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[19769]: audit 2024-02-13T03:18:11.420446+0000 mon.a (mon.0) 369 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:18:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:11 smithi183 bash[20824]: audit 2024-02-13T03:18:10.709305+0000 mon.a (mon.0) 364 : audit [INF] from='osd.3 v1:172.21.15.161:6813/2788336304' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-02-13T03:18:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:11 smithi183 bash[20824]: cluster 2024-02-13T03:18:10.709565+0000 mon.a (mon.0) 365 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-13T03:18:11.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:11 smithi183 bash[20824]: audit 2024-02-13T03:18:10.709782+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:18:11.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:11 smithi183 bash[20824]: audit 2024-02-13T03:18:10.715284+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:18:11.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:11 smithi183 bash[20824]: cluster 2024-02-13T03:18:10.980903+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:11.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:11 smithi183 bash[20824]: audit 2024-02-13T03:18:11.412078+0000 mon.a (mon.0) 368 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:11.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:11 smithi183 bash[20824]: audit 2024-02-13T03:18:11.420446+0000 mon.a (mon.0) 369 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:18:12.015 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[27879]: audit 2024-02-13T03:18:10.709305+0000 mon.a (mon.0) 364 : audit [INF] from='osd.3 v1:172.21.15.161:6813/2788336304' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-02-13T03:18:12.015 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[27879]: cluster 2024-02-13T03:18:10.709565+0000 mon.a (mon.0) 365 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-13T03:18:12.015 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[27879]: audit 2024-02-13T03:18:10.709782+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:18:12.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[27879]: audit 2024-02-13T03:18:10.715284+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:18:12.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[27879]: cluster 2024-02-13T03:18:10.980903+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-13T03:18:12.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[27879]: audit 2024-02-13T03:18:11.412078+0000 mon.a (mon.0) 368 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:12.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:11 smithi161 bash[27879]: audit 2024-02-13T03:18:11.420446+0000 mon.a (mon.0) 369 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:18:12.017 DEBUG:teuthology.orchestra.run.smithi161:osd.3> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.3.service 2024-02-13T03:18:12.019 INFO:tasks.cephadm:Deploying osd.4 on smithi183 with /dev/vg_nvme/lv_4... 2024-02-13T03:18:12.020 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-13T03:18:12.257 INFO:journalctl@ceph.osd.3.smithi161.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:18:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:12 smithi183 bash[20824]: cluster 2024-02-13T03:18:10.307420+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:18:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:12 smithi183 bash[20824]: cluster 2024-02-13T03:18:10.307588+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:18:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:12 smithi183 bash[20824]: cluster 2024-02-13T03:18:11.713756+0000 mon.a (mon.0) 370 : cluster [INF] osd.3 v1:172.21.15.161:6813/2788336304 boot 2024-02-13T03:18:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:12 smithi183 bash[20824]: cluster 2024-02-13T03:18:11.713862+0000 mon.a (mon.0) 371 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-02-13T03:18:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:12 smithi183 bash[20824]: audit 2024-02-13T03:18:11.714788+0000 mon.a (mon.0) 372 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:18:13.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:12 smithi161 bash[19769]: cluster 2024-02-13T03:18:10.307420+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:18:13.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:12 smithi161 bash[19769]: cluster 2024-02-13T03:18:10.307588+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:18:13.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:12 smithi161 bash[19769]: cluster 2024-02-13T03:18:11.713756+0000 mon.a (mon.0) 370 : cluster [INF] osd.3 v1:172.21.15.161:6813/2788336304 boot 2024-02-13T03:18:13.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:12 smithi161 bash[19769]: cluster 2024-02-13T03:18:11.713862+0000 mon.a (mon.0) 371 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-02-13T03:18:13.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:12 smithi161 bash[19769]: audit 2024-02-13T03:18:11.714788+0000 mon.a (mon.0) 372 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:18:13.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:12 smithi161 bash[27879]: cluster 2024-02-13T03:18:10.307420+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:18:13.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:12 smithi161 bash[27879]: cluster 2024-02-13T03:18:10.307588+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:18:13.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:12 smithi161 bash[27879]: cluster 2024-02-13T03:18:11.713756+0000 mon.a (mon.0) 370 : cluster [INF] osd.3 v1:172.21.15.161:6813/2788336304 boot 2024-02-13T03:18:13.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:12 smithi161 bash[27879]: cluster 2024-02-13T03:18:11.713862+0000 mon.a (mon.0) 371 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-02-13T03:18:13.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:12 smithi161 bash[27879]: audit 2024-02-13T03:18:11.714788+0000 mon.a (mon.0) 372 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:18:14.457 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:14 smithi183 bash[20824]: cluster 2024-02-13T03:18:12.981485+0000 mgr.y (mgr.14146) 159 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:14.457 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:14 smithi183 bash[20824]: cluster 2024-02-13T03:18:13.187602+0000 mon.a (mon.0) 373 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-13T03:18:14.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:14 smithi161 bash[19769]: cluster 2024-02-13T03:18:12.981485+0000 mgr.y (mgr.14146) 159 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:14.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:14 smithi161 bash[19769]: cluster 2024-02-13T03:18:13.187602+0000 mon.a (mon.0) 373 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-13T03:18:14.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:14 smithi161 bash[27879]: cluster 2024-02-13T03:18:12.981485+0000 mgr.y (mgr.14146) 159 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:14.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:14 smithi161 bash[27879]: cluster 2024-02-13T03:18:13.187602+0000 mon.a (mon.0) 373 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-13T03:18:15.024 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-13T03:18:15.024 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-13T03:18:15.024 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-02-13T03:18:15.024 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-02-13T03:18:15.024 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0156367 s, 671 MB/s 2024-02-13T03:18:15.024 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-02-13T03:18:15.606 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-02-13T03:18:16.479 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:16 smithi183 bash[20824]: cluster 2024-02-13T03:18:14.982036+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:16.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:16 smithi161 bash[19769]: cluster 2024-02-13T03:18:14.982036+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:16.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:16 smithi161 bash[27879]: cluster 2024-02-13T03:18:14.982036+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:18.086 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2877511907 >> v1:172.21.15.183:6789/0 conn(0x7f8a380ff9d0 legacy=0x7f8a38101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:18:18.087 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2877511907 shutdown_connections 2024-02-13T03:18:18.087 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2877511907 >> 172.21.15.183:0/2877511907 conn(0x7f8a380f3050 msgr2=0x7f8a380f5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:18:18.087 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2877511907 shutdown_connections 2024-02-13T03:18:18.087 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2877511907 wait complete. 2024-02-13T03:18:18.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a3e27f700 1 Processor -- start 2024-02-13T03:18:18.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a3e27f700 1 -- start start 2024-02-13T03:18:18.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a3e27f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8a38108390 con 0x7f8a380fbda0 2024-02-13T03:18:18.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a3e27f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8a38113bf0 con 0x7f8a380f78d0 2024-02-13T03:18:18.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a3e27f700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8a38114dd0 con 0x7f8a380ff9d0 2024-02-13T03:18:18.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a37fff700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f8a380f78d0 0x7f8a381077c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:33904/0 (socket says 172.21.15.183:33904) 2024-02-13T03:18:18.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a37fff700 1 -- 172.21.15.183:0/2785546022 learned_addr learned my addr 172.21.15.183:0/2785546022 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:18:18.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976568432 0 0) 0x7f8a38113bf0 con 0x7f8a380f78d0 2024-02-13T03:18:18.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a24003630 con 0x7f8a380f78d0 2024-02-13T03:18:18.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880564559 0 0) 0x7f8a38108390 con 0x7f8a380fbda0 2024-02-13T03:18:18.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a38113bf0 con 0x7f8a380fbda0 2024-02-13T03:18:18.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222131900 0 0) 0x7f8a38114dd0 con 0x7f8a380ff9d0 2024-02-13T03:18:18.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a38108390 con 0x7f8a380ff9d0 2024-02-13T03:18:18.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 533071620 0 0) 0x7f8a24003630 con 0x7f8a380f78d0 2024-02-13T03:18:18.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8a38114dd0 con 0x7f8a380f78d0 2024-02-13T03:18:18.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f8a30001960 con 0x7f8a380f78d0 2024-02-13T03:18:18.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.085+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1743024737 0 0) 0x7f8a38113bf0 con 0x7f8a380fbda0 2024-02-13T03:18:18.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.089+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8a24003630 con 0x7f8a380fbda0 2024-02-13T03:18:18.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.089+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4215823639 0 0) 0x7f8a38114dd0 con 0x7f8a380f78d0 2024-02-13T03:18:18.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.089+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 >> v1:172.21.15.161:6790/0 conn(0x7f8a380ff9d0 legacy=0x7f8a381114c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:18:18.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.089+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 >> v1:172.21.15.161:6789/0 conn(0x7f8a380fbda0 legacy=0x7f8a3810dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:18:18.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.089+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a38115fb0 con 0x7f8a380f78d0 2024-02-13T03:18:18.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.089+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2785546022 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a38112c40 con 0x7f8a380f78d0 2024-02-13T03:18:18.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.089+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2785546022 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a381130b0 con 0x7f8a380f78d0 2024-02-13T03:18:18.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.089+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f8a30002420 con 0x7f8a380f78d0 2024-02-13T03:18:18.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.089+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f8a30005ae0 con 0x7f8a380f78d0 2024-02-13T03:18:18.092 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.089+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2785546022 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a18004f50 con 0x7f8a380f78d0 2024-02-13T03:18:18.092 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.089+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 1965425884 0 0) 0x7f8a30001f80 con 0x7f8a380f78d0 2024-02-13T03:18:18.092 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.089+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(26..26 src has 1..26) v4 ==== 2776+0+0 (unknown 2106174822 0 0) 0x7f8a300777f0 con 0x7f8a380f78d0 2024-02-13T03:18:18.099 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.097+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f8a300485d0 con 0x7f8a380f78d0 2024-02-13T03:18:18.290 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:18.289+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2785546022 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a18000bc0 con 0x7f8a24061ec0 2024-02-13T03:18:18.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:18 smithi183 bash[20824]: cluster 2024-02-13T03:18:16.982515+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:18.651 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:18 smithi161 bash[19769]: cluster 2024-02-13T03:18:16.982515+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:18.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:18 smithi161 bash[27879]: cluster 2024-02-13T03:18:16.982515+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:19.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:19 smithi183 bash[20824]: audit 2024-02-13T03:18:18.291196+0000 mgr.y (mgr.14146) 162 : audit [DBG] from='client.24226 v1:172.21.15.183:0/2785546022' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:18:19.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:19 smithi183 bash[20824]: audit 2024-02-13T03:18:18.298654+0000 mon.a (mon.0) 374 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:18:19.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:19 smithi183 bash[20824]: audit 2024-02-13T03:18:18.302427+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:18:19.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:19 smithi183 bash[20824]: audit 2024-02-13T03:18:18.303394+0000 mon.a (mon.0) 376 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:19.609 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:19 smithi161 bash[19769]: audit 2024-02-13T03:18:18.291196+0000 mgr.y (mgr.14146) 162 : audit [DBG] from='client.24226 v1:172.21.15.183:0/2785546022' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:18:19.609 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:19 smithi161 bash[19769]: audit 2024-02-13T03:18:18.298654+0000 mon.a (mon.0) 374 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:18:19.609 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:19 smithi161 bash[19769]: audit 2024-02-13T03:18:18.302427+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:18:19.609 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:19 smithi161 bash[19769]: audit 2024-02-13T03:18:18.303394+0000 mon.a (mon.0) 376 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:19.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:19 smithi161 bash[27879]: audit 2024-02-13T03:18:18.291196+0000 mgr.y (mgr.14146) 162 : audit [DBG] from='client.24226 v1:172.21.15.183:0/2785546022' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:18:19.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:19 smithi161 bash[27879]: audit 2024-02-13T03:18:18.298654+0000 mon.a (mon.0) 374 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:18:19.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:19 smithi161 bash[27879]: audit 2024-02-13T03:18:18.302427+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:18:19.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:19 smithi161 bash[27879]: audit 2024-02-13T03:18:18.303394+0000 mon.a (mon.0) 376 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:20.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:20 smithi183 bash[20824]: cluster 2024-02-13T03:18:18.983028+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:20.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:20 smithi161 bash[19769]: cluster 2024-02-13T03:18:18.983028+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:20.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:20 smithi161 bash[27879]: cluster 2024-02-13T03:18:18.983028+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:21.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[27879]: cephadm 2024-02-13T03:18:20.409205+0000 mgr.y (mgr.14146) 164 : cephadm [INF] Detected new or changed devices on smithi161 2024-02-13T03:18:21.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[27879]: audit 2024-02-13T03:18:20.418376+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:21.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[27879]: audit 2024-02-13T03:18:20.420094+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:18:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[27879]: audit 2024-02-13T03:18:20.421324+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:18:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[27879]: audit 2024-02-13T03:18:20.422517+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:18:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[27879]: audit 2024-02-13T03:18:20.423759+0000 mon.a (mon.0) 381 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:18:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[27879]: cephadm 2024-02-13T03:18:20.424699+0000 mgr.y (mgr.14146) 165 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 4073M 2024-02-13T03:18:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[27879]: audit 2024-02-13T03:18:20.431490+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[27879]: audit 2024-02-13T03:18:20.432780+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[27879]: audit 2024-02-13T03:18:20.433756+0000 mon.a (mon.0) 384 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:18:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[27879]: audit 2024-02-13T03:18:20.440343+0000 mon.a (mon.0) 385 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:21.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[19769]: cephadm 2024-02-13T03:18:20.409205+0000 mgr.y (mgr.14146) 164 : cephadm [INF] Detected new or changed devices on smithi161 2024-02-13T03:18:21.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[19769]: audit 2024-02-13T03:18:20.418376+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:21.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[19769]: audit 2024-02-13T03:18:20.420094+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:18:21.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[19769]: audit 2024-02-13T03:18:20.421324+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:18:21.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[19769]: audit 2024-02-13T03:18:20.422517+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:18:21.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[19769]: audit 2024-02-13T03:18:20.423759+0000 mon.a (mon.0) 381 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:18:21.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[19769]: cephadm 2024-02-13T03:18:20.424699+0000 mgr.y (mgr.14146) 165 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 4073M 2024-02-13T03:18:21.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[19769]: audit 2024-02-13T03:18:20.431490+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:21.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[19769]: audit 2024-02-13T03:18:20.432780+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:21.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[19769]: audit 2024-02-13T03:18:20.433756+0000 mon.a (mon.0) 384 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:18:21.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:21 smithi161 bash[19769]: audit 2024-02-13T03:18:20.440343+0000 mon.a (mon.0) 385 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:21 smithi183 bash[20824]: cephadm 2024-02-13T03:18:20.409205+0000 mgr.y (mgr.14146) 164 : cephadm [INF] Detected new or changed devices on smithi161 2024-02-13T03:18:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:21 smithi183 bash[20824]: audit 2024-02-13T03:18:20.418376+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:21 smithi183 bash[20824]: audit 2024-02-13T03:18:20.420094+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:18:21.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:21 smithi183 bash[20824]: audit 2024-02-13T03:18:20.421324+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:18:21.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:21 smithi183 bash[20824]: audit 2024-02-13T03:18:20.422517+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:18:21.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:21 smithi183 bash[20824]: audit 2024-02-13T03:18:20.423759+0000 mon.a (mon.0) 381 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:18:21.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:21 smithi183 bash[20824]: cephadm 2024-02-13T03:18:20.424699+0000 mgr.y (mgr.14146) 165 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 4073M 2024-02-13T03:18:21.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:21 smithi183 bash[20824]: audit 2024-02-13T03:18:20.431490+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:21.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:21 smithi183 bash[20824]: audit 2024-02-13T03:18:20.432780+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:21.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:21 smithi183 bash[20824]: audit 2024-02-13T03:18:20.433756+0000 mon.a (mon.0) 384 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:18:21.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:21 smithi183 bash[20824]: audit 2024-02-13T03:18:20.440343+0000 mon.a (mon.0) 385 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:22 smithi183 bash[20824]: cluster 2024-02-13T03:18:20.983564+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:22.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:22 smithi161 bash[27879]: cluster 2024-02-13T03:18:20.983564+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:22.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:22 smithi161 bash[19769]: cluster 2024-02-13T03:18:20.983564+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:24 smithi183 bash[20824]: cluster 2024-02-13T03:18:22.984101+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:24.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:24 smithi161 bash[19769]: cluster 2024-02-13T03:18:22.984101+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:24.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:24 smithi161 bash[27879]: cluster 2024-02-13T03:18:22.984101+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:26.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:26 smithi183 bash[20824]: cluster 2024-02-13T03:18:24.984591+0000 mgr.y (mgr.14146) 168 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:26.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:26 smithi183 bash[20824]: audit 2024-02-13T03:18:26.347393+0000 mon.a (mon.0) 386 : audit [INF] from='client.24232 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02e5c987-89d4-490f-b6cb-be43e286cb30"}]: dispatch 2024-02-13T03:18:26.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:26 smithi183 bash[20824]: audit 2024-02-13T03:18:26.348822+0000 mon.b (mon.1) 3 : audit [INF] from='client.? v1:172.21.15.183:0/750153676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02e5c987-89d4-490f-b6cb-be43e286cb30"}]: dispatch 2024-02-13T03:18:26.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:26 smithi183 bash[20824]: audit 2024-02-13T03:18:26.356002+0000 mon.a (mon.0) 387 : audit [INF] from='client.24232 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02e5c987-89d4-490f-b6cb-be43e286cb30"}]': finished 2024-02-13T03:18:26.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:26 smithi183 bash[20824]: cluster 2024-02-13T03:18:26.356203+0000 mon.a (mon.0) 388 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-02-13T03:18:26.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:26 smithi183 bash[20824]: audit 2024-02-13T03:18:26.356410+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:26.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:26 smithi161 bash[19769]: cluster 2024-02-13T03:18:24.984591+0000 mgr.y (mgr.14146) 168 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:26.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:26 smithi161 bash[19769]: audit 2024-02-13T03:18:26.347393+0000 mon.a (mon.0) 386 : audit [INF] from='client.24232 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02e5c987-89d4-490f-b6cb-be43e286cb30"}]: dispatch 2024-02-13T03:18:26.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:26 smithi161 bash[19769]: audit 2024-02-13T03:18:26.348822+0000 mon.b (mon.1) 3 : audit [INF] from='client.? v1:172.21.15.183:0/750153676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02e5c987-89d4-490f-b6cb-be43e286cb30"}]: dispatch 2024-02-13T03:18:26.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:26 smithi161 bash[19769]: audit 2024-02-13T03:18:26.356002+0000 mon.a (mon.0) 387 : audit [INF] from='client.24232 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02e5c987-89d4-490f-b6cb-be43e286cb30"}]': finished 2024-02-13T03:18:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:26 smithi161 bash[19769]: cluster 2024-02-13T03:18:26.356203+0000 mon.a (mon.0) 388 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-02-13T03:18:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:26 smithi161 bash[19769]: audit 2024-02-13T03:18:26.356410+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:26 smithi161 bash[27879]: cluster 2024-02-13T03:18:24.984591+0000 mgr.y (mgr.14146) 168 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:26 smithi161 bash[27879]: audit 2024-02-13T03:18:26.347393+0000 mon.a (mon.0) 386 : audit [INF] from='client.24232 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02e5c987-89d4-490f-b6cb-be43e286cb30"}]: dispatch 2024-02-13T03:18:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:26 smithi161 bash[27879]: audit 2024-02-13T03:18:26.348822+0000 mon.b (mon.1) 3 : audit [INF] from='client.? v1:172.21.15.183:0/750153676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02e5c987-89d4-490f-b6cb-be43e286cb30"}]: dispatch 2024-02-13T03:18:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:26 smithi161 bash[27879]: audit 2024-02-13T03:18:26.356002+0000 mon.a (mon.0) 387 : audit [INF] from='client.24232 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02e5c987-89d4-490f-b6cb-be43e286cb30"}]': finished 2024-02-13T03:18:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:26 smithi161 bash[27879]: cluster 2024-02-13T03:18:26.356203+0000 mon.a (mon.0) 388 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-02-13T03:18:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:26 smithi161 bash[27879]: audit 2024-02-13T03:18:26.356410+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:27 smithi183 bash[20824]: audit 2024-02-13T03:18:26.939999+0000 mon.b (mon.1) 4 : audit [DBG] from='client.? v1:172.21.15.183:0/2365942311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:18:27.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:27 smithi161 bash[19769]: audit 2024-02-13T03:18:26.939999+0000 mon.b (mon.1) 4 : audit [DBG] from='client.? v1:172.21.15.183:0/2365942311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:18:27.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:27 smithi161 bash[27879]: audit 2024-02-13T03:18:26.939999+0000 mon.b (mon.1) 4 : audit [DBG] from='client.? v1:172.21.15.183:0/2365942311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:18:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:28 smithi183 bash[20824]: cluster 2024-02-13T03:18:26.985137+0000 mgr.y (mgr.14146) 169 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:28.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:28 smithi161 bash[19769]: cluster 2024-02-13T03:18:26.985137+0000 mgr.y (mgr.14146) 169 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:28.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:28 smithi161 bash[27879]: cluster 2024-02-13T03:18:26.985137+0000 mgr.y (mgr.14146) 169 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:30 smithi183 bash[20824]: cluster 2024-02-13T03:18:28.985778+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:30.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:30 smithi161 bash[27879]: cluster 2024-02-13T03:18:28.985778+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:30.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:30 smithi161 bash[19769]: cluster 2024-02-13T03:18:28.985778+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:32 smithi183 bash[20824]: cluster 2024-02-13T03:18:30.986442+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:32 smithi161 bash[27879]: cluster 2024-02-13T03:18:30.986442+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:32.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:32 smithi161 bash[19769]: cluster 2024-02-13T03:18:30.986442+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:34 smithi183 bash[20824]: cluster 2024-02-13T03:18:32.987050+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:34.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:34 smithi161 bash[19769]: cluster 2024-02-13T03:18:32.987050+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:34 smithi161 bash[27879]: cluster 2024-02-13T03:18:32.987050+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:36.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:36 smithi161 bash[19769]: cluster 2024-02-13T03:18:34.987621+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:36 smithi161 bash[27879]: cluster 2024-02-13T03:18:34.987621+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:36 smithi183 bash[20824]: cluster 2024-02-13T03:18:34.987621+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:38 smithi161 bash[19769]: cluster 2024-02-13T03:18:36.988178+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:38 smithi161 bash[19769]: audit 2024-02-13T03:18:38.133521+0000 mon.a (mon.0) 390 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-13T03:18:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:38 smithi161 bash[19769]: audit 2024-02-13T03:18:38.135010+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:38 smithi161 bash[27879]: cluster 2024-02-13T03:18:36.988178+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:38 smithi161 bash[27879]: audit 2024-02-13T03:18:38.133521+0000 mon.a (mon.0) 390 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-13T03:18:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:38 smithi161 bash[27879]: audit 2024-02-13T03:18:38.135010+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:38.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:38 smithi183 bash[20824]: cluster 2024-02-13T03:18:36.988178+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:38.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:38 smithi183 bash[20824]: audit 2024-02-13T03:18:38.133521+0000 mon.a (mon.0) 390 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-13T03:18:38.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:38 smithi183 bash[20824]: audit 2024-02-13T03:18:38.135010+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:39.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:39 smithi161 bash[19769]: cephadm 2024-02-13T03:18:38.136349+0000 mgr.y (mgr.14146) 175 : cephadm [INF] Deploying daemon osd.4 on smithi183 2024-02-13T03:18:39.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:39 smithi161 bash[27879]: cephadm 2024-02-13T03:18:38.136349+0000 mgr.y (mgr.14146) 175 : cephadm [INF] Deploying daemon osd.4 on smithi183 2024-02-13T03:18:39.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:39 smithi183 bash[20824]: cephadm 2024-02-13T03:18:38.136349+0000 mgr.y (mgr.14146) 175 : cephadm [INF] Deploying daemon osd.4 on smithi183 2024-02-13T03:18:40.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:40 smithi161 bash[19769]: cluster 2024-02-13T03:18:38.988706+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:40.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:40 smithi161 bash[27879]: cluster 2024-02-13T03:18:38.988706+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:40.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:40 smithi183 bash[20824]: cluster 2024-02-13T03:18:38.988706+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:42.812 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:42 smithi183 bash[20824]: cluster 2024-02-13T03:18:40.989251+0000 mgr.y (mgr.14146) 177 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:42.812 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:42 smithi183 bash[20824]: audit 2024-02-13T03:18:41.904360+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:42.812 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:42 smithi183 bash[20824]: audit 2024-02-13T03:18:41.914538+0000 mon.a (mon.0) 393 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:18:42.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:42 smithi161 bash[27879]: cluster 2024-02-13T03:18:40.989251+0000 mgr.y (mgr.14146) 177 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:42.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:42 smithi161 bash[27879]: audit 2024-02-13T03:18:41.904360+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:42 smithi161 bash[27879]: audit 2024-02-13T03:18:41.914538+0000 mon.a (mon.0) 393 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:18:42.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:42 smithi161 bash[19769]: cluster 2024-02-13T03:18:40.989251+0000 mgr.y (mgr.14146) 177 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:42.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:42 smithi161 bash[19769]: audit 2024-02-13T03:18:41.904360+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:42.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:42 smithi161 bash[19769]: audit 2024-02-13T03:18:41.914538+0000 mon.a (mon.0) 393 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:18:44.790 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:44 smithi183 bash[20824]: cluster 2024-02-13T03:18:42.989835+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:44.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:44 smithi161 bash[19769]: cluster 2024-02-13T03:18:42.989835+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:44.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:44 smithi161 bash[27879]: cluster 2024-02-13T03:18:42.989835+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:46.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:46 smithi161 bash[19769]: cluster 2024-02-13T03:18:44.990372+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:46.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:46 smithi161 bash[27879]: cluster 2024-02-13T03:18:44.990372+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:46.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:46 smithi183 bash[20824]: cluster 2024-02-13T03:18:44.990372+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:48.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:48 smithi161 bash[19769]: cluster 2024-02-13T03:18:46.990951+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:48.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:48 smithi161 bash[19769]: audit 2024-02-13T03:18:47.171149+0000 mon.a (mon.0) 394 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:48.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:48 smithi161 bash[19769]: audit 2024-02-13T03:18:47.450591+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:48 smithi161 bash[19769]: audit 2024-02-13T03:18:47.452495+0000 mon.a (mon.0) 396 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:18:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:48 smithi161 bash[19769]: audit 2024-02-13T03:18:47.462224+0000 mon.a (mon.0) 397 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:48 smithi161 bash[27879]: cluster 2024-02-13T03:18:46.990951+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:48 smithi161 bash[27879]: audit 2024-02-13T03:18:47.171149+0000 mon.a (mon.0) 394 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:48 smithi161 bash[27879]: audit 2024-02-13T03:18:47.450591+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:48 smithi161 bash[27879]: audit 2024-02-13T03:18:47.452495+0000 mon.a (mon.0) 396 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:18:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:48 smithi161 bash[27879]: audit 2024-02-13T03:18:47.462224+0000 mon.a (mon.0) 397 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:48 smithi183 bash[20824]: cluster 2024-02-13T03:18:46.990951+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:48 smithi183 bash[20824]: audit 2024-02-13T03:18:47.171149+0000 mon.a (mon.0) 394 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:48.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:48 smithi183 bash[20824]: audit 2024-02-13T03:18:47.450591+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:48.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:48 smithi183 bash[20824]: audit 2024-02-13T03:18:47.452495+0000 mon.a (mon.0) 396 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:18:48.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:48 smithi183 bash[20824]: audit 2024-02-13T03:18:47.462224+0000 mon.a (mon.0) 397 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:49.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:49 smithi161 bash[19769]: audit 2024-02-13T03:18:48.364733+0000 mon.a (mon.0) 398 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-13T03:18:49.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:49 smithi161 bash[19769]: audit 2024-02-13T03:18:48.366297+0000 mon.b (mon.1) 5 : audit [INF] from='osd.4 v1:172.21.15.183:6800/384629816' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-13T03:18:49.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:49 smithi161 bash[27879]: audit 2024-02-13T03:18:48.364733+0000 mon.a (mon.0) 398 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-13T03:18:49.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:49 smithi161 bash[27879]: audit 2024-02-13T03:18:48.366297+0000 mon.b (mon.1) 5 : audit [INF] from='osd.4 v1:172.21.15.183:6800/384629816' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-13T03:18:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:49 smithi183 bash[20824]: audit 2024-02-13T03:18:48.364733+0000 mon.a (mon.0) 398 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-13T03:18:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:49 smithi183 bash[20824]: audit 2024-02-13T03:18:48.366297+0000 mon.b (mon.1) 5 : audit [INF] from='osd.4 v1:172.21.15.183:6800/384629816' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-13T03:18:50.208 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:50.204+0000 7f8a35775700 1 -- 172.21.15.183:0/2785546022 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1297213289) 0x7f8a18000bc0 con 0x7f8a24061ec0 2024-02-13T03:18:50.209 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 4 on host 'smithi183' 2024-02-13T03:18:50.211 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:50.208+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2785546022 >> v1:172.21.15.161:6800/1187535987 conn(0x7f8a24061ec0 legacy=0x7f8a24064380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:18:50.212 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:50.208+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2785546022 >> v1:172.21.15.183:6789/0 conn(0x7f8a380f78d0 legacy=0x7f8a381077c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:18:50.212 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:50.208+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2785546022 shutdown_connections 2024-02-13T03:18:50.212 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:50.208+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2785546022 >> 172.21.15.183:0/2785546022 conn(0x7f8a380f3050 msgr2=0x7f8a380f3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:18:50.212 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:50.208+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2785546022 shutdown_connections 2024-02-13T03:18:50.212 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:50.208+0000 7f8a3e27f700 1 -- 172.21.15.183:0/2785546022 wait complete. 2024-02-13T03:18:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:50 smithi183 bash[20824]: cluster 2024-02-13T03:18:48.991496+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:50 smithi183 bash[20824]: audit 2024-02-13T03:18:49.183272+0000 mon.a (mon.0) 399 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-13T03:18:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:50 smithi183 bash[20824]: cluster 2024-02-13T03:18:49.183374+0000 mon.a (mon.0) 400 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-13T03:18:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:50 smithi183 bash[20824]: audit 2024-02-13T03:18:49.183609+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:50 smithi183 bash[20824]: audit 2024-02-13T03:18:49.185309+0000 mon.a (mon.0) 402 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:18:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:50 smithi183 bash[20824]: audit 2024-02-13T03:18:49.186933+0000 mon.b (mon.1) 6 : audit [INF] from='osd.4 v1:172.21.15.183:6800/384629816' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:18:50.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:50 smithi161 bash[19769]: cluster 2024-02-13T03:18:48.991496+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:50.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:50 smithi161 bash[19769]: audit 2024-02-13T03:18:49.183272+0000 mon.a (mon.0) 399 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-13T03:18:50.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:50 smithi161 bash[19769]: cluster 2024-02-13T03:18:49.183374+0000 mon.a (mon.0) 400 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-13T03:18:50.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:50 smithi161 bash[19769]: audit 2024-02-13T03:18:49.183609+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:50.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:50 smithi161 bash[19769]: audit 2024-02-13T03:18:49.185309+0000 mon.a (mon.0) 402 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:18:50.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:50 smithi161 bash[19769]: audit 2024-02-13T03:18:49.186933+0000 mon.b (mon.1) 6 : audit [INF] from='osd.4 v1:172.21.15.183:6800/384629816' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:18:50.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:50 smithi161 bash[27879]: cluster 2024-02-13T03:18:48.991496+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:50.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:50 smithi161 bash[27879]: audit 2024-02-13T03:18:49.183272+0000 mon.a (mon.0) 399 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-13T03:18:50.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:50 smithi161 bash[27879]: cluster 2024-02-13T03:18:49.183374+0000 mon.a (mon.0) 400 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-13T03:18:50.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:50 smithi161 bash[27879]: audit 2024-02-13T03:18:49.183609+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:50.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:50 smithi161 bash[27879]: audit 2024-02-13T03:18:49.185309+0000 mon.a (mon.0) 402 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:18:50.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:50 smithi161 bash[27879]: audit 2024-02-13T03:18:49.186933+0000 mon.b (mon.1) 6 : audit [INF] from='osd.4 v1:172.21.15.183:6800/384629816' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:18:50.806 DEBUG:teuthology.orchestra.run.smithi183:osd.4> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.4.service 2024-02-13T03:18:50.809 INFO:tasks.cephadm:Deploying osd.5 on smithi183 with /dev/vg_nvme/lv_3... 2024-02-13T03:18:50.809 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-13T03:18:50.943 INFO:journalctl@ceph.osd.4.smithi183.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:18:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:51 smithi183 bash[20824]: audit 2024-02-13T03:18:50.188345+0000 mon.a (mon.0) 403 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-02-13T03:18:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:51 smithi183 bash[20824]: cluster 2024-02-13T03:18:50.188497+0000 mon.a (mon.0) 404 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-13T03:18:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:51 smithi183 bash[20824]: audit 2024-02-13T03:18:50.189071+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:51 smithi183 bash[20824]: audit 2024-02-13T03:18:50.195800+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:51 smithi183 bash[20824]: audit 2024-02-13T03:18:50.206534+0000 mon.a (mon.0) 407 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:51 smithi183 bash[20824]: audit 2024-02-13T03:18:50.237556+0000 mon.a (mon.0) 408 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:18:51.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:51 smithi161 bash[19769]: audit 2024-02-13T03:18:50.188345+0000 mon.a (mon.0) 403 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-02-13T03:18:51.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:51 smithi161 bash[19769]: cluster 2024-02-13T03:18:50.188497+0000 mon.a (mon.0) 404 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-13T03:18:51.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:51 smithi161 bash[19769]: audit 2024-02-13T03:18:50.189071+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:51.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:51 smithi161 bash[19769]: audit 2024-02-13T03:18:50.195800+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:51.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:51 smithi161 bash[19769]: audit 2024-02-13T03:18:50.206534+0000 mon.a (mon.0) 407 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:51.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:51 smithi161 bash[19769]: audit 2024-02-13T03:18:50.237556+0000 mon.a (mon.0) 408 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:18:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:51 smithi161 bash[27879]: audit 2024-02-13T03:18:50.188345+0000 mon.a (mon.0) 403 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-02-13T03:18:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:51 smithi161 bash[27879]: cluster 2024-02-13T03:18:50.188497+0000 mon.a (mon.0) 404 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-13T03:18:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:51 smithi161 bash[27879]: audit 2024-02-13T03:18:50.189071+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:51 smithi161 bash[27879]: audit 2024-02-13T03:18:50.195800+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:51 smithi161 bash[27879]: audit 2024-02-13T03:18:50.206534+0000 mon.a (mon.0) 407 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:18:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:51 smithi161 bash[27879]: audit 2024-02-13T03:18:50.237556+0000 mon.a (mon.0) 408 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:18:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:52 smithi183 bash[20824]: cluster 2024-02-13T03:18:49.391670+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:18:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:52 smithi183 bash[20824]: cluster 2024-02-13T03:18:49.391769+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:18:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:52 smithi183 bash[20824]: cluster 2024-02-13T03:18:50.992063+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v163: 1 pgs: 1 peering; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:52 smithi183 bash[20824]: cluster 2024-02-13T03:18:51.191486+0000 mon.a (mon.0) 409 : cluster [INF] osd.4 v1:172.21.15.183:6800/384629816 boot 2024-02-13T03:18:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:52 smithi183 bash[20824]: cluster 2024-02-13T03:18:51.191652+0000 mon.a (mon.0) 410 : cluster [DBG] osdmap e30: 5 total, 5 up, 5 in 2024-02-13T03:18:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:52 smithi183 bash[20824]: audit 2024-02-13T03:18:51.191921+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:52 smithi183 bash[20824]: cluster 2024-02-13T03:18:52.192517+0000 mon.a (mon.0) 412 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-02-13T03:18:52.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[19769]: cluster 2024-02-13T03:18:49.391670+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:18:52.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[19769]: cluster 2024-02-13T03:18:49.391769+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:18:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[19769]: cluster 2024-02-13T03:18:50.992063+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v163: 1 pgs: 1 peering; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[19769]: cluster 2024-02-13T03:18:51.191486+0000 mon.a (mon.0) 409 : cluster [INF] osd.4 v1:172.21.15.183:6800/384629816 boot 2024-02-13T03:18:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[19769]: cluster 2024-02-13T03:18:51.191652+0000 mon.a (mon.0) 410 : cluster [DBG] osdmap e30: 5 total, 5 up, 5 in 2024-02-13T03:18:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[19769]: audit 2024-02-13T03:18:51.191921+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[19769]: cluster 2024-02-13T03:18:52.192517+0000 mon.a (mon.0) 412 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-02-13T03:18:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[27879]: cluster 2024-02-13T03:18:49.391670+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:18:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[27879]: cluster 2024-02-13T03:18:49.391769+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:18:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[27879]: cluster 2024-02-13T03:18:50.992063+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v163: 1 pgs: 1 peering; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-13T03:18:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[27879]: cluster 2024-02-13T03:18:51.191486+0000 mon.a (mon.0) 409 : cluster [INF] osd.4 v1:172.21.15.183:6800/384629816 boot 2024-02-13T03:18:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[27879]: cluster 2024-02-13T03:18:51.191652+0000 mon.a (mon.0) 410 : cluster [DBG] osdmap e30: 5 total, 5 up, 5 in 2024-02-13T03:18:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[27879]: audit 2024-02-13T03:18:51.191921+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:18:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:52 smithi161 bash[27879]: cluster 2024-02-13T03:18:52.192517+0000 mon.a (mon.0) 412 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-02-13T03:18:53.883 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-13T03:18:53.884 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-13T03:18:53.884 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-02-13T03:18:53.884 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-02-13T03:18:53.884 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0166497 s, 630 MB/s 2024-02-13T03:18:53.884 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-02-13T03:18:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:54 smithi183 bash[20824]: cluster 2024-02-13T03:18:52.992600+0000 mgr.y (mgr.14146) 183 : cluster [DBG] pgmap v166: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:18:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:54 smithi183 bash[20824]: cluster 2024-02-13T03:18:53.195340+0000 mon.a (mon.0) 413 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-13T03:18:54.451 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-02-13T03:18:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:54 smithi161 bash[27879]: cluster 2024-02-13T03:18:52.992600+0000 mgr.y (mgr.14146) 183 : cluster [DBG] pgmap v166: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:18:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:54 smithi161 bash[27879]: cluster 2024-02-13T03:18:53.195340+0000 mon.a (mon.0) 413 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-13T03:18:54.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:54 smithi161 bash[19769]: cluster 2024-02-13T03:18:52.992600+0000 mgr.y (mgr.14146) 183 : cluster [DBG] pgmap v166: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:18:54.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:54 smithi161 bash[19769]: cluster 2024-02-13T03:18:53.195340+0000 mon.a (mon.0) 413 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-13T03:18:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:56 smithi183 bash[20824]: cluster 2024-02-13T03:18:54.993133+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v168: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:18:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:56 smithi161 bash[27879]: cluster 2024-02-13T03:18:54.993133+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v168: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:18:56.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:56 smithi161 bash[19769]: cluster 2024-02-13T03:18:54.993133+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v168: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:18:58.266 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37c5373700 1 -- 172.21.15.183:0/4291497277 >> v1:172.21.15.183:6789/0 conn(0x7f37c00fac90 legacy=0x7f37c00fd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:18:58.266 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37c5373700 1 -- 172.21.15.183:0/4291497277 shutdown_connections 2024-02-13T03:18:58.266 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37c5373700 1 -- 172.21.15.183:0/4291497277 >> 172.21.15.183:0/4291497277 conn(0x7f37c00f1f00 msgr2=0x7f37c00f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:18:58.266 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37c5373700 1 -- 172.21.15.183:0/4291497277 shutdown_connections 2024-02-13T03:18:58.266 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37c5373700 1 -- 172.21.15.183:0/4291497277 wait complete. 2024-02-13T03:18:58.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37c5373700 1 Processor -- start 2024-02-13T03:18:58.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37c5373700 1 -- start start 2024-02-13T03:18:58.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37c5373700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f37c0112c80 con 0x7f37c00fac90 2024-02-13T03:18:58.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37c5373700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f37c0113e20 con 0x7f37c00fe8c0 2024-02-13T03:18:58.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37c5373700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f37c0114fc0 con 0x7f37c00f67c0 2024-02-13T03:18:58.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bf7fe700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f37c00fe8c0 0x7f37c0111560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:57974/0 (socket says 172.21.15.183:57974) 2024-02-13T03:18:58.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bf7fe700 1 -- 172.21.15.183:0/3001012852 learned_addr learned my addr 172.21.15.183:0/3001012852 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:18:58.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175694956 0 0) 0x7f37c0113e20 con 0x7f37c00fe8c0 2024-02-13T03:18:58.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37ac003030 con 0x7f37c00fe8c0 2024-02-13T03:18:58.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590779092 0 0) 0x7f37c0112c80 con 0x7f37c00fac90 2024-02-13T03:18:58.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37c0113e20 con 0x7f37c00fac90 2024-02-13T03:18:58.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076576511 0 0) 0x7f37c0114fc0 con 0x7f37c00f67c0 2024-02-13T03:18:58.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37c0112c80 con 0x7f37c00f67c0 2024-02-13T03:18:58.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3916086762 0 0) 0x7f37ac003030 con 0x7f37c00fe8c0 2024-02-13T03:18:58.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f37c0114fc0 con 0x7f37c00fe8c0 2024-02-13T03:18:58.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f37b8001c40 con 0x7f37c00fe8c0 2024-02-13T03:18:58.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 472659751 0 0) 0x7f37c0114fc0 con 0x7f37c00fe8c0 2024-02-13T03:18:58.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 >> v1:172.21.15.161:6790/0 conn(0x7f37c00f67c0 legacy=0x7f37c010a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:18:58.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 >> v1:172.21.15.161:6789/0 conn(0x7f37c00fac90 legacy=0x7f37c010de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:18:58.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c0116160 con 0x7f37c00fe8c0 2024-02-13T03:18:58.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37c5373700 1 -- 172.21.15.183:0/3001012852 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c0112e70 con 0x7f37c00fe8c0 2024-02-13T03:18:58.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.264+0000 7f37c5373700 1 -- 172.21.15.183:0/3001012852 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c0113360 con 0x7f37c00fe8c0 2024-02-13T03:18:58.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.268+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f37b8001fd0 con 0x7f37c00fe8c0 2024-02-13T03:18:58.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.268+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f37b8005f40 con 0x7f37c00fe8c0 2024-02-13T03:18:58.271 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.268+0000 7f37c5373700 1 -- 172.21.15.183:0/3001012852 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a0004f50 con 0x7f37c00fe8c0 2024-02-13T03:18:58.271 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.268+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 1965425884 0 0) 0x7f37b8018e10 con 0x7f37c00fe8c0 2024-02-13T03:18:58.272 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.268+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(32..32 src has 1..32) v4 ==== 3178+0+0 (unknown 2452839674 0 0) 0x7f37b8077cb0 con 0x7f37c00fe8c0 2024-02-13T03:18:58.277 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.276+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f37b80488f0 con 0x7f37c00fe8c0 2024-02-13T03:18:58.466 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:18:58.464+0000 7f37c5373700 1 -- 172.21.15.183:0/3001012852 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f37a0000bc0 con 0x7f37ac061e50 2024-02-13T03:18:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:58 smithi183 bash[20824]: cluster 2024-02-13T03:18:56.993714+0000 mgr.y (mgr.14146) 185 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:18:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:58 smithi161 bash[27879]: cluster 2024-02-13T03:18:56.993714+0000 mgr.y (mgr.14146) 185 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:18:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:58 smithi161 bash[19769]: cluster 2024-02-13T03:18:56.993714+0000 mgr.y (mgr.14146) 185 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:18:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:59 smithi183 bash[20824]: audit 2024-02-13T03:18:58.467000+0000 mgr.y (mgr.14146) 186 : audit [DBG] from='client.24253 v1:172.21.15.183:0/3001012852' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:18:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:59 smithi183 bash[20824]: audit 2024-02-13T03:18:58.473796+0000 mon.a (mon.0) 414 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:18:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:59 smithi183 bash[20824]: audit 2024-02-13T03:18:58.480443+0000 mon.a (mon.0) 415 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:18:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:18:59 smithi183 bash[20824]: audit 2024-02-13T03:18:58.481970+0000 mon.a (mon.0) 416 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:59.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:59 smithi161 bash[19769]: audit 2024-02-13T03:18:58.467000+0000 mgr.y (mgr.14146) 186 : audit [DBG] from='client.24253 v1:172.21.15.183:0/3001012852' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:18:59.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:59 smithi161 bash[19769]: audit 2024-02-13T03:18:58.473796+0000 mon.a (mon.0) 414 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:18:59.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:59 smithi161 bash[19769]: audit 2024-02-13T03:18:58.480443+0000 mon.a (mon.0) 415 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:18:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:18:59 smithi161 bash[19769]: audit 2024-02-13T03:18:58.481970+0000 mon.a (mon.0) 416 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:18:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:59 smithi161 bash[27879]: audit 2024-02-13T03:18:58.467000+0000 mgr.y (mgr.14146) 186 : audit [DBG] from='client.24253 v1:172.21.15.183:0/3001012852' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:18:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:59 smithi161 bash[27879]: audit 2024-02-13T03:18:58.473796+0000 mon.a (mon.0) 414 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:18:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:59 smithi161 bash[27879]: audit 2024-02-13T03:18:58.480443+0000 mon.a (mon.0) 415 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:18:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:18:59 smithi161 bash[27879]: audit 2024-02-13T03:18:58.481970+0000 mon.a (mon.0) 416 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:00 smithi183 bash[20824]: cluster 2024-02-13T03:18:58.994280+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:00 smithi183 bash[20824]: audit 2024-02-13T03:18:59.626651+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:00 smithi183 bash[20824]: audit 2024-02-13T03:18:59.628116+0000 mon.a (mon.0) 418 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:19:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:00 smithi183 bash[20824]: audit 2024-02-13T03:18:59.639071+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:00 smithi183 bash[20824]: audit 2024-02-13T03:18:59.640769+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:00 smithi183 bash[20824]: audit 2024-02-13T03:18:59.642204+0000 mon.a (mon.0) 421 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:19:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:00 smithi183 bash[20824]: audit 2024-02-13T03:18:59.652872+0000 mon.a (mon.0) 422 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:00.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[19769]: cluster 2024-02-13T03:18:58.994280+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:00.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[19769]: audit 2024-02-13T03:18:59.626651+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:00.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[19769]: audit 2024-02-13T03:18:59.628116+0000 mon.a (mon.0) 418 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:19:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[19769]: audit 2024-02-13T03:18:59.639071+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[19769]: audit 2024-02-13T03:18:59.640769+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[19769]: audit 2024-02-13T03:18:59.642204+0000 mon.a (mon.0) 421 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:19:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[19769]: audit 2024-02-13T03:18:59.652872+0000 mon.a (mon.0) 422 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[27879]: cluster 2024-02-13T03:18:58.994280+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[27879]: audit 2024-02-13T03:18:59.626651+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[27879]: audit 2024-02-13T03:18:59.628116+0000 mon.a (mon.0) 418 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:19:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[27879]: audit 2024-02-13T03:18:59.639071+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[27879]: audit 2024-02-13T03:18:59.640769+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[27879]: audit 2024-02-13T03:18:59.642204+0000 mon.a (mon.0) 421 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:19:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:00 smithi161 bash[27879]: audit 2024-02-13T03:18:59.652872+0000 mon.a (mon.0) 422 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:01 smithi183 bash[20824]: cephadm 2024-02-13T03:18:59.615411+0000 mgr.y (mgr.14146) 188 : cephadm [INF] Detected new or changed devices on smithi183 2024-02-13T03:19:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:01 smithi183 bash[20824]: cephadm 2024-02-13T03:18:59.629123+0000 mgr.y (mgr.14146) 189 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 17318M 2024-02-13T03:19:01.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:01 smithi161 bash[19769]: cephadm 2024-02-13T03:18:59.615411+0000 mgr.y (mgr.14146) 188 : cephadm [INF] Detected new or changed devices on smithi183 2024-02-13T03:19:01.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:01 smithi161 bash[19769]: cephadm 2024-02-13T03:18:59.629123+0000 mgr.y (mgr.14146) 189 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 17318M 2024-02-13T03:19:01.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:01 smithi161 bash[27879]: cephadm 2024-02-13T03:18:59.615411+0000 mgr.y (mgr.14146) 188 : cephadm [INF] Detected new or changed devices on smithi183 2024-02-13T03:19:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:01 smithi161 bash[27879]: cephadm 2024-02-13T03:18:59.629123+0000 mgr.y (mgr.14146) 189 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 17318M 2024-02-13T03:19:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:02 smithi183 bash[20824]: cluster 2024-02-13T03:19:00.994822+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:02 smithi161 bash[19769]: cluster 2024-02-13T03:19:00.994822+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:02.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:02 smithi161 bash[27879]: cluster 2024-02-13T03:19:00.994822+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:04.344 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:04 smithi183 bash[20824]: cluster 2024-02-13T03:19:02.995337+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:04.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:04 smithi161 bash[19769]: cluster 2024-02-13T03:19:02.995337+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:04.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:04 smithi161 bash[27879]: cluster 2024-02-13T03:19:02.995337+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:06 smithi161 bash[19769]: cluster 2024-02-13T03:19:04.995865+0000 mgr.y (mgr.14146) 192 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:06 smithi161 bash[27879]: cluster 2024-02-13T03:19:04.995865+0000 mgr.y (mgr.14146) 192 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:06.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:06 smithi183 bash[20824]: cluster 2024-02-13T03:19:04.995865+0000 mgr.y (mgr.14146) 192 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:07 smithi161 bash[19769]: audit 2024-02-13T03:19:06.418513+0000 mon.a (mon.0) 423 : audit [INF] from='client.24259 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68fc8d37-f86b-4f2e-8cfe-5e47161d27c5"}]: dispatch 2024-02-13T03:19:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:07 smithi161 bash[19769]: audit 2024-02-13T03:19:06.420073+0000 mon.b (mon.1) 7 : audit [INF] from='client.? v1:172.21.15.183:0/1082645149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68fc8d37-f86b-4f2e-8cfe-5e47161d27c5"}]: dispatch 2024-02-13T03:19:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:07 smithi161 bash[19769]: audit 2024-02-13T03:19:06.427159+0000 mon.a (mon.0) 424 : audit [INF] from='client.24259 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68fc8d37-f86b-4f2e-8cfe-5e47161d27c5"}]': finished 2024-02-13T03:19:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:07 smithi161 bash[19769]: cluster 2024-02-13T03:19:06.427441+0000 mon.a (mon.0) 425 : cluster [DBG] osdmap e33: 6 total, 5 up, 6 in 2024-02-13T03:19:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:07 smithi161 bash[19769]: audit 2024-02-13T03:19:06.427742+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:07 smithi161 bash[19769]: audit 2024-02-13T03:19:07.048864+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? v1:172.21.15.183:0/3120333005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:19:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:07 smithi161 bash[27879]: audit 2024-02-13T03:19:06.418513+0000 mon.a (mon.0) 423 : audit [INF] from='client.24259 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68fc8d37-f86b-4f2e-8cfe-5e47161d27c5"}]: dispatch 2024-02-13T03:19:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:07 smithi161 bash[27879]: audit 2024-02-13T03:19:06.420073+0000 mon.b (mon.1) 7 : audit [INF] from='client.? v1:172.21.15.183:0/1082645149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68fc8d37-f86b-4f2e-8cfe-5e47161d27c5"}]: dispatch 2024-02-13T03:19:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:07 smithi161 bash[27879]: audit 2024-02-13T03:19:06.427159+0000 mon.a (mon.0) 424 : audit [INF] from='client.24259 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68fc8d37-f86b-4f2e-8cfe-5e47161d27c5"}]': finished 2024-02-13T03:19:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:07 smithi161 bash[27879]: cluster 2024-02-13T03:19:06.427441+0000 mon.a (mon.0) 425 : cluster [DBG] osdmap e33: 6 total, 5 up, 6 in 2024-02-13T03:19:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:07 smithi161 bash[27879]: audit 2024-02-13T03:19:06.427742+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:07 smithi161 bash[27879]: audit 2024-02-13T03:19:07.048864+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? v1:172.21.15.183:0/3120333005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:19:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:07 smithi183 bash[20824]: audit 2024-02-13T03:19:06.418513+0000 mon.a (mon.0) 423 : audit [INF] from='client.24259 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68fc8d37-f86b-4f2e-8cfe-5e47161d27c5"}]: dispatch 2024-02-13T03:19:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:07 smithi183 bash[20824]: audit 2024-02-13T03:19:06.420073+0000 mon.b (mon.1) 7 : audit [INF] from='client.? v1:172.21.15.183:0/1082645149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68fc8d37-f86b-4f2e-8cfe-5e47161d27c5"}]: dispatch 2024-02-13T03:19:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:07 smithi183 bash[20824]: audit 2024-02-13T03:19:06.427159+0000 mon.a (mon.0) 424 : audit [INF] from='client.24259 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68fc8d37-f86b-4f2e-8cfe-5e47161d27c5"}]': finished 2024-02-13T03:19:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:07 smithi183 bash[20824]: cluster 2024-02-13T03:19:06.427441+0000 mon.a (mon.0) 425 : cluster [DBG] osdmap e33: 6 total, 5 up, 6 in 2024-02-13T03:19:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:07 smithi183 bash[20824]: audit 2024-02-13T03:19:06.427742+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:07 smithi183 bash[20824]: audit 2024-02-13T03:19:07.048864+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? v1:172.21.15.183:0/3120333005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:19:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:08 smithi183 bash[20824]: cluster 2024-02-13T03:19:06.996405+0000 mgr.y (mgr.14146) 193 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:08.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:08 smithi161 bash[19769]: cluster 2024-02-13T03:19:06.996405+0000 mgr.y (mgr.14146) 193 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:08.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:08 smithi161 bash[27879]: cluster 2024-02-13T03:19:06.996405+0000 mgr.y (mgr.14146) 193 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:10 smithi183 bash[20824]: cluster 2024-02-13T03:19:08.996907+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:10.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:10 smithi161 bash[19769]: cluster 2024-02-13T03:19:08.996907+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:10.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:10 smithi161 bash[27879]: cluster 2024-02-13T03:19:08.996907+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:12 smithi183 bash[20824]: cluster 2024-02-13T03:19:10.997415+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:12.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:12 smithi161 bash[19769]: cluster 2024-02-13T03:19:10.997415+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:12.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:12 smithi161 bash[27879]: cluster 2024-02-13T03:19:10.997415+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:14 smithi183 bash[20824]: cluster 2024-02-13T03:19:12.997984+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:14.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:14 smithi161 bash[19769]: cluster 2024-02-13T03:19:12.997984+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:14.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:14 smithi161 bash[27879]: cluster 2024-02-13T03:19:12.997984+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:16 smithi183 bash[20824]: cluster 2024-02-13T03:19:14.998531+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:16.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:16 smithi161 bash[19769]: cluster 2024-02-13T03:19:14.998531+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:16.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:16 smithi161 bash[27879]: cluster 2024-02-13T03:19:14.998531+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:18 smithi183 bash[20824]: cluster 2024-02-13T03:19:16.999094+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:18 smithi183 bash[20824]: audit 2024-02-13T03:19:18.262085+0000 mon.a (mon.0) 427 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-13T03:19:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:18 smithi183 bash[20824]: audit 2024-02-13T03:19:18.263670+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:18.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:18 smithi161 bash[27879]: cluster 2024-02-13T03:19:16.999094+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:18.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:18 smithi161 bash[27879]: audit 2024-02-13T03:19:18.262085+0000 mon.a (mon.0) 427 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-13T03:19:18.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:18 smithi161 bash[27879]: audit 2024-02-13T03:19:18.263670+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:18.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:18 smithi161 bash[19769]: cluster 2024-02-13T03:19:16.999094+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:18.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:18 smithi161 bash[19769]: audit 2024-02-13T03:19:18.262085+0000 mon.a (mon.0) 427 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-13T03:19:18.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:18 smithi161 bash[19769]: audit 2024-02-13T03:19:18.263670+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:19.612 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:19 smithi183 bash[20824]: cephadm 2024-02-13T03:19:18.265045+0000 mgr.y (mgr.14146) 199 : cephadm [INF] Deploying daemon osd.5 on smithi183 2024-02-13T03:19:19.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:19 smithi161 bash[19769]: cephadm 2024-02-13T03:19:18.265045+0000 mgr.y (mgr.14146) 199 : cephadm [INF] Deploying daemon osd.5 on smithi183 2024-02-13T03:19:19.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:19 smithi161 bash[27879]: cephadm 2024-02-13T03:19:18.265045+0000 mgr.y (mgr.14146) 199 : cephadm [INF] Deploying daemon osd.5 on smithi183 2024-02-13T03:19:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:20 smithi183 bash[20824]: cluster 2024-02-13T03:19:18.999675+0000 mgr.y (mgr.14146) 200 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:20.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:20 smithi161 bash[19769]: cluster 2024-02-13T03:19:18.999675+0000 mgr.y (mgr.14146) 200 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:20 smithi161 bash[27879]: cluster 2024-02-13T03:19:18.999675+0000 mgr.y (mgr.14146) 200 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:22 smithi183 bash[20824]: cluster 2024-02-13T03:19:21.000224+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:22 smithi183 bash[20824]: audit 2024-02-13T03:19:22.134761+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:19:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:22 smithi183 bash[20824]: audit 2024-02-13T03:19:22.141622+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:22.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:22 smithi161 bash[19769]: cluster 2024-02-13T03:19:21.000224+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:22.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:22 smithi161 bash[19769]: audit 2024-02-13T03:19:22.134761+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:19:22.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:22 smithi161 bash[19769]: audit 2024-02-13T03:19:22.141622+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:22.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:22 smithi161 bash[27879]: cluster 2024-02-13T03:19:21.000224+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:22.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:22 smithi161 bash[27879]: audit 2024-02-13T03:19:22.134761+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:19:22.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:22 smithi161 bash[27879]: audit 2024-02-13T03:19:22.141622+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:24.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:24 smithi183 bash[20824]: cluster 2024-02-13T03:19:23.000778+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:24.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:24 smithi161 bash[19769]: cluster 2024-02-13T03:19:23.000778+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:24.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:24 smithi161 bash[27879]: cluster 2024-02-13T03:19:23.000778+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:26.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:26 smithi183 bash[20824]: cluster 2024-02-13T03:19:25.001296+0000 mgr.y (mgr.14146) 203 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:26.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:26 smithi161 bash[19769]: cluster 2024-02-13T03:19:25.001296+0000 mgr.y (mgr.14146) 203 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:26 smithi161 bash[27879]: cluster 2024-02-13T03:19:25.001296+0000 mgr.y (mgr.14146) 203 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:28 smithi183 bash[20824]: cluster 2024-02-13T03:19:27.001897+0000 mgr.y (mgr.14146) 204 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:28 smithi183 bash[20824]: audit 2024-02-13T03:19:28.474954+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:28 smithi183 bash[20824]: audit 2024-02-13T03:19:28.476772+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:28 smithi183 bash[20824]: audit 2024-02-13T03:19:28.478683+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:19:28.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:28 smithi161 bash[27879]: cluster 2024-02-13T03:19:27.001897+0000 mgr.y (mgr.14146) 204 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:28.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:28 smithi161 bash[27879]: audit 2024-02-13T03:19:28.474954+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:28.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:28 smithi161 bash[27879]: audit 2024-02-13T03:19:28.476772+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:28.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:28 smithi161 bash[27879]: audit 2024-02-13T03:19:28.478683+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:19:28.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:28 smithi161 bash[19769]: cluster 2024-02-13T03:19:27.001897+0000 mgr.y (mgr.14146) 204 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:28.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:28 smithi161 bash[19769]: audit 2024-02-13T03:19:28.474954+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:28.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:28 smithi161 bash[19769]: audit 2024-02-13T03:19:28.476772+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:28.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:28 smithi161 bash[19769]: audit 2024-02-13T03:19:28.478683+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:19:29.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:29 smithi161 bash[19769]: audit 2024-02-13T03:19:28.490883+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:29.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:29 smithi161 bash[19769]: audit 2024-02-13T03:19:28.899839+0000 mon.a (mon.0) 435 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-13T03:19:29.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:29 smithi161 bash[19769]: audit 2024-02-13T03:19:28.901770+0000 mon.b (mon.1) 9 : audit [INF] from='osd.5 v1:172.21.15.183:6804/2958722347' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-13T03:19:29.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:29 smithi161 bash[27879]: audit 2024-02-13T03:19:28.490883+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:29.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:29 smithi161 bash[27879]: audit 2024-02-13T03:19:28.899839+0000 mon.a (mon.0) 435 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-13T03:19:29.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:29 smithi161 bash[27879]: audit 2024-02-13T03:19:28.901770+0000 mon.b (mon.1) 9 : audit [INF] from='osd.5 v1:172.21.15.183:6804/2958722347' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-13T03:19:29.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:29 smithi183 bash[20824]: audit 2024-02-13T03:19:28.490883+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:29.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:29 smithi183 bash[20824]: audit 2024-02-13T03:19:28.899839+0000 mon.a (mon.0) 435 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-13T03:19:29.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:29 smithi183 bash[20824]: audit 2024-02-13T03:19:28.901770+0000 mon.b (mon.1) 9 : audit [INF] from='osd.5 v1:172.21.15.183:6804/2958722347' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-13T03:19:30.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:30 smithi161 bash[19769]: cluster 2024-02-13T03:19:29.002462+0000 mgr.y (mgr.14146) 205 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:30.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:30 smithi161 bash[19769]: audit 2024-02-13T03:19:29.502656+0000 mon.a (mon.0) 436 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-13T03:19:30.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:30 smithi161 bash[19769]: cluster 2024-02-13T03:19:29.502744+0000 mon.a (mon.0) 437 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-02-13T03:19:30.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:30 smithi161 bash[19769]: audit 2024-02-13T03:19:29.502888+0000 mon.a (mon.0) 438 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:30.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:30 smithi161 bash[19769]: audit 2024-02-13T03:19:29.507395+0000 mon.a (mon.0) 439 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:19:30.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:30 smithi161 bash[19769]: audit 2024-02-13T03:19:29.509380+0000 mon.b (mon.1) 10 : audit [INF] from='osd.5 v1:172.21.15.183:6804/2958722347' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:19:30.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:30 smithi161 bash[27879]: cluster 2024-02-13T03:19:29.002462+0000 mgr.y (mgr.14146) 205 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:30.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:30 smithi161 bash[27879]: audit 2024-02-13T03:19:29.502656+0000 mon.a (mon.0) 436 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-13T03:19:30.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:30 smithi161 bash[27879]: cluster 2024-02-13T03:19:29.502744+0000 mon.a (mon.0) 437 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-02-13T03:19:30.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:30 smithi161 bash[27879]: audit 2024-02-13T03:19:29.502888+0000 mon.a (mon.0) 438 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:30.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:30 smithi161 bash[27879]: audit 2024-02-13T03:19:29.507395+0000 mon.a (mon.0) 439 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:19:30.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:30 smithi161 bash[27879]: audit 2024-02-13T03:19:29.509380+0000 mon.b (mon.1) 10 : audit [INF] from='osd.5 v1:172.21.15.183:6804/2958722347' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:19:30.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:30 smithi183 bash[20824]: cluster 2024-02-13T03:19:29.002462+0000 mgr.y (mgr.14146) 205 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:30.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:30 smithi183 bash[20824]: audit 2024-02-13T03:19:29.502656+0000 mon.a (mon.0) 436 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-13T03:19:30.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:30 smithi183 bash[20824]: cluster 2024-02-13T03:19:29.502744+0000 mon.a (mon.0) 437 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-02-13T03:19:30.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:30 smithi183 bash[20824]: audit 2024-02-13T03:19:29.502888+0000 mon.a (mon.0) 438 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:30.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:30 smithi183 bash[20824]: audit 2024-02-13T03:19:29.507395+0000 mon.a (mon.0) 439 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:19:30.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:30 smithi183 bash[20824]: audit 2024-02-13T03:19:29.509380+0000 mon.b (mon.1) 10 : audit [INF] from='osd.5 v1:172.21.15.183:6804/2958722347' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:19:31.601 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:31.595+0000 7f37bcff9700 1 -- 172.21.15.183:0/3001012852 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 289671645) 0x7f37a0000bc0 con 0x7f37ac061e50 2024-02-13T03:19:31.601 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 5 on host 'smithi183' 2024-02-13T03:19:31.604 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:31.599+0000 7f37c5373700 1 -- 172.21.15.183:0/3001012852 >> v1:172.21.15.161:6800/1187535987 conn(0x7f37ac061e50 legacy=0x7f37ac064310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:19:31.604 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:31.599+0000 7f37c5373700 1 -- 172.21.15.183:0/3001012852 >> v1:172.21.15.183:6789/0 conn(0x7f37c00fe8c0 legacy=0x7f37c0111560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:19:31.604 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:31.599+0000 7f37c5373700 1 -- 172.21.15.183:0/3001012852 shutdown_connections 2024-02-13T03:19:31.604 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:31.599+0000 7f37c5373700 1 -- 172.21.15.183:0/3001012852 >> 172.21.15.183:0/3001012852 conn(0x7f37c00f1f00 msgr2=0x7f37c00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:19:31.605 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:31.599+0000 7f37c5373700 1 -- 172.21.15.183:0/3001012852 shutdown_connections 2024-02-13T03:19:31.605 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:31.599+0000 7f37c5373700 1 -- 172.21.15.183:0/3001012852 wait complete. 2024-02-13T03:19:31.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:31 smithi161 bash[19769]: audit 2024-02-13T03:19:30.504344+0000 mon.a (mon.0) 440 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-02-13T03:19:31.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:31 smithi161 bash[19769]: cluster 2024-02-13T03:19:30.504511+0000 mon.a (mon.0) 441 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-13T03:19:31.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:31 smithi161 bash[19769]: audit 2024-02-13T03:19:30.505076+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:31.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:31 smithi161 bash[19769]: audit 2024-02-13T03:19:30.511474+0000 mon.a (mon.0) 443 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:31 smithi161 bash[27879]: audit 2024-02-13T03:19:30.504344+0000 mon.a (mon.0) 440 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-02-13T03:19:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:31 smithi161 bash[27879]: cluster 2024-02-13T03:19:30.504511+0000 mon.a (mon.0) 441 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-13T03:19:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:31 smithi161 bash[27879]: audit 2024-02-13T03:19:30.505076+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:31 smithi161 bash[27879]: audit 2024-02-13T03:19:30.511474+0000 mon.a (mon.0) 443 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:31 smithi183 bash[20824]: audit 2024-02-13T03:19:30.504344+0000 mon.a (mon.0) 440 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-02-13T03:19:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:31 smithi183 bash[20824]: cluster 2024-02-13T03:19:30.504511+0000 mon.a (mon.0) 441 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-13T03:19:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:31 smithi183 bash[20824]: audit 2024-02-13T03:19:30.505076+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:31 smithi183 bash[20824]: audit 2024-02-13T03:19:30.511474+0000 mon.a (mon.0) 443 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:32.220 DEBUG:teuthology.orchestra.run.smithi183:osd.5> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.5.service 2024-02-13T03:19:32.222 INFO:tasks.cephadm:Deploying osd.6 on smithi183 with /dev/vg_nvme/lv_2... 2024-02-13T03:19:32.223 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-13T03:19:32.343 INFO:journalctl@ceph.osd.5.smithi183.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:19:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[19769]: cluster 2024-02-13T03:19:29.925782+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:19:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[19769]: cluster 2024-02-13T03:19:29.925893+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:19:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[19769]: cluster 2024-02-13T03:19:31.003011+0000 mgr.y (mgr.14146) 206 : cluster [DBG] pgmap v189: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[19769]: cluster 2024-02-13T03:19:31.508124+0000 mon.a (mon.0) 444 : cluster [INF] osd.5 v1:172.21.15.183:6804/2958722347 boot 2024-02-13T03:19:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[19769]: cluster 2024-02-13T03:19:31.508246+0000 mon.a (mon.0) 445 : cluster [DBG] osdmap e36: 6 total, 6 up, 6 in 2024-02-13T03:19:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[19769]: audit 2024-02-13T03:19:31.508453+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[19769]: audit 2024-02-13T03:19:31.598338+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[19769]: audit 2024-02-13T03:19:31.617337+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:19:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[19769]: cluster 2024-02-13T03:19:32.212219+0000 mon.a (mon.0) 449 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-02-13T03:19:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[27879]: cluster 2024-02-13T03:19:29.925782+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:19:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[27879]: cluster 2024-02-13T03:19:29.925893+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:19:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[27879]: cluster 2024-02-13T03:19:31.003011+0000 mgr.y (mgr.14146) 206 : cluster [DBG] pgmap v189: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[27879]: cluster 2024-02-13T03:19:31.508124+0000 mon.a (mon.0) 444 : cluster [INF] osd.5 v1:172.21.15.183:6804/2958722347 boot 2024-02-13T03:19:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[27879]: cluster 2024-02-13T03:19:31.508246+0000 mon.a (mon.0) 445 : cluster [DBG] osdmap e36: 6 total, 6 up, 6 in 2024-02-13T03:19:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[27879]: audit 2024-02-13T03:19:31.508453+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[27879]: audit 2024-02-13T03:19:31.598338+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[27879]: audit 2024-02-13T03:19:31.617337+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:19:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:32 smithi161 bash[27879]: cluster 2024-02-13T03:19:32.212219+0000 mon.a (mon.0) 449 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-02-13T03:19:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:32 smithi183 bash[20824]: cluster 2024-02-13T03:19:29.925782+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:19:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:32 smithi183 bash[20824]: cluster 2024-02-13T03:19:29.925893+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:19:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:32 smithi183 bash[20824]: cluster 2024-02-13T03:19:31.003011+0000 mgr.y (mgr.14146) 206 : cluster [DBG] pgmap v189: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-13T03:19:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:32 smithi183 bash[20824]: cluster 2024-02-13T03:19:31.508124+0000 mon.a (mon.0) 444 : cluster [INF] osd.5 v1:172.21.15.183:6804/2958722347 boot 2024-02-13T03:19:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:32 smithi183 bash[20824]: cluster 2024-02-13T03:19:31.508246+0000 mon.a (mon.0) 445 : cluster [DBG] osdmap e36: 6 total, 6 up, 6 in 2024-02-13T03:19:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:32 smithi183 bash[20824]: audit 2024-02-13T03:19:31.508453+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:19:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:32 smithi183 bash[20824]: audit 2024-02-13T03:19:31.598338+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:32.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:32 smithi183 bash[20824]: audit 2024-02-13T03:19:31.617337+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:19:32.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:32 smithi183 bash[20824]: cluster 2024-02-13T03:19:32.212219+0000 mon.a (mon.0) 449 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-02-13T03:19:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:34 smithi183 bash[20824]: cluster 2024-02-13T03:19:33.003599+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v192: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:34 smithi183 bash[20824]: cluster 2024-02-13T03:19:33.221969+0000 mon.a (mon.0) 450 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-02-13T03:19:34.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:34 smithi161 bash[19769]: cluster 2024-02-13T03:19:33.003599+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v192: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:34.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:34 smithi161 bash[19769]: cluster 2024-02-13T03:19:33.221969+0000 mon.a (mon.0) 450 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-02-13T03:19:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:34 smithi161 bash[27879]: cluster 2024-02-13T03:19:33.003599+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v192: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:34 smithi161 bash[27879]: cluster 2024-02-13T03:19:33.221969+0000 mon.a (mon.0) 450 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-02-13T03:19:35.535 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-13T03:19:35.535 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-13T03:19:35.536 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-02-13T03:19:35.536 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-02-13T03:19:35.536 INFO:teuthology.orchestra.run.smithi183.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0140951 s, 744 MB/s 2024-02-13T03:19:35.536 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-02-13T03:19:36.084 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-02-13T03:19:36.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:36 smithi183 bash[20824]: cluster 2024-02-13T03:19:35.004136+0000 mgr.y (mgr.14146) 208 : cluster [DBG] pgmap v194: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:36.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:36 smithi161 bash[19769]: cluster 2024-02-13T03:19:35.004136+0000 mgr.y (mgr.14146) 208 : cluster [DBG] pgmap v194: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:36.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:36 smithi161 bash[27879]: cluster 2024-02-13T03:19:35.004136+0000 mgr.y (mgr.14146) 208 : cluster [DBG] pgmap v194: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:38.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:38 smithi183 bash[20824]: cluster 2024-02-13T03:19:37.004721+0000 mgr.y (mgr.14146) 209 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:38.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:38 smithi161 bash[19769]: cluster 2024-02-13T03:19:37.004721+0000 mgr.y (mgr.14146) 209 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:38 smithi161 bash[27879]: cluster 2024-02-13T03:19:37.004721+0000 mgr.y (mgr.14146) 209 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:40.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.023+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/3656979282 >> v1:172.21.15.183:6789/0 conn(0x7fbd840fe050 legacy=0x7fbd84100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:19:40.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.023+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/3656979282 shutdown_connections 2024-02-13T03:19:40.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.023+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/3656979282 >> 172.21.15.183:0/3656979282 conn(0x7fbd840f16b0 msgr2=0x7fbd840f3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:19:40.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.023+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/3656979282 shutdown_connections 2024-02-13T03:19:40.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.023+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/3656979282 wait complete. 2024-02-13T03:19:40.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.023+0000 7fbd8a2d0700 1 Processor -- start 2024-02-13T03:19:40.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.023+0000 7fbd8a2d0700 1 -- start start 2024-02-13T03:19:40.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd8a2d0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbd841086f0 con 0x7fbd840fe050 2024-02-13T03:19:40.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd8a2d0700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbd84113f00 con 0x7fbd840fa420 2024-02-13T03:19:40.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd8a2d0700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbd841150e0 con 0x7fbd840f5f50 2024-02-13T03:19:40.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd837fe700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fbd840fa420 0x7fbd8410e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:46742/0 (socket says 172.21.15.183:46742) 2024-02-13T03:19:40.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd837fe700 1 -- 172.21.15.183:0/1221719372 learned_addr learned my addr 172.21.15.183:0/1221719372 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:19:40.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3786885840 0 0) 0x7fbd84113f00 con 0x7fbd840fa420 2024-02-13T03:19:40.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd6c001eb0 con 0x7fbd840fa420 2024-02-13T03:19:40.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307244871 0 0) 0x7fbd841150e0 con 0x7fbd840f5f50 2024-02-13T03:19:40.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd84113f00 con 0x7fbd840f5f50 2024-02-13T03:19:40.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2367240801 0 0) 0x7fbd6c001eb0 con 0x7fbd840fa420 2024-02-13T03:19:40.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd841150e0 con 0x7fbd840fa420 2024-02-13T03:19:40.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fbd78001940 con 0x7fbd840fa420 2024-02-13T03:19:40.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850806315 0 0) 0x7fbd841086f0 con 0x7fbd840fe050 2024-02-13T03:19:40.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd6c001eb0 con 0x7fbd840fe050 2024-02-13T03:19:40.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 178596566 0 0) 0x7fbd841150e0 con 0x7fbd840fa420 2024-02-13T03:19:40.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 >> v1:172.21.15.161:6790/0 conn(0x7fbd840f5f50 legacy=0x7fbd84107b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:19:40.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 >> v1:172.21.15.161:6789/0 conn(0x7fbd840fe050 legacy=0x7fbd841117d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:19:40.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd84116300 con 0x7fbd840fa420 2024-02-13T03:19:40.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/1221719372 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd841152d0 con 0x7fbd840fa420 2024-02-13T03:19:40.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/1221719372 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd84115740 con 0x7fbd840fa420 2024-02-13T03:19:40.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fbd780024a0 con 0x7fbd840fa420 2024-02-13T03:19:40.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fbd78005bd0 con 0x7fbd840fa420 2024-02-13T03:19:40.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/1221719372 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd70004f50 con 0x7fbd840fa420 2024-02-13T03:19:40.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 1965425884 0 0) 0x7fbd78018ac0 con 0x7fbd840fa420 2024-02-13T03:19:40.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.027+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(38..38 src has 1..38) v4 ==== 3470+0+0 (unknown 2839416835 0 0) 0x7fbd78077b30 con 0x7fbd840fa420 2024-02-13T03:19:40.037 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.035+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fbd78048650 con 0x7fbd840fa420 2024-02-13T03:19:40.229 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:19:40.227+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/1221719372 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd70000bc0 con 0x7fbd6c062110 2024-02-13T03:19:40.591 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:40 smithi183 bash[20824]: cluster 2024-02-13T03:19:39.005281+0000 mgr.y (mgr.14146) 210 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:40.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:40 smithi161 bash[27879]: cluster 2024-02-13T03:19:39.005281+0000 mgr.y (mgr.14146) 210 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:40.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:40 smithi161 bash[19769]: cluster 2024-02-13T03:19:39.005281+0000 mgr.y (mgr.14146) 210 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:41.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:41 smithi161 bash[19769]: audit 2024-02-13T03:19:40.229770+0000 mgr.y (mgr.14146) 211 : audit [DBG] from='client.24280 v1:172.21.15.183:0/1221719372' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:19:41.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:41 smithi161 bash[19769]: audit 2024-02-13T03:19:40.241055+0000 mon.a (mon.0) 451 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:19:41.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:41 smithi161 bash[19769]: audit 2024-02-13T03:19:40.246801+0000 mon.a (mon.0) 452 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:19:41.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:41 smithi161 bash[19769]: audit 2024-02-13T03:19:40.248302+0000 mon.a (mon.0) 453 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:41 smithi161 bash[27879]: audit 2024-02-13T03:19:40.229770+0000 mgr.y (mgr.14146) 211 : audit [DBG] from='client.24280 v1:172.21.15.183:0/1221719372' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:19:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:41 smithi161 bash[27879]: audit 2024-02-13T03:19:40.241055+0000 mon.a (mon.0) 451 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:19:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:41 smithi161 bash[27879]: audit 2024-02-13T03:19:40.246801+0000 mon.a (mon.0) 452 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:19:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:41 smithi161 bash[27879]: audit 2024-02-13T03:19:40.248302+0000 mon.a (mon.0) 453 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:41.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:41 smithi183 bash[20824]: audit 2024-02-13T03:19:40.229770+0000 mgr.y (mgr.14146) 211 : audit [DBG] from='client.24280 v1:172.21.15.183:0/1221719372' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:19:41.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:41 smithi183 bash[20824]: audit 2024-02-13T03:19:40.241055+0000 mon.a (mon.0) 451 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:19:41.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:41 smithi183 bash[20824]: audit 2024-02-13T03:19:40.246801+0000 mon.a (mon.0) 452 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:19:41.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:41 smithi183 bash[20824]: audit 2024-02-13T03:19:40.248302+0000 mon.a (mon.0) 453 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[19769]: cluster 2024-02-13T03:19:41.005936+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[19769]: cephadm 2024-02-13T03:19:41.368304+0000 mgr.y (mgr.14146) 213 : cephadm [INF] Detected new or changed devices on smithi183 2024-02-13T03:19:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[19769]: audit 2024-02-13T03:19:41.377933+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[19769]: audit 2024-02-13T03:19:41.379540+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:19:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[19769]: audit 2024-02-13T03:19:41.381193+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:19:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[19769]: cephadm 2024-02-13T03:19:41.382486+0000 mgr.y (mgr.14146) 214 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 8659M 2024-02-13T03:19:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[19769]: audit 2024-02-13T03:19:41.390505+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[19769]: audit 2024-02-13T03:19:41.392156+0000 mon.a (mon.0) 458 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[19769]: audit 2024-02-13T03:19:41.393461+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:19:42.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[19769]: audit 2024-02-13T03:19:41.400477+0000 mon.a (mon.0) 460 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:42.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[27879]: cluster 2024-02-13T03:19:41.005936+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:42.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[27879]: cephadm 2024-02-13T03:19:41.368304+0000 mgr.y (mgr.14146) 213 : cephadm [INF] Detected new or changed devices on smithi183 2024-02-13T03:19:42.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[27879]: audit 2024-02-13T03:19:41.377933+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:42.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[27879]: audit 2024-02-13T03:19:41.379540+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:19:42.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[27879]: audit 2024-02-13T03:19:41.381193+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:19:42.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[27879]: cephadm 2024-02-13T03:19:41.382486+0000 mgr.y (mgr.14146) 214 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 8659M 2024-02-13T03:19:42.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[27879]: audit 2024-02-13T03:19:41.390505+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:42.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[27879]: audit 2024-02-13T03:19:41.392156+0000 mon.a (mon.0) 458 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:42.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[27879]: audit 2024-02-13T03:19:41.393461+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:19:42.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:42 smithi161 bash[27879]: audit 2024-02-13T03:19:41.400477+0000 mon.a (mon.0) 460 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:42.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:42 smithi183 bash[20824]: cluster 2024-02-13T03:19:41.005936+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:42.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:42 smithi183 bash[20824]: cephadm 2024-02-13T03:19:41.368304+0000 mgr.y (mgr.14146) 213 : cephadm [INF] Detected new or changed devices on smithi183 2024-02-13T03:19:42.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:42 smithi183 bash[20824]: audit 2024-02-13T03:19:41.377933+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:42.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:42 smithi183 bash[20824]: audit 2024-02-13T03:19:41.379540+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:19:42.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:42 smithi183 bash[20824]: audit 2024-02-13T03:19:41.381193+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:19:42.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:42 smithi183 bash[20824]: cephadm 2024-02-13T03:19:41.382486+0000 mgr.y (mgr.14146) 214 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 8659M 2024-02-13T03:19:42.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:42 smithi183 bash[20824]: audit 2024-02-13T03:19:41.390505+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:42.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:42 smithi183 bash[20824]: audit 2024-02-13T03:19:41.392156+0000 mon.a (mon.0) 458 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:19:42.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:42 smithi183 bash[20824]: audit 2024-02-13T03:19:41.393461+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:19:42.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:42 smithi183 bash[20824]: audit 2024-02-13T03:19:41.400477+0000 mon.a (mon.0) 460 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:19:44.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:44 smithi161 bash[19769]: cluster 2024-02-13T03:19:43.006548+0000 mgr.y (mgr.14146) 215 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:44 smithi161 bash[27879]: cluster 2024-02-13T03:19:43.006548+0000 mgr.y (mgr.14146) 215 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:44.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:44 smithi183 bash[20824]: cluster 2024-02-13T03:19:43.006548+0000 mgr.y (mgr.14146) 215 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:46 smithi183 bash[20824]: cluster 2024-02-13T03:19:45.007086+0000 mgr.y (mgr.14146) 216 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:46.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:46 smithi161 bash[19769]: cluster 2024-02-13T03:19:45.007086+0000 mgr.y (mgr.14146) 216 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:46 smithi161 bash[27879]: cluster 2024-02-13T03:19:45.007086+0000 mgr.y (mgr.14146) 216 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:48.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:48 smithi161 bash[19769]: cluster 2024-02-13T03:19:47.007688+0000 mgr.y (mgr.14146) 217 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:48.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:48 smithi161 bash[27879]: cluster 2024-02-13T03:19:47.007688+0000 mgr.y (mgr.14146) 217 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:48.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:48 smithi183 bash[20824]: cluster 2024-02-13T03:19:47.007688+0000 mgr.y (mgr.14146) 217 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:49.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:49 smithi161 bash[19769]: audit 2024-02-13T03:19:48.449754+0000 mon.a (mon.0) 461 : audit [INF] from='client.24286 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aad4c7d8-2a9d-46f2-bf74-fb1bfa83a64c"}]: dispatch 2024-02-13T03:19:49.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:49 smithi161 bash[19769]: audit 2024-02-13T03:19:48.451402+0000 mon.b (mon.1) 11 : audit [INF] from='client.? v1:172.21.15.183:0/1715517919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aad4c7d8-2a9d-46f2-bf74-fb1bfa83a64c"}]: dispatch 2024-02-13T03:19:49.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:49 smithi161 bash[19769]: audit 2024-02-13T03:19:48.461118+0000 mon.a (mon.0) 462 : audit [INF] from='client.24286 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aad4c7d8-2a9d-46f2-bf74-fb1bfa83a64c"}]': finished 2024-02-13T03:19:49.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:49 smithi161 bash[19769]: cluster 2024-02-13T03:19:48.461292+0000 mon.a (mon.0) 463 : cluster [DBG] osdmap e39: 7 total, 6 up, 7 in 2024-02-13T03:19:49.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:49 smithi161 bash[19769]: audit 2024-02-13T03:19:48.461681+0000 mon.a (mon.0) 464 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:19:49.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:49 smithi161 bash[19769]: audit 2024-02-13T03:19:49.061123+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? v1:172.21.15.183:0/3535537051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:19:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:49 smithi161 bash[27879]: audit 2024-02-13T03:19:48.449754+0000 mon.a (mon.0) 461 : audit [INF] from='client.24286 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aad4c7d8-2a9d-46f2-bf74-fb1bfa83a64c"}]: dispatch 2024-02-13T03:19:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:49 smithi161 bash[27879]: audit 2024-02-13T03:19:48.451402+0000 mon.b (mon.1) 11 : audit [INF] from='client.? v1:172.21.15.183:0/1715517919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aad4c7d8-2a9d-46f2-bf74-fb1bfa83a64c"}]: dispatch 2024-02-13T03:19:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:49 smithi161 bash[27879]: audit 2024-02-13T03:19:48.461118+0000 mon.a (mon.0) 462 : audit [INF] from='client.24286 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aad4c7d8-2a9d-46f2-bf74-fb1bfa83a64c"}]': finished 2024-02-13T03:19:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:49 smithi161 bash[27879]: cluster 2024-02-13T03:19:48.461292+0000 mon.a (mon.0) 463 : cluster [DBG] osdmap e39: 7 total, 6 up, 7 in 2024-02-13T03:19:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:49 smithi161 bash[27879]: audit 2024-02-13T03:19:48.461681+0000 mon.a (mon.0) 464 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:19:49.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:49 smithi161 bash[27879]: audit 2024-02-13T03:19:49.061123+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? v1:172.21.15.183:0/3535537051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:19:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:49 smithi183 bash[20824]: audit 2024-02-13T03:19:48.449754+0000 mon.a (mon.0) 461 : audit [INF] from='client.24286 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aad4c7d8-2a9d-46f2-bf74-fb1bfa83a64c"}]: dispatch 2024-02-13T03:19:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:49 smithi183 bash[20824]: audit 2024-02-13T03:19:48.451402+0000 mon.b (mon.1) 11 : audit [INF] from='client.? v1:172.21.15.183:0/1715517919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aad4c7d8-2a9d-46f2-bf74-fb1bfa83a64c"}]: dispatch 2024-02-13T03:19:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:49 smithi183 bash[20824]: audit 2024-02-13T03:19:48.461118+0000 mon.a (mon.0) 462 : audit [INF] from='client.24286 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aad4c7d8-2a9d-46f2-bf74-fb1bfa83a64c"}]': finished 2024-02-13T03:19:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:49 smithi183 bash[20824]: cluster 2024-02-13T03:19:48.461292+0000 mon.a (mon.0) 463 : cluster [DBG] osdmap e39: 7 total, 6 up, 7 in 2024-02-13T03:19:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:49 smithi183 bash[20824]: audit 2024-02-13T03:19:48.461681+0000 mon.a (mon.0) 464 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:19:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:49 smithi183 bash[20824]: audit 2024-02-13T03:19:49.061123+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? v1:172.21.15.183:0/3535537051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:19:50.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:50 smithi183 bash[20824]: cluster 2024-02-13T03:19:49.008203+0000 mgr.y (mgr.14146) 218 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:50.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:50 smithi161 bash[19769]: cluster 2024-02-13T03:19:49.008203+0000 mgr.y (mgr.14146) 218 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:50.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:50 smithi161 bash[27879]: cluster 2024-02-13T03:19:49.008203+0000 mgr.y (mgr.14146) 218 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:52.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:52 smithi183 bash[20824]: cluster 2024-02-13T03:19:51.008757+0000 mgr.y (mgr.14146) 219 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:52.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:52 smithi161 bash[19769]: cluster 2024-02-13T03:19:51.008757+0000 mgr.y (mgr.14146) 219 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:52.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:52 smithi161 bash[27879]: cluster 2024-02-13T03:19:51.008757+0000 mgr.y (mgr.14146) 219 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:54 smithi183 bash[20824]: cluster 2024-02-13T03:19:53.009377+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:54.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:54 smithi161 bash[19769]: cluster 2024-02-13T03:19:53.009377+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:54.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:54 smithi161 bash[27879]: cluster 2024-02-13T03:19:53.009377+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:56 smithi183 bash[20824]: cluster 2024-02-13T03:19:55.010034+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:56.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:56 smithi161 bash[19769]: cluster 2024-02-13T03:19:55.010034+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:56 smithi161 bash[27879]: cluster 2024-02-13T03:19:55.010034+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:19:58 smithi183 bash[20824]: cluster 2024-02-13T03:19:57.010761+0000 mgr.y (mgr.14146) 222 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:58.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:19:58 smithi161 bash[19769]: cluster 2024-02-13T03:19:57.010761+0000 mgr.y (mgr.14146) 222 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:19:58.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:19:58 smithi161 bash[27879]: cluster 2024-02-13T03:19:57.010761+0000 mgr.y (mgr.14146) 222 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:00.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:00 smithi161 bash[19769]: cluster 2024-02-13T03:19:59.011358+0000 mgr.y (mgr.14146) 223 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:00.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:00 smithi161 bash[19769]: cluster 2024-02-13T03:20:00.000160+0000 mon.a (mon.0) 465 : cluster [INF] overall HEALTH_OK 2024-02-13T03:20:00.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:00 smithi161 bash[19769]: audit 2024-02-13T03:20:00.468353+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-13T03:20:00.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:00 smithi161 bash[19769]: audit 2024-02-13T03:20:00.470278+0000 mon.a (mon.0) 467 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:00.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:00 smithi161 bash[27879]: cluster 2024-02-13T03:19:59.011358+0000 mgr.y (mgr.14146) 223 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:00.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:00 smithi161 bash[27879]: cluster 2024-02-13T03:20:00.000160+0000 mon.a (mon.0) 465 : cluster [INF] overall HEALTH_OK 2024-02-13T03:20:00.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:00 smithi161 bash[27879]: audit 2024-02-13T03:20:00.468353+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-13T03:20:00.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:00 smithi161 bash[27879]: audit 2024-02-13T03:20:00.470278+0000 mon.a (mon.0) 467 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:00 smithi183 bash[20824]: cluster 2024-02-13T03:19:59.011358+0000 mgr.y (mgr.14146) 223 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:00 smithi183 bash[20824]: cluster 2024-02-13T03:20:00.000160+0000 mon.a (mon.0) 465 : cluster [INF] overall HEALTH_OK 2024-02-13T03:20:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:00 smithi183 bash[20824]: audit 2024-02-13T03:20:00.468353+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-13T03:20:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:00 smithi183 bash[20824]: audit 2024-02-13T03:20:00.470278+0000 mon.a (mon.0) 467 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:01.803 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:01 smithi183 bash[20824]: cephadm 2024-02-13T03:20:00.471696+0000 mgr.y (mgr.14146) 224 : cephadm [INF] Deploying daemon osd.6 on smithi183 2024-02-13T03:20:01.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:01 smithi161 bash[19769]: cephadm 2024-02-13T03:20:00.471696+0000 mgr.y (mgr.14146) 224 : cephadm [INF] Deploying daemon osd.6 on smithi183 2024-02-13T03:20:01.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:01 smithi161 bash[27879]: cephadm 2024-02-13T03:20:00.471696+0000 mgr.y (mgr.14146) 224 : cephadm [INF] Deploying daemon osd.6 on smithi183 2024-02-13T03:20:02.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:02 smithi161 bash[27879]: cluster 2024-02-13T03:20:01.011864+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:02.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:02 smithi161 bash[19769]: cluster 2024-02-13T03:20:01.011864+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:02.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:02 smithi183 bash[20824]: cluster 2024-02-13T03:20:01.011864+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:04.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:04 smithi161 bash[19769]: cluster 2024-02-13T03:20:03.012428+0000 mgr.y (mgr.14146) 226 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:04.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:04 smithi161 bash[19769]: audit 2024-02-13T03:20:04.386148+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:04.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:04 smithi161 bash[19769]: audit 2024-02-13T03:20:04.386769+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:20:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:04 smithi161 bash[27879]: cluster 2024-02-13T03:20:03.012428+0000 mgr.y (mgr.14146) 226 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:04 smithi161 bash[27879]: audit 2024-02-13T03:20:04.386148+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:04 smithi161 bash[27879]: audit 2024-02-13T03:20:04.386769+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:20:04.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:04 smithi183 bash[20824]: cluster 2024-02-13T03:20:03.012428+0000 mgr.y (mgr.14146) 226 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:04.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:04 smithi183 bash[20824]: audit 2024-02-13T03:20:04.386148+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:04.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:04 smithi183 bash[20824]: audit 2024-02-13T03:20:04.386769+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:20:06.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:06 smithi161 bash[19769]: cluster 2024-02-13T03:20:05.012975+0000 mgr.y (mgr.14146) 227 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:06.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:06 smithi161 bash[27879]: cluster 2024-02-13T03:20:05.012975+0000 mgr.y (mgr.14146) 227 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:06.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:06 smithi183 bash[20824]: cluster 2024-02-13T03:20:05.012975+0000 mgr.y (mgr.14146) 227 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:08.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:08 smithi161 bash[19769]: cluster 2024-02-13T03:20:07.013549+0000 mgr.y (mgr.14146) 228 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:08.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:08 smithi161 bash[27879]: cluster 2024-02-13T03:20:07.013549+0000 mgr.y (mgr.14146) 228 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:08 smithi183 bash[20824]: cluster 2024-02-13T03:20:07.013549+0000 mgr.y (mgr.14146) 228 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:10 smithi183 bash[20824]: cluster 2024-02-13T03:20:09.014143+0000 mgr.y (mgr.14146) 229 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:10 smithi183 bash[20824]: audit 2024-02-13T03:20:09.787439+0000 mon.a (mon.0) 470 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:10 smithi183 bash[20824]: audit 2024-02-13T03:20:10.075866+0000 mon.a (mon.0) 471 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:10 smithi183 bash[20824]: audit 2024-02-13T03:20:10.077668+0000 mon.a (mon.0) 472 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:20:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:10 smithi183 bash[20824]: audit 2024-02-13T03:20:10.086657+0000 mon.a (mon.0) 473 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:10.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:10 smithi161 bash[19769]: cluster 2024-02-13T03:20:09.014143+0000 mgr.y (mgr.14146) 229 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:10.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:10 smithi161 bash[19769]: audit 2024-02-13T03:20:09.787439+0000 mon.a (mon.0) 470 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:10.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:10 smithi161 bash[19769]: audit 2024-02-13T03:20:10.075866+0000 mon.a (mon.0) 471 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:10.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:10 smithi161 bash[19769]: audit 2024-02-13T03:20:10.077668+0000 mon.a (mon.0) 472 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:20:10.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:10 smithi161 bash[19769]: audit 2024-02-13T03:20:10.086657+0000 mon.a (mon.0) 473 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:10 smithi161 bash[27879]: cluster 2024-02-13T03:20:09.014143+0000 mgr.y (mgr.14146) 229 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:10 smithi161 bash[27879]: audit 2024-02-13T03:20:09.787439+0000 mon.a (mon.0) 470 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:10 smithi161 bash[27879]: audit 2024-02-13T03:20:10.075866+0000 mon.a (mon.0) 471 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:10 smithi161 bash[27879]: audit 2024-02-13T03:20:10.077668+0000 mon.a (mon.0) 472 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:20:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:10 smithi161 bash[27879]: audit 2024-02-13T03:20:10.086657+0000 mon.a (mon.0) 473 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:11.817 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:11 smithi183 bash[20824]: audit 2024-02-13T03:20:10.938713+0000 mon.a (mon.0) 474 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-13T03:20:11.817 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:11 smithi183 bash[20824]: audit 2024-02-13T03:20:10.940847+0000 mon.b (mon.1) 13 : audit [INF] from='osd.6 v1:172.21.15.183:6808/3716679598' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-13T03:20:11.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:11 smithi161 bash[27879]: audit 2024-02-13T03:20:10.938713+0000 mon.a (mon.0) 474 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-13T03:20:11.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:11 smithi161 bash[27879]: audit 2024-02-13T03:20:10.940847+0000 mon.b (mon.1) 13 : audit [INF] from='osd.6 v1:172.21.15.183:6808/3716679598' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-13T03:20:11.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:11 smithi161 bash[19769]: audit 2024-02-13T03:20:10.938713+0000 mon.a (mon.0) 474 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-13T03:20:11.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:11 smithi161 bash[19769]: audit 2024-02-13T03:20:10.940847+0000 mon.b (mon.1) 13 : audit [INF] from='osd.6 v1:172.21.15.183:6808/3716679598' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-13T03:20:12.838 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:12.834+0000 7fbd81775700 1 -- 172.21.15.183:0/1221719372 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 4118428161) 0x7fbd70000bc0 con 0x7fbd6c062110 2024-02-13T03:20:12.838 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 6 on host 'smithi183' 2024-02-13T03:20:12.842 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:12.838+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/1221719372 >> v1:172.21.15.161:6800/1187535987 conn(0x7fbd6c062110 legacy=0x7fbd6c0645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:20:12.842 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:12.838+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/1221719372 >> v1:172.21.15.183:6789/0 conn(0x7fbd840fa420 legacy=0x7fbd8410e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:20:12.842 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:12.838+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/1221719372 shutdown_connections 2024-02-13T03:20:12.842 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:12.838+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/1221719372 >> 172.21.15.183:0/1221719372 conn(0x7fbd840f16b0 msgr2=0x7fbd84103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:20:12.842 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:12.838+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/1221719372 shutdown_connections 2024-02-13T03:20:12.842 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:12.838+0000 7fbd8a2d0700 1 -- 172.21.15.183:0/1221719372 wait complete. 2024-02-13T03:20:12.899 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:12 smithi183 bash[20824]: cluster 2024-02-13T03:20:11.014713+0000 mgr.y (mgr.14146) 230 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:12 smithi183 bash[20824]: audit 2024-02-13T03:20:11.534421+0000 mon.a (mon.0) 475 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-13T03:20:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:12 smithi183 bash[20824]: cluster 2024-02-13T03:20:11.534526+0000 mon.a (mon.0) 476 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-02-13T03:20:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:12 smithi183 bash[20824]: audit 2024-02-13T03:20:11.534718+0000 mon.a (mon.0) 477 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:20:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:12 smithi183 bash[20824]: audit 2024-02-13T03:20:11.535982+0000 mon.a (mon.0) 478 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:20:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:12 smithi183 bash[20824]: audit 2024-02-13T03:20:11.538134+0000 mon.b (mon.1) 14 : audit [INF] from='osd.6 v1:172.21.15.183:6808/3716679598' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:20:12.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:12 smithi161 bash[19769]: cluster 2024-02-13T03:20:11.014713+0000 mgr.y (mgr.14146) 230 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:12.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:12 smithi161 bash[19769]: audit 2024-02-13T03:20:11.534421+0000 mon.a (mon.0) 475 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-13T03:20:12.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:12 smithi161 bash[19769]: cluster 2024-02-13T03:20:11.534526+0000 mon.a (mon.0) 476 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-02-13T03:20:12.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:12 smithi161 bash[19769]: audit 2024-02-13T03:20:11.534718+0000 mon.a (mon.0) 477 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:20:12.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:12 smithi161 bash[19769]: audit 2024-02-13T03:20:11.535982+0000 mon.a (mon.0) 478 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:20:12.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:12 smithi161 bash[19769]: audit 2024-02-13T03:20:11.538134+0000 mon.b (mon.1) 14 : audit [INF] from='osd.6 v1:172.21.15.183:6808/3716679598' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:20:12.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:12 smithi161 bash[27879]: cluster 2024-02-13T03:20:11.014713+0000 mgr.y (mgr.14146) 230 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:12.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:12 smithi161 bash[27879]: audit 2024-02-13T03:20:11.534421+0000 mon.a (mon.0) 475 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-13T03:20:12.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:12 smithi161 bash[27879]: cluster 2024-02-13T03:20:11.534526+0000 mon.a (mon.0) 476 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-02-13T03:20:12.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:12 smithi161 bash[27879]: audit 2024-02-13T03:20:11.534718+0000 mon.a (mon.0) 477 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:20:12.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:12 smithi161 bash[27879]: audit 2024-02-13T03:20:11.535982+0000 mon.a (mon.0) 478 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:20:12.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:12 smithi161 bash[27879]: audit 2024-02-13T03:20:11.538134+0000 mon.b (mon.1) 14 : audit [INF] from='osd.6 v1:172.21.15.183:6808/3716679598' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:20:13.444 DEBUG:teuthology.orchestra.run.smithi183:osd.6> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.6.service 2024-02-13T03:20:13.446 INFO:tasks.cephadm:Deploying osd.7 on smithi183 with /dev/vg_nvme/lv_1... 2024-02-13T03:20:13.446 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-13T03:20:13.563 INFO:journalctl@ceph.osd.6.smithi183.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:20:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:13 smithi183 bash[20824]: audit 2024-02-13T03:20:12.537575+0000 mon.a (mon.0) 479 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-02-13T03:20:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:13 smithi183 bash[20824]: cluster 2024-02-13T03:20:12.537846+0000 mon.a (mon.0) 480 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-02-13T03:20:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:13 smithi183 bash[20824]: audit 2024-02-13T03:20:12.538661+0000 mon.a (mon.0) 481 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:20:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:13 smithi183 bash[20824]: audit 2024-02-13T03:20:12.544888+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:20:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:13 smithi183 bash[20824]: audit 2024-02-13T03:20:12.835320+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:13 smithi183 bash[20824]: audit 2024-02-13T03:20:12.863499+0000 mon.a (mon.0) 484 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:20:13.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:13 smithi161 bash[19769]: audit 2024-02-13T03:20:12.537575+0000 mon.a (mon.0) 479 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-02-13T03:20:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:13 smithi161 bash[19769]: cluster 2024-02-13T03:20:12.537846+0000 mon.a (mon.0) 480 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-02-13T03:20:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:13 smithi161 bash[19769]: audit 2024-02-13T03:20:12.538661+0000 mon.a (mon.0) 481 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:20:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:13 smithi161 bash[19769]: audit 2024-02-13T03:20:12.544888+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:20:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:13 smithi161 bash[19769]: audit 2024-02-13T03:20:12.835320+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:13 smithi161 bash[19769]: audit 2024-02-13T03:20:12.863499+0000 mon.a (mon.0) 484 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:20:13.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:13 smithi161 bash[27879]: audit 2024-02-13T03:20:12.537575+0000 mon.a (mon.0) 479 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-02-13T03:20:13.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:13 smithi161 bash[27879]: cluster 2024-02-13T03:20:12.537846+0000 mon.a (mon.0) 480 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-02-13T03:20:13.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:13 smithi161 bash[27879]: audit 2024-02-13T03:20:12.538661+0000 mon.a (mon.0) 481 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:20:13.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:13 smithi161 bash[27879]: audit 2024-02-13T03:20:12.544888+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:20:13.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:13 smithi161 bash[27879]: audit 2024-02-13T03:20:12.835320+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:13 smithi161 bash[27879]: audit 2024-02-13T03:20:12.863499+0000 mon.a (mon.0) 484 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:20:14.787 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:14 smithi183 bash[20824]: cluster 2024-02-13T03:20:11.948296+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:20:14.787 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:14 smithi183 bash[20824]: cluster 2024-02-13T03:20:11.948389+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:20:14.787 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:14 smithi183 bash[20824]: cluster 2024-02-13T03:20:13.015274+0000 mgr.y (mgr.14146) 231 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:14.787 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:14 smithi183 bash[20824]: cluster 2024-02-13T03:20:13.549157+0000 mon.a (mon.0) 485 : cluster [INF] osd.6 v1:172.21.15.183:6808/3716679598 boot 2024-02-13T03:20:14.787 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:14 smithi183 bash[20824]: cluster 2024-02-13T03:20:13.549292+0000 mon.a (mon.0) 486 : cluster [DBG] osdmap e42: 7 total, 7 up, 7 in 2024-02-13T03:20:14.787 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:14 smithi183 bash[20824]: audit 2024-02-13T03:20:13.550048+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:20:14.787 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:14 smithi183 bash[20824]: cluster 2024-02-13T03:20:14.548689+0000 mon.a (mon.0) 488 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-02-13T03:20:14.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[19769]: cluster 2024-02-13T03:20:11.948296+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:20:14.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[19769]: cluster 2024-02-13T03:20:11.948389+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:20:14.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[19769]: cluster 2024-02-13T03:20:13.015274+0000 mgr.y (mgr.14146) 231 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:14.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[19769]: cluster 2024-02-13T03:20:13.549157+0000 mon.a (mon.0) 485 : cluster [INF] osd.6 v1:172.21.15.183:6808/3716679598 boot 2024-02-13T03:20:14.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[19769]: cluster 2024-02-13T03:20:13.549292+0000 mon.a (mon.0) 486 : cluster [DBG] osdmap e42: 7 total, 7 up, 7 in 2024-02-13T03:20:14.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[19769]: audit 2024-02-13T03:20:13.550048+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:20:14.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[19769]: cluster 2024-02-13T03:20:14.548689+0000 mon.a (mon.0) 488 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-02-13T03:20:14.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[27879]: cluster 2024-02-13T03:20:11.948296+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:20:14.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[27879]: cluster 2024-02-13T03:20:11.948389+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:20:14.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[27879]: cluster 2024-02-13T03:20:13.015274+0000 mgr.y (mgr.14146) 231 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-13T03:20:14.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[27879]: cluster 2024-02-13T03:20:13.549157+0000 mon.a (mon.0) 485 : cluster [INF] osd.6 v1:172.21.15.183:6808/3716679598 boot 2024-02-13T03:20:14.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[27879]: cluster 2024-02-13T03:20:13.549292+0000 mon.a (mon.0) 486 : cluster [DBG] osdmap e42: 7 total, 7 up, 7 in 2024-02-13T03:20:14.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[27879]: audit 2024-02-13T03:20:13.550048+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:20:14.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:14 smithi161 bash[27879]: cluster 2024-02-13T03:20:14.548689+0000 mon.a (mon.0) 488 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-02-13T03:20:16.691 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-13T03:20:16.691 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-13T03:20:16.691 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-02-13T03:20:16.691 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-02-13T03:20:16.691 INFO:teuthology.orchestra.run.smithi183.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0137474 s, 763 MB/s 2024-02-13T03:20:16.691 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-02-13T03:20:16.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:16 smithi161 bash[19769]: cluster 2024-02-13T03:20:15.015844+0000 mgr.y (mgr.14146) 232 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:16.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:16 smithi161 bash[19769]: cluster 2024-02-13T03:20:15.554391+0000 mon.a (mon.0) 489 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-02-13T03:20:16.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:16 smithi161 bash[27879]: cluster 2024-02-13T03:20:15.015844+0000 mgr.y (mgr.14146) 232 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:16.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:16 smithi161 bash[27879]: cluster 2024-02-13T03:20:15.554391+0000 mon.a (mon.0) 489 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-02-13T03:20:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:16 smithi183 bash[20824]: cluster 2024-02-13T03:20:15.015844+0000 mgr.y (mgr.14146) 232 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:16 smithi183 bash[20824]: cluster 2024-02-13T03:20:15.554391+0000 mon.a (mon.0) 489 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-02-13T03:20:17.312 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-02-13T03:20:18.858 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:18 smithi183 bash[20824]: cluster 2024-02-13T03:20:17.016441+0000 mgr.y (mgr.14146) 233 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:18.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:18 smithi161 bash[19769]: cluster 2024-02-13T03:20:17.016441+0000 mgr.y (mgr.14146) 233 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:18.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:18 smithi161 bash[27879]: cluster 2024-02-13T03:20:17.016441+0000 mgr.y (mgr.14146) 233 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:20.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:20 smithi161 bash[19769]: cluster 2024-02-13T03:20:19.016966+0000 mgr.y (mgr.14146) 234 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:20 smithi161 bash[27879]: cluster 2024-02-13T03:20:19.016966+0000 mgr.y (mgr.14146) 234 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:20.976 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:20 smithi183 bash[20824]: cluster 2024-02-13T03:20:19.016966+0000 mgr.y (mgr.14146) 234 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:21.225 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.222+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/1556958183 >> v1:172.21.15.183:6789/0 conn(0x7f3c040f7db0 legacy=0x7f3c040f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:20:21.225 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.222+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/1556958183 shutdown_connections 2024-02-13T03:20:21.225 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.222+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/1556958183 >> 172.21.15.183:0/1556958183 conn(0x7f3c040f16b0 msgr2=0x7f3c040f3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:20:21.225 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.222+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/1556958183 shutdown_connections 2024-02-13T03:20:21.226 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.222+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/1556958183 wait complete. 2024-02-13T03:20:21.226 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3c0b6d1700 1 Processor -- start 2024-02-13T03:20:21.226 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3c0b6d1700 1 -- start start 2024-02-13T03:20:21.229 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3c0b6d1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3c04112e10 con 0x7f3c040fad40 2024-02-13T03:20:21.229 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3c0b6d1700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3c04113fb0 con 0x7f3c040fe8e0 2024-02-13T03:20:21.229 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3c0b6d1700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3c04115150 con 0x7f3c040f7db0 2024-02-13T03:20:21.229 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3c09c6e700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f3c040fe8e0 0x7f3c041116f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:45386/0 (socket says 172.21.15.183:45386) 2024-02-13T03:20:21.229 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3c09c6e700 1 -- 172.21.15.183:0/2000239285 learned_addr learned my addr 172.21.15.183:0/2000239285 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:20:21.229 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321972091 0 0) 0x7f3c04113fb0 con 0x7f3c040fe8e0 2024-02-13T03:20:21.229 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bf0003030 con 0x7f3c040fe8e0 2024-02-13T03:20:21.229 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164072597 0 0) 0x7f3c04115150 con 0x7f3c040f7db0 2024-02-13T03:20:21.229 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c04113fb0 con 0x7f3c040f7db0 2024-02-13T03:20:21.229 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335056021 0 0) 0x7f3c04112e10 con 0x7f3c040fad40 2024-02-13T03:20:21.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c04115150 con 0x7f3c040fad40 2024-02-13T03:20:21.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3895320552 0 0) 0x7f3bf0003030 con 0x7f3c040fe8e0 2024-02-13T03:20:21.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3c04112e10 con 0x7f3c040fe8e0 2024-02-13T03:20:21.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f3bfc001c80 con 0x7f3c040fe8e0 2024-02-13T03:20:21.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4252423244 0 0) 0x7f3c04112e10 con 0x7f3c040fe8e0 2024-02-13T03:20:21.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 >> v1:172.21.15.161:6790/0 conn(0x7f3c040f7db0 legacy=0x7f3c0410a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:20:21.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 >> v1:172.21.15.161:6789/0 conn(0x7f3c040fad40 legacy=0x7f3c0410dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:20:21.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c041162f0 con 0x7f3c040fe8e0 2024-02-13T03:20:21.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/2000239285 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c04115340 con 0x7f3c040fe8e0 2024-02-13T03:20:21.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f3bfc002010 con 0x7f3c040fe8e0 2024-02-13T03:20:21.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f3bfc005e60 con 0x7f3c040fe8e0 2024-02-13T03:20:21.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/2000239285 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c041157b0 con 0x7f3c040fe8e0 2024-02-13T03:20:21.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.226+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/2000239285 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c04060710 con 0x7f3c040fe8e0 2024-02-13T03:20:21.231 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.230+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 1965425884 0 0) 0x7f3bfc018dc0 con 0x7f3c040fe8e0 2024-02-13T03:20:21.231 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.230+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(44..44 src has 1..44) v4 ==== 3762+0+0 (unknown 1087729600 0 0) 0x7f3bfc077ee0 con 0x7f3c040fe8e0 2024-02-13T03:20:21.238 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.238+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f3c04060710 con 0x7f3c040fe8e0 2024-02-13T03:20:21.426 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:21.426+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/2000239285 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c04060710 con 0x7f3bf00620f0 2024-02-13T03:20:21.821 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:21 smithi183 bash[20824]: audit 2024-02-13T03:20:21.433294+0000 mon.a (mon.0) 490 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:20:21.821 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:21 smithi183 bash[20824]: audit 2024-02-13T03:20:21.439411+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:20:21.821 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:21 smithi183 bash[20824]: audit 2024-02-13T03:20:21.440992+0000 mon.a (mon.0) 492 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:21.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:21 smithi161 bash[19769]: audit 2024-02-13T03:20:21.433294+0000 mon.a (mon.0) 490 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:20:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:21 smithi161 bash[19769]: audit 2024-02-13T03:20:21.439411+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:20:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:21 smithi161 bash[19769]: audit 2024-02-13T03:20:21.440992+0000 mon.a (mon.0) 492 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:21 smithi161 bash[27879]: audit 2024-02-13T03:20:21.433294+0000 mon.a (mon.0) 490 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T03:20:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:21 smithi161 bash[27879]: audit 2024-02-13T03:20:21.439411+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T03:20:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:21 smithi161 bash[27879]: audit 2024-02-13T03:20:21.440992+0000 mon.a (mon.0) 492 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:22.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:22 smithi161 bash[19769]: cluster 2024-02-13T03:20:21.017549+0000 mgr.y (mgr.14146) 235 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:22.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:22 smithi161 bash[19769]: audit 2024-02-13T03:20:21.426324+0000 mgr.y (mgr.14146) 236 : audit [DBG] from='client.24307 v1:172.21.15.183:0/2000239285' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:20:22.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:22 smithi161 bash[27879]: cluster 2024-02-13T03:20:21.017549+0000 mgr.y (mgr.14146) 235 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:22.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:22 smithi161 bash[27879]: audit 2024-02-13T03:20:21.426324+0000 mgr.y (mgr.14146) 236 : audit [DBG] from='client.24307 v1:172.21.15.183:0/2000239285' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:20:22.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:22 smithi183 bash[20824]: cluster 2024-02-13T03:20:21.017549+0000 mgr.y (mgr.14146) 235 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:22.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:22 smithi183 bash[20824]: audit 2024-02-13T03:20:21.426324+0000 mgr.y (mgr.14146) 236 : audit [DBG] from='client.24307 v1:172.21.15.183:0/2000239285' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:20:23.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[19769]: cephadm 2024-02-13T03:20:22.634779+0000 mgr.y (mgr.14146) 237 : cephadm [INF] Detected new or changed devices on smithi183 2024-02-13T03:20:23.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[19769]: audit 2024-02-13T03:20:22.646172+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:23.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[19769]: audit 2024-02-13T03:20:22.647843+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:20:23.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[19769]: audit 2024-02-13T03:20:22.649407+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:20:23.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[19769]: audit 2024-02-13T03:20:22.650835+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:20:23.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[19769]: cephadm 2024-02-13T03:20:22.652011+0000 mgr.y (mgr.14146) 238 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 5772M 2024-02-13T03:20:23.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[19769]: audit 2024-02-13T03:20:22.659458+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:23.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[19769]: audit 2024-02-13T03:20:22.661284+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:23.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[19769]: audit 2024-02-13T03:20:22.662973+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:20:23.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[19769]: audit 2024-02-13T03:20:22.671158+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:23.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[19769]: cluster 2024-02-13T03:20:23.018177+0000 mgr.y (mgr.14146) 239 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:23.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[27879]: cephadm 2024-02-13T03:20:22.634779+0000 mgr.y (mgr.14146) 237 : cephadm [INF] Detected new or changed devices on smithi183 2024-02-13T03:20:23.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[27879]: audit 2024-02-13T03:20:22.646172+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:23.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[27879]: audit 2024-02-13T03:20:22.647843+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:20:23.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[27879]: audit 2024-02-13T03:20:22.649407+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:20:23.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[27879]: audit 2024-02-13T03:20:22.650835+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:20:23.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[27879]: cephadm 2024-02-13T03:20:22.652011+0000 mgr.y (mgr.14146) 238 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 5772M 2024-02-13T03:20:23.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[27879]: audit 2024-02-13T03:20:22.659458+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:23.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[27879]: audit 2024-02-13T03:20:22.661284+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:23.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[27879]: audit 2024-02-13T03:20:22.662973+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:20:23.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[27879]: audit 2024-02-13T03:20:22.671158+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:23.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:23 smithi161 bash[27879]: cluster 2024-02-13T03:20:23.018177+0000 mgr.y (mgr.14146) 239 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:23 smithi183 bash[20824]: cephadm 2024-02-13T03:20:22.634779+0000 mgr.y (mgr.14146) 237 : cephadm [INF] Detected new or changed devices on smithi183 2024-02-13T03:20:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:23 smithi183 bash[20824]: audit 2024-02-13T03:20:22.646172+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:23 smithi183 bash[20824]: audit 2024-02-13T03:20:22.647843+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:20:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:23 smithi183 bash[20824]: audit 2024-02-13T03:20:22.649407+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:20:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:23 smithi183 bash[20824]: audit 2024-02-13T03:20:22.650835+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:20:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:23 smithi183 bash[20824]: cephadm 2024-02-13T03:20:22.652011+0000 mgr.y (mgr.14146) 238 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 5772M 2024-02-13T03:20:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:23 smithi183 bash[20824]: audit 2024-02-13T03:20:22.659458+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:23 smithi183 bash[20824]: audit 2024-02-13T03:20:22.661284+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:23.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:23 smithi183 bash[20824]: audit 2024-02-13T03:20:22.662973+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:20:23.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:23 smithi183 bash[20824]: audit 2024-02-13T03:20:22.671158+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:23.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:23 smithi183 bash[20824]: cluster 2024-02-13T03:20:23.018177+0000 mgr.y (mgr.14146) 239 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:26 smithi161 bash[19769]: cluster 2024-02-13T03:20:25.018763+0000 mgr.y (mgr.14146) 240 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:26.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:26 smithi161 bash[27879]: cluster 2024-02-13T03:20:25.018763+0000 mgr.y (mgr.14146) 240 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:26.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:26 smithi183 bash[20824]: cluster 2024-02-13T03:20:25.018763+0000 mgr.y (mgr.14146) 240 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:28 smithi183 bash[20824]: cluster 2024-02-13T03:20:27.019364+0000 mgr.y (mgr.14146) 241 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:28.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:28 smithi161 bash[27879]: cluster 2024-02-13T03:20:27.019364+0000 mgr.y (mgr.14146) 241 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:28.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:28 smithi161 bash[19769]: cluster 2024-02-13T03:20:27.019364+0000 mgr.y (mgr.14146) 241 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:30 smithi183 bash[20824]: cluster 2024-02-13T03:20:29.019918+0000 mgr.y (mgr.14146) 242 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:30 smithi183 bash[20824]: audit 2024-02-13T03:20:29.452667+0000 mon.a (mon.0) 501 : audit [INF] from='client.24313 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dc175e5-44c6-4454-81e3-92a465398a17"}]: dispatch 2024-02-13T03:20:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:30 smithi183 bash[20824]: audit 2024-02-13T03:20:29.454598+0000 mon.b (mon.1) 15 : audit [INF] from='client.? v1:172.21.15.183:0/1478714167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dc175e5-44c6-4454-81e3-92a465398a17"}]: dispatch 2024-02-13T03:20:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:30 smithi183 bash[20824]: audit 2024-02-13T03:20:29.463664+0000 mon.a (mon.0) 502 : audit [INF] from='client.24313 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4dc175e5-44c6-4454-81e3-92a465398a17"}]': finished 2024-02-13T03:20:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:30 smithi183 bash[20824]: cluster 2024-02-13T03:20:29.463854+0000 mon.a (mon.0) 503 : cluster [DBG] osdmap e45: 8 total, 7 up, 8 in 2024-02-13T03:20:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:30 smithi183 bash[20824]: audit 2024-02-13T03:20:29.464083+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:30 smithi183 bash[20824]: audit 2024-02-13T03:20:30.061867+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? v1:172.21.15.183:0/647717484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:20:30.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[27879]: cluster 2024-02-13T03:20:29.019918+0000 mgr.y (mgr.14146) 242 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[27879]: audit 2024-02-13T03:20:29.452667+0000 mon.a (mon.0) 501 : audit [INF] from='client.24313 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dc175e5-44c6-4454-81e3-92a465398a17"}]: dispatch 2024-02-13T03:20:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[27879]: audit 2024-02-13T03:20:29.454598+0000 mon.b (mon.1) 15 : audit [INF] from='client.? v1:172.21.15.183:0/1478714167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dc175e5-44c6-4454-81e3-92a465398a17"}]: dispatch 2024-02-13T03:20:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[27879]: audit 2024-02-13T03:20:29.463664+0000 mon.a (mon.0) 502 : audit [INF] from='client.24313 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4dc175e5-44c6-4454-81e3-92a465398a17"}]': finished 2024-02-13T03:20:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[27879]: cluster 2024-02-13T03:20:29.463854+0000 mon.a (mon.0) 503 : cluster [DBG] osdmap e45: 8 total, 7 up, 8 in 2024-02-13T03:20:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[27879]: audit 2024-02-13T03:20:29.464083+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[27879]: audit 2024-02-13T03:20:30.061867+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? v1:172.21.15.183:0/647717484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:20:30.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[19769]: cluster 2024-02-13T03:20:29.019918+0000 mgr.y (mgr.14146) 242 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:30.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[19769]: audit 2024-02-13T03:20:29.452667+0000 mon.a (mon.0) 501 : audit [INF] from='client.24313 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dc175e5-44c6-4454-81e3-92a465398a17"}]: dispatch 2024-02-13T03:20:30.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[19769]: audit 2024-02-13T03:20:29.454598+0000 mon.b (mon.1) 15 : audit [INF] from='client.? v1:172.21.15.183:0/1478714167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4dc175e5-44c6-4454-81e3-92a465398a17"}]: dispatch 2024-02-13T03:20:30.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[19769]: audit 2024-02-13T03:20:29.463664+0000 mon.a (mon.0) 502 : audit [INF] from='client.24313 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4dc175e5-44c6-4454-81e3-92a465398a17"}]': finished 2024-02-13T03:20:30.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[19769]: cluster 2024-02-13T03:20:29.463854+0000 mon.a (mon.0) 503 : cluster [DBG] osdmap e45: 8 total, 7 up, 8 in 2024-02-13T03:20:30.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[19769]: audit 2024-02-13T03:20:29.464083+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:30.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:30 smithi161 bash[19769]: audit 2024-02-13T03:20:30.061867+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? v1:172.21.15.183:0/647717484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T03:20:32.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:32 smithi183 bash[20824]: cluster 2024-02-13T03:20:31.020505+0000 mgr.y (mgr.14146) 243 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:32.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:32 smithi161 bash[19769]: cluster 2024-02-13T03:20:31.020505+0000 mgr.y (mgr.14146) 243 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:32.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:32 smithi161 bash[27879]: cluster 2024-02-13T03:20:31.020505+0000 mgr.y (mgr.14146) 243 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:34 smithi161 bash[27879]: cluster 2024-02-13T03:20:33.021089+0000 mgr.y (mgr.14146) 244 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:34.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:34 smithi161 bash[19769]: cluster 2024-02-13T03:20:33.021089+0000 mgr.y (mgr.14146) 244 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:34 smithi183 bash[20824]: cluster 2024-02-13T03:20:33.021089+0000 mgr.y (mgr.14146) 244 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:36.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:36 smithi161 bash[19769]: cluster 2024-02-13T03:20:35.021799+0000 mgr.y (mgr.14146) 245 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:36.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:36 smithi161 bash[27879]: cluster 2024-02-13T03:20:35.021799+0000 mgr.y (mgr.14146) 245 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:36 smithi183 bash[20824]: cluster 2024-02-13T03:20:35.021799+0000 mgr.y (mgr.14146) 245 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:38.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:38 smithi161 bash[19769]: cluster 2024-02-13T03:20:37.022418+0000 mgr.y (mgr.14146) 246 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:38 smithi161 bash[27879]: cluster 2024-02-13T03:20:37.022418+0000 mgr.y (mgr.14146) 246 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:38.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:38 smithi183 bash[20824]: cluster 2024-02-13T03:20:37.022418+0000 mgr.y (mgr.14146) 246 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:40.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:40 smithi161 bash[19769]: cluster 2024-02-13T03:20:39.022994+0000 mgr.y (mgr.14146) 247 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:40 smithi161 bash[27879]: cluster 2024-02-13T03:20:39.022994+0000 mgr.y (mgr.14146) 247 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:40.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:40 smithi183 bash[20824]: cluster 2024-02-13T03:20:39.022994+0000 mgr.y (mgr.14146) 247 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:42.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:42 smithi183 bash[20824]: cluster 2024-02-13T03:20:41.023571+0000 mgr.y (mgr.14146) 248 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:42.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:42 smithi183 bash[20824]: audit 2024-02-13T03:20:41.406230+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-13T03:20:42.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:42 smithi183 bash[20824]: audit 2024-02-13T03:20:41.408027+0000 mon.a (mon.0) 506 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:42.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:42 smithi183 bash[20824]: cephadm 2024-02-13T03:20:41.409540+0000 mgr.y (mgr.14146) 249 : cephadm [INF] Deploying daemon osd.7 on smithi183 2024-02-13T03:20:42.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:42 smithi161 bash[19769]: cluster 2024-02-13T03:20:41.023571+0000 mgr.y (mgr.14146) 248 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:42 smithi161 bash[19769]: audit 2024-02-13T03:20:41.406230+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-13T03:20:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:42 smithi161 bash[19769]: audit 2024-02-13T03:20:41.408027+0000 mon.a (mon.0) 506 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:42 smithi161 bash[19769]: cephadm 2024-02-13T03:20:41.409540+0000 mgr.y (mgr.14146) 249 : cephadm [INF] Deploying daemon osd.7 on smithi183 2024-02-13T03:20:42.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:42 smithi161 bash[27879]: cluster 2024-02-13T03:20:41.023571+0000 mgr.y (mgr.14146) 248 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:42.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:42 smithi161 bash[27879]: audit 2024-02-13T03:20:41.406230+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-13T03:20:42.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:42 smithi161 bash[27879]: audit 2024-02-13T03:20:41.408027+0000 mon.a (mon.0) 506 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:42.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:42 smithi161 bash[27879]: cephadm 2024-02-13T03:20:41.409540+0000 mgr.y (mgr.14146) 249 : cephadm [INF] Deploying daemon osd.7 on smithi183 2024-02-13T03:20:44.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:44 smithi161 bash[19769]: cluster 2024-02-13T03:20:43.024185+0000 mgr.y (mgr.14146) 250 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:44 smithi161 bash[27879]: cluster 2024-02-13T03:20:43.024185+0000 mgr.y (mgr.14146) 250 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:44.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:44 smithi183 bash[20824]: cluster 2024-02-13T03:20:43.024185+0000 mgr.y (mgr.14146) 250 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:46.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:46 smithi161 bash[19769]: cluster 2024-02-13T03:20:45.024772+0000 mgr.y (mgr.14146) 251 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:46.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:46 smithi161 bash[19769]: audit 2024-02-13T03:20:45.267846+0000 mon.a (mon.0) 507 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:46.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:46 smithi161 bash[19769]: audit 2024-02-13T03:20:45.299592+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:20:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:46 smithi161 bash[27879]: cluster 2024-02-13T03:20:45.024772+0000 mgr.y (mgr.14146) 251 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:46 smithi161 bash[27879]: audit 2024-02-13T03:20:45.267846+0000 mon.a (mon.0) 507 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:46 smithi161 bash[27879]: audit 2024-02-13T03:20:45.299592+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:20:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:46 smithi183 bash[20824]: cluster 2024-02-13T03:20:45.024772+0000 mgr.y (mgr.14146) 251 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:46 smithi183 bash[20824]: audit 2024-02-13T03:20:45.267846+0000 mon.a (mon.0) 507 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:46 smithi183 bash[20824]: audit 2024-02-13T03:20:45.299592+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:20:48.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:48 smithi161 bash[19769]: cluster 2024-02-13T03:20:47.025304+0000 mgr.y (mgr.14146) 252 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:48.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:48 smithi161 bash[27879]: cluster 2024-02-13T03:20:47.025304+0000 mgr.y (mgr.14146) 252 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:48.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:48 smithi183 bash[20824]: cluster 2024-02-13T03:20:47.025304+0000 mgr.y (mgr.14146) 252 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:50.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:50 smithi161 bash[19769]: cluster 2024-02-13T03:20:49.025888+0000 mgr.y (mgr.14146) 253 : cluster [DBG] pgmap v238: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:50.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:50 smithi161 bash[27879]: cluster 2024-02-13T03:20:49.025888+0000 mgr.y (mgr.14146) 253 : cluster [DBG] pgmap v238: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:50.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:50 smithi183 bash[20824]: cluster 2024-02-13T03:20:49.025888+0000 mgr.y (mgr.14146) 253 : cluster [DBG] pgmap v238: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:52.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:51 smithi161 bash[19769]: audit 2024-02-13T03:20:50.755286+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:52.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:51 smithi161 bash[19769]: audit 2024-02-13T03:20:50.756963+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:52.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:51 smithi161 bash[19769]: audit 2024-02-13T03:20:50.758924+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:20:52.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:51 smithi161 bash[19769]: audit 2024-02-13T03:20:50.767916+0000 mon.a (mon.0) 512 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:52.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:51 smithi161 bash[19769]: cluster 2024-02-13T03:20:51.026483+0000 mgr.y (mgr.14146) 254 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:52.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:51 smithi161 bash[27879]: audit 2024-02-13T03:20:50.755286+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:52.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:51 smithi161 bash[27879]: audit 2024-02-13T03:20:50.756963+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:52.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:51 smithi161 bash[27879]: audit 2024-02-13T03:20:50.758924+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:20:52.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:51 smithi161 bash[27879]: audit 2024-02-13T03:20:50.767916+0000 mon.a (mon.0) 512 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:52.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:51 smithi161 bash[27879]: cluster 2024-02-13T03:20:51.026483+0000 mgr.y (mgr.14146) 254 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:52.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:51 smithi183 bash[20824]: audit 2024-02-13T03:20:50.755286+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:52.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:51 smithi183 bash[20824]: audit 2024-02-13T03:20:50.756963+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:20:52.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:51 smithi183 bash[20824]: audit 2024-02-13T03:20:50.758924+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:20:52.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:51 smithi183 bash[20824]: audit 2024-02-13T03:20:50.767916+0000 mon.a (mon.0) 512 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:52.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:51 smithi183 bash[20824]: cluster 2024-02-13T03:20:51.026483+0000 mgr.y (mgr.14146) 254 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:53.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:52 smithi161 bash[27879]: audit 2024-02-13T03:20:52.008139+0000 mon.a (mon.0) 513 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-13T03:20:53.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:52 smithi161 bash[27879]: audit 2024-02-13T03:20:52.010466+0000 mon.b (mon.1) 17 : audit [INF] from='osd.7 v1:172.21.15.183:6812/1821141317' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-13T03:20:53.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:52 smithi161 bash[19769]: audit 2024-02-13T03:20:52.008139+0000 mon.a (mon.0) 513 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-13T03:20:53.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:52 smithi161 bash[19769]: audit 2024-02-13T03:20:52.010466+0000 mon.b (mon.1) 17 : audit [INF] from='osd.7 v1:172.21.15.183:6812/1821141317' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-13T03:20:53.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:52 smithi183 bash[20824]: audit 2024-02-13T03:20:52.008139+0000 mon.a (mon.0) 513 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-13T03:20:53.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:52 smithi183 bash[20824]: audit 2024-02-13T03:20:52.010466+0000 mon.b (mon.1) 17 : audit [INF] from='osd.7 v1:172.21.15.183:6812/1821141317' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-13T03:20:53.907 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:53.905+0000 7f3bfa7fc700 1 -- 172.21.15.183:0/2000239285 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 2842678453) 0x7f3c04060710 con 0x7f3bf00620f0 2024-02-13T03:20:53.907 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 7 on host 'smithi183' 2024-02-13T03:20:53.911 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:53.909+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/2000239285 >> v1:172.21.15.161:6800/1187535987 conn(0x7f3bf00620f0 legacy=0x7f3bf00645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:20:53.911 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:53.909+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/2000239285 >> v1:172.21.15.183:6789/0 conn(0x7f3c040fe8e0 legacy=0x7f3c041116f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:20:53.911 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:53.909+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/2000239285 shutdown_connections 2024-02-13T03:20:53.911 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:53.909+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/2000239285 >> 172.21.15.183:0/2000239285 conn(0x7f3c040f16b0 msgr2=0x7f3c040fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:20:53.911 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:53.909+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/2000239285 shutdown_connections 2024-02-13T03:20:53.911 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:20:53.909+0000 7f3c0b6d1700 1 -- 172.21.15.183:0/2000239285 wait complete. 2024-02-13T03:20:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:53 smithi161 bash[19769]: audit 2024-02-13T03:20:52.775201+0000 mon.a (mon.0) 514 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-13T03:20:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:53 smithi161 bash[19769]: cluster 2024-02-13T03:20:52.775385+0000 mon.a (mon.0) 515 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-02-13T03:20:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:53 smithi161 bash[19769]: audit 2024-02-13T03:20:52.776028+0000 mon.a (mon.0) 516 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:53 smithi161 bash[19769]: audit 2024-02-13T03:20:52.779866+0000 mon.b (mon.1) 18 : audit [INF] from='osd.7 v1:172.21.15.183:6812/1821141317' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:20:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:53 smithi161 bash[19769]: audit 2024-02-13T03:20:52.786744+0000 mon.a (mon.0) 517 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:20:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:53 smithi161 bash[19769]: cluster 2024-02-13T03:20:53.027049+0000 mgr.y (mgr.14146) 255 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:54.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:53 smithi161 bash[27879]: audit 2024-02-13T03:20:52.775201+0000 mon.a (mon.0) 514 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-13T03:20:54.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:53 smithi161 bash[27879]: cluster 2024-02-13T03:20:52.775385+0000 mon.a (mon.0) 515 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-02-13T03:20:54.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:53 smithi161 bash[27879]: audit 2024-02-13T03:20:52.776028+0000 mon.a (mon.0) 516 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:54.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:53 smithi161 bash[27879]: audit 2024-02-13T03:20:52.779866+0000 mon.b (mon.1) 18 : audit [INF] from='osd.7 v1:172.21.15.183:6812/1821141317' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:20:54.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:53 smithi161 bash[27879]: audit 2024-02-13T03:20:52.786744+0000 mon.a (mon.0) 517 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:20:54.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:53 smithi161 bash[27879]: cluster 2024-02-13T03:20:53.027049+0000 mgr.y (mgr.14146) 255 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:53 smithi183 bash[20824]: audit 2024-02-13T03:20:52.775201+0000 mon.a (mon.0) 514 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-13T03:20:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:53 smithi183 bash[20824]: cluster 2024-02-13T03:20:52.775385+0000 mon.a (mon.0) 515 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-02-13T03:20:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:53 smithi183 bash[20824]: audit 2024-02-13T03:20:52.776028+0000 mon.a (mon.0) 516 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:53 smithi183 bash[20824]: audit 2024-02-13T03:20:52.779866+0000 mon.b (mon.1) 18 : audit [INF] from='osd.7 v1:172.21.15.183:6812/1821141317' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:20:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:53 smithi183 bash[20824]: audit 2024-02-13T03:20:52.786744+0000 mon.a (mon.0) 517 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-02-13T03:20:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:53 smithi183 bash[20824]: cluster 2024-02-13T03:20:53.027049+0000 mgr.y (mgr.14146) 255 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-13T03:20:54.614 DEBUG:teuthology.orchestra.run.smithi183:osd.7> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.7.service 2024-02-13T03:20:54.616 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-02-13T03:20:54.617 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd stat -f json 2024-02-13T03:20:54.717 INFO:journalctl@ceph.osd.7.smithi183.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:20:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[27879]: audit 2024-02-13T03:20:53.789939+0000 mon.a (mon.0) 518 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-02-13T03:20:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[27879]: cluster 2024-02-13T03:20:53.790038+0000 mon.a (mon.0) 519 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-02-13T03:20:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[27879]: audit 2024-02-13T03:20:53.790568+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[27879]: audit 2024-02-13T03:20:53.797122+0000 mon.a (mon.0) 521 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[27879]: audit 2024-02-13T03:20:53.896962+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:20:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[27879]: audit 2024-02-13T03:20:53.903321+0000 mon.a (mon.0) 523 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[27879]: cluster 2024-02-13T03:20:54.791193+0000 mon.a (mon.0) 524 : cluster [INF] osd.7 v1:172.21.15.183:6812/1821141317 boot 2024-02-13T03:20:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[27879]: cluster 2024-02-13T03:20:54.791330+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e48: 8 total, 8 up, 8 in 2024-02-13T03:20:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[27879]: audit 2024-02-13T03:20:54.791580+0000 mon.a (mon.0) 526 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:54.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[19769]: audit 2024-02-13T03:20:53.789939+0000 mon.a (mon.0) 518 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-02-13T03:20:54.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[19769]: cluster 2024-02-13T03:20:53.790038+0000 mon.a (mon.0) 519 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-02-13T03:20:54.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[19769]: audit 2024-02-13T03:20:53.790568+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:54.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[19769]: audit 2024-02-13T03:20:53.797122+0000 mon.a (mon.0) 521 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:54.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[19769]: audit 2024-02-13T03:20:53.896962+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:20:54.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[19769]: audit 2024-02-13T03:20:53.903321+0000 mon.a (mon.0) 523 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:54.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[19769]: cluster 2024-02-13T03:20:54.791193+0000 mon.a (mon.0) 524 : cluster [INF] osd.7 v1:172.21.15.183:6812/1821141317 boot 2024-02-13T03:20:54.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[19769]: cluster 2024-02-13T03:20:54.791330+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e48: 8 total, 8 up, 8 in 2024-02-13T03:20:54.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:54 smithi161 bash[19769]: audit 2024-02-13T03:20:54.791580+0000 mon.a (mon.0) 526 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:54 smithi183 bash[20824]: audit 2024-02-13T03:20:53.789939+0000 mon.a (mon.0) 518 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-02-13T03:20:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:54 smithi183 bash[20824]: cluster 2024-02-13T03:20:53.790038+0000 mon.a (mon.0) 519 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-02-13T03:20:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:54 smithi183 bash[20824]: audit 2024-02-13T03:20:53.790568+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:54 smithi183 bash[20824]: audit 2024-02-13T03:20:53.797122+0000 mon.a (mon.0) 521 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:54 smithi183 bash[20824]: audit 2024-02-13T03:20:53.896962+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:20:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:54 smithi183 bash[20824]: audit 2024-02-13T03:20:53.903321+0000 mon.a (mon.0) 523 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:20:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:54 smithi183 bash[20824]: cluster 2024-02-13T03:20:54.791193+0000 mon.a (mon.0) 524 : cluster [INF] osd.7 v1:172.21.15.183:6812/1821141317 boot 2024-02-13T03:20:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:54 smithi183 bash[20824]: cluster 2024-02-13T03:20:54.791330+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e48: 8 total, 8 up, 8 in 2024-02-13T03:20:54.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:54 smithi183 bash[20824]: audit 2024-02-13T03:20:54.791580+0000 mon.a (mon.0) 526 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:20:56.092 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:55 smithi161 bash[19769]: cluster 2024-02-13T03:20:52.997409+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:20:56.092 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:55 smithi161 bash[19769]: cluster 2024-02-13T03:20:52.997521+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:20:56.093 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:55 smithi161 bash[19769]: cluster 2024-02-13T03:20:55.027667+0000 mgr.y (mgr.14146) 256 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:20:56.093 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:55 smithi161 bash[19769]: cluster 2024-02-13T03:20:55.793735+0000 mon.a (mon.0) 527 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-02-13T03:20:56.093 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:55 smithi161 bash[27879]: cluster 2024-02-13T03:20:52.997409+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:20:56.093 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:55 smithi161 bash[27879]: cluster 2024-02-13T03:20:52.997521+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:20:56.093 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:55 smithi161 bash[27879]: cluster 2024-02-13T03:20:55.027667+0000 mgr.y (mgr.14146) 256 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:20:56.093 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:55 smithi161 bash[27879]: cluster 2024-02-13T03:20:55.793735+0000 mon.a (mon.0) 527 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-02-13T03:20:56.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:55 smithi183 bash[20824]: cluster 2024-02-13T03:20:52.997409+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-13T03:20:56.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:55 smithi183 bash[20824]: cluster 2024-02-13T03:20:52.997521+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-13T03:20:56.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:55 smithi183 bash[20824]: cluster 2024-02-13T03:20:55.027667+0000 mgr.y (mgr.14146) 256 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:20:56.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:55 smithi183 bash[20824]: cluster 2024-02-13T03:20:55.793735+0000 mon.a (mon.0) 527 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-02-13T03:20:57.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53c4d03700 1 -- 172.21.15.161:0/4090813151 >> v1:172.21.15.161:6790/0 conn(0x7f53c00fbfa0 legacy=0x7f53c00fe390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:20:57.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53c4d03700 1 -- 172.21.15.161:0/4090813151 shutdown_connections 2024-02-13T03:20:57.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53c4d03700 1 -- 172.21.15.161:0/4090813151 >> 172.21.15.161:0/4090813151 conn(0x7f53c00f16d0 msgr2=0x7f53c00f3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:20:57.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53c4d03700 1 -- 172.21.15.161:0/4090813151 shutdown_connections 2024-02-13T03:20:57.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53c4d03700 1 -- 172.21.15.161:0/4090813151 wait complete. 2024-02-13T03:20:57.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53c4d03700 1 Processor -- start 2024-02-13T03:20:57.641 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53c4d03700 1 -- start start 2024-02-13T03:20:57.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53c4d03700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f53c01084f0 con 0x7f53c00f8f30 2024-02-13T03:20:57.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53c4d03700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f53c0113d50 con 0x7f53c00ffab0 2024-02-13T03:20:57.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53c4d03700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f53c0114f30 con 0x7f53c00fbfa0 2024-02-13T03:20:57.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53be59c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f53c00f8f30 0x7f53c0107920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:37038/0 (socket says 172.21.15.161:37038) 2024-02-13T03:20:57.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53be59c700 1 -- 172.21.15.161:0/4104590637 learned_addr learned my addr 172.21.15.161:0/4104590637 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:20:57.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039987753 0 0) 0x7f53c01084f0 con 0x7f53c00f8f30 2024-02-13T03:20:57.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53a80033f0 con 0x7f53c00f8f30 2024-02-13T03:20:57.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664053460 0 0) 0x7f53c0114f30 con 0x7f53c00fbfa0 2024-02-13T03:20:57.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c01084f0 con 0x7f53c00fbfa0 2024-02-13T03:20:57.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543981127 0 0) 0x7f53c0113d50 con 0x7f53c00ffab0 2024-02-13T03:20:57.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.635+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c0114f30 con 0x7f53c00ffab0 2024-02-13T03:20:57.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 361964530 0 0) 0x7f53a80033f0 con 0x7f53c00f8f30 2024-02-13T03:20:57.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53c0113d50 con 0x7f53c00f8f30 2024-02-13T03:20:57.643 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3323072804 0 0) 0x7f53c01084f0 con 0x7f53c00fbfa0 2024-02-13T03:20:57.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53a80033f0 con 0x7f53c00fbfa0 2024-02-13T03:20:57.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f53b0002990 con 0x7f53c00f8f30 2024-02-13T03:20:57.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f53b4002c60 con 0x7f53c00fbfa0 2024-02-13T03:20:57.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3731013879 0 0) 0x7f53c0114f30 con 0x7f53c00ffab0 2024-02-13T03:20:57.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53c01084f0 con 0x7f53c00ffab0 2024-02-13T03:20:57.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f53b8001b80 con 0x7f53c00ffab0 2024-02-13T03:20:57.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 519003360 0 0) 0x7f53c0113d50 con 0x7f53c00f8f30 2024-02-13T03:20:57.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 >> v1:172.21.15.161:6790/0 conn(0x7f53c00fbfa0 legacy=0x7f53c010def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:20:57.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 >> v1:172.21.15.183:6789/0 conn(0x7f53c00ffab0 legacy=0x7f53c0111620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:20:57.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c0116110 con 0x7f53c00f8f30 2024-02-13T03:20:57.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53c4d03700 1 -- 172.21.15.161:0/4104590637 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c0115120 con 0x7f53c00f8f30 2024-02-13T03:20:57.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53c4d03700 1 -- 172.21.15.161:0/4104590637 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c0115610 con 0x7f53c00f8f30 2024-02-13T03:20:57.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53c4d03700 1 -- 172.21.15.161:0/4104590637 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c00475e0 con 0x7f53c00f8f30 2024-02-13T03:20:57.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f53b0002b80 con 0x7f53c00f8f30 2024-02-13T03:20:57.646 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.639+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f53b0005a20 con 0x7f53c00f8f30 2024-02-13T03:20:57.647 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.643+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 1965425884 0 0) 0x7f53b00189b0 con 0x7f53c00f8f30 2024-02-13T03:20:57.648 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.643+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4054+0+0 (unknown 3714854458 0 0) 0x7f53b0077d50 con 0x7f53c00f8f30 2024-02-13T03:20:57.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.651+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f53b00485d0 con 0x7f53c00f8f30 2024-02-13T03:20:57.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.835+0000 7f53c4d03700 1 -- 172.21.15.161:0/4104590637 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f53c0105850 con 0x7f53c00f8f30 2024-02-13T03:20:57.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.835+0000 7f53af7fe700 1 -- 172.21.15.161:0/4104590637 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v50) v1 ==== 74+0+130 (unknown 24364321 0 1401565662) 0x7f53b004a500 con 0x7f53c00f8f30 2024-02-13T03:20:57.842 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-02-13T03:20:57.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.839+0000 7f53c4d03700 1 -- 172.21.15.161:0/4104590637 >> v1:172.21.15.161:6800/1187535987 conn(0x7f53a8066670 legacy=0x7f53a8068b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:20:57.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.839+0000 7f53c4d03700 1 -- 172.21.15.161:0/4104590637 >> v1:172.21.15.161:6789/0 conn(0x7f53c00f8f30 legacy=0x7f53c0107920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:20:57.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.839+0000 7f53c4d03700 1 -- 172.21.15.161:0/4104590637 shutdown_connections 2024-02-13T03:20:57.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.839+0000 7f53c4d03700 1 -- 172.21.15.161:0/4104590637 >> 172.21.15.161:0/4104590637 conn(0x7f53c00f16d0 msgr2=0x7f53c0101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:20:57.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.839+0000 7f53c4d03700 1 -- 172.21.15.161:0/4104590637 shutdown_connections 2024-02-13T03:20:57.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:20:57.839+0000 7f53c4d03700 1 -- 172.21.15.161:0/4104590637 wait complete. 2024-02-13T03:20:58.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:57 smithi161 bash[19769]: cluster 2024-02-13T03:20:56.796420+0000 mon.a (mon.0) 528 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-02-13T03:20:58.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:57 smithi161 bash[19769]: cluster 2024-02-13T03:20:57.028313+0000 mgr.y (mgr.14146) 257 : cluster [DBG] pgmap v247: 1 pgs: 1 remapped+peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:20:58.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:57 smithi161 bash[27879]: cluster 2024-02-13T03:20:56.796420+0000 mon.a (mon.0) 528 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-02-13T03:20:58.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:57 smithi161 bash[27879]: cluster 2024-02-13T03:20:57.028313+0000 mgr.y (mgr.14146) 257 : cluster [DBG] pgmap v247: 1 pgs: 1 remapped+peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:20:58.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:57 smithi183 bash[20824]: cluster 2024-02-13T03:20:56.796420+0000 mon.a (mon.0) 528 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-02-13T03:20:58.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:57 smithi183 bash[20824]: cluster 2024-02-13T03:20:57.028313+0000 mgr.y (mgr.14146) 257 : cluster [DBG] pgmap v247: 1 pgs: 1 remapped+peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:20:58.449 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":50,"num_osds":8,"num_up_osds":8,"osd_up_since":1707794454,"num_in_osds":8,"osd_in_since":1707794429,"num_remapped_pgs":0} 2024-02-13T03:20:58.450 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi161 2024-02-13T03:20:58.450 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi161=foo.a' 2024-02-13T03:20:59.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:58 smithi161 bash[27879]: cluster 2024-02-13T03:20:57.799493+0000 mon.a (mon.0) 529 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-02-13T03:20:59.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:58 smithi161 bash[27879]: audit 2024-02-13T03:20:57.840319+0000 mon.a (mon.0) 530 : audit [DBG] from='client.? v1:172.21.15.161:0/4104590637' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-13T03:20:59.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:58 smithi161 bash[19769]: cluster 2024-02-13T03:20:57.799493+0000 mon.a (mon.0) 529 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-02-13T03:20:59.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:58 smithi161 bash[19769]: audit 2024-02-13T03:20:57.840319+0000 mon.a (mon.0) 530 : audit [DBG] from='client.? v1:172.21.15.161:0/4104590637' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-13T03:20:59.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:58 smithi183 bash[20824]: cluster 2024-02-13T03:20:57.799493+0000 mon.a (mon.0) 529 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-02-13T03:20:59.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:58 smithi183 bash[20824]: audit 2024-02-13T03:20:57.840319+0000 mon.a (mon.0) 530 : audit [DBG] from='client.? v1:172.21.15.161:0/4104590637' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-13T03:21:00.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:20:59 smithi161 bash[19769]: cluster 2024-02-13T03:20:59.028968+0000 mgr.y (mgr.14146) 258 : cluster [DBG] pgmap v248: 1 pgs: 1 remapped+peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:00.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:20:59 smithi161 bash[27879]: cluster 2024-02-13T03:20:59.028968+0000 mgr.y (mgr.14146) 258 : cluster [DBG] pgmap v248: 1 pgs: 1 remapped+peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:00.199 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:20:59 smithi183 bash[20824]: cluster 2024-02-13T03:20:59.028968+0000 mgr.y (mgr.14146) 258 : cluster [DBG] pgmap v248: 1 pgs: 1 remapped+peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:01 smithi161 bash[27879]: cluster 2024-02-13T03:21:01.080787+0000 mon.a (mon.0) 531 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-13T03:21:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:01 smithi161 bash[27879]: cluster 2024-02-13T03:21:01.080857+0000 mon.a (mon.0) 532 : cluster [INF] Cluster is now healthy 2024-02-13T03:21:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:01 smithi161 bash[19769]: cluster 2024-02-13T03:21:01.080787+0000 mon.a (mon.0) 531 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-13T03:21:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:01 smithi161 bash[19769]: cluster 2024-02-13T03:21:01.080857+0000 mon.a (mon.0) 532 : cluster [INF] Cluster is now healthy 2024-02-13T03:21:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:01 smithi183 bash[20824]: cluster 2024-02-13T03:21:01.080787+0000 mon.a (mon.0) 531 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-13T03:21:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:01 smithi183 bash[20824]: cluster 2024-02-13T03:21:01.080857+0000 mon.a (mon.0) 532 : cluster [INF] Cluster is now healthy 2024-02-13T03:21:02.376 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.373+0000 7f6331b04700 1 -- 172.21.15.183:0/3750037992 >> v1:172.21.15.183:6789/0 conn(0x7f632c0fe8e0 legacy=0x7f632c100d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:02.376 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.373+0000 7f6331b04700 1 -- 172.21.15.183:0/3750037992 shutdown_connections 2024-02-13T03:21:02.376 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.373+0000 7f6331b04700 1 -- 172.21.15.183:0/3750037992 >> 172.21.15.183:0/3750037992 conn(0x7f632c0f16b0 msgr2=0x7f632c0f3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:02.376 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.373+0000 7f6331b04700 1 -- 172.21.15.183:0/3750037992 shutdown_connections 2024-02-13T03:21:02.376 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.373+0000 7f6331b04700 1 -- 172.21.15.183:0/3750037992 wait complete. 2024-02-13T03:21:02.377 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6331b04700 1 Processor -- start 2024-02-13T03:21:02.377 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6331b04700 1 -- start start 2024-02-13T03:21:02.377 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6331b04700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f632c114ea0 con 0x7f632c0fe8e0 2024-02-13T03:21:02.378 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6331b04700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f632c116080 con 0x7f632c0fad40 2024-02-13T03:21:02.378 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6331b04700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f632c117280 con 0x7f632c0f7db0 2024-02-13T03:21:02.378 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f632affd700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f632c0fad40 0x7f632c10fc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:44546/0 (socket says 172.21.15.183:44546) 2024-02-13T03:21:02.378 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f632affd700 1 -- 172.21.15.183:0/2360278415 learned_addr learned my addr 172.21.15.183:0/2360278415 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:21:02.378 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519623216 0 0) 0x7f632c116080 con 0x7f632c0fad40 2024-02-13T03:21:02.378 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6314003030 con 0x7f632c0fad40 2024-02-13T03:21:02.378 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085105630 0 0) 0x7f632c117280 con 0x7f632c0f7db0 2024-02-13T03:21:02.378 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f632c116080 con 0x7f632c0f7db0 2024-02-13T03:21:02.378 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011392949 0 0) 0x7f632c114ea0 con 0x7f632c0fe8e0 2024-02-13T03:21:02.378 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f632c117280 con 0x7f632c0fe8e0 2024-02-13T03:21:02.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1049858163 0 0) 0x7f6314003030 con 0x7f632c0fad40 2024-02-13T03:21:02.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f632c114ea0 con 0x7f632c0fad40 2024-02-13T03:21:02.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f6320001900 con 0x7f632c0fad40 2024-02-13T03:21:02.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 677487961 0 0) 0x7f632c116080 con 0x7f632c0f7db0 2024-02-13T03:21:02.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6314003030 con 0x7f632c0f7db0 2024-02-13T03:21:02.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f631c001c00 con 0x7f632c0f7db0 2024-02-13T03:21:02.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1614528076 0 0) 0x7f632c117280 con 0x7f632c0fe8e0 2024-02-13T03:21:02.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f632c116080 con 0x7f632c0fe8e0 2024-02-13T03:21:02.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f6324001070 con 0x7f632c0fe8e0 2024-02-13T03:21:02.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1127889849 0 0) 0x7f632c114ea0 con 0x7f632c0fad40 2024-02-13T03:21:02.381 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 >> v1:172.21.15.161:6790/0 conn(0x7f632c0f7db0 legacy=0x7f632c10c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:02.381 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 >> v1:172.21.15.161:6789/0 conn(0x7f632c0fe8e0 legacy=0x7f632c113560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:02.381 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c118480 con 0x7f632c0fad40 2024-02-13T03:21:02.381 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6331b04700 1 -- 172.21.15.183:0/2360278415 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632c117470 con 0x7f632c0fad40 2024-02-13T03:21:02.381 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6331b04700 1 -- 172.21.15.183:0/2360278415 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f632c117960 con 0x7f632c0fad40 2024-02-13T03:21:02.381 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f6320001c90 con 0x7f632c0fad40 2024-02-13T03:21:02.381 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f6320005bc0 con 0x7f632c0fad40 2024-02-13T03:21:02.381 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.377+0000 7f6331b04700 1 -- 172.21.15.183:0/2360278415 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6318004f20 con 0x7f632c0fad40 2024-02-13T03:21:02.382 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.381+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 1965425884 0 0) 0x7f6320005d20 con 0x7f632c0fad40 2024-02-13T03:21:02.382 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.381+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4054+0+0 (unknown 3714854458 0 0) 0x7f6320077c50 con 0x7f632c0fad40 2024-02-13T03:21:02.389 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.389+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f6320048680 con 0x7f632c0fad40 2024-02-13T03:21:02.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:02 smithi161 bash[27879]: cluster 2024-02-13T03:21:01.029632+0000 mgr.y (mgr.14146) 259 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:02.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:02 smithi161 bash[19769]: cluster 2024-02-13T03:21:01.029632+0000 mgr.y (mgr.14146) 259 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:02 smithi183 bash[20824]: cluster 2024-02-13T03:21:01.029632+0000 mgr.y (mgr.14146) 259 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:02.591 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.589+0000 7f6331b04700 1 -- 172.21.15.183:0/2360278415 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi161=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f6318000b90 con 0x7f63140622a0 2024-02-13T03:21:02.605 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.605+0000 7f6328ff9700 1 -- 172.21.15.183:0/2360278415 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (unknown 0 0 1123153589) 0x7f6318000b90 con 0x7f63140622a0 2024-02-13T03:21:02.605 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled rgw.foo.a update... 2024-02-13T03:21:02.607 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.605+0000 7f6331b04700 1 -- 172.21.15.183:0/2360278415 >> v1:172.21.15.161:6800/1187535987 conn(0x7f63140622a0 legacy=0x7f6314064760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:02.607 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.605+0000 7f6331b04700 1 -- 172.21.15.183:0/2360278415 >> v1:172.21.15.183:6789/0 conn(0x7f632c0fad40 legacy=0x7f632c10fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:02.607 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.605+0000 7f6331b04700 1 -- 172.21.15.183:0/2360278415 shutdown_connections 2024-02-13T03:21:02.607 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.605+0000 7f6331b04700 1 -- 172.21.15.183:0/2360278415 >> 172.21.15.183:0/2360278415 conn(0x7f632c0f16b0 msgr2=0x7f632c0f59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:02.607 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.605+0000 7f6331b04700 1 -- 172.21.15.183:0/2360278415 shutdown_connections 2024-02-13T03:21:02.607 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:02.605+0000 7f6331b04700 1 -- 172.21.15.183:0/2360278415 wait complete. 2024-02-13T03:21:03.414 DEBUG:teuthology.orchestra.run.smithi161:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@rgw.foo.a.service 2024-02-13T03:21:03.416 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi183 2024-02-13T03:21:03.416 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd pool create iscsi 3 3 replicated 2024-02-13T03:21:03.712 INFO:journalctl@ceph.rgw.foo.a.smithi161.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:21:03.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:03 smithi183 bash[20824]: audit 2024-02-13T03:21:02.591546+0000 mgr.y (mgr.14146) 260 : audit [DBG] from='client.24340 v1:172.21.15.183:0/2360278415' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi161=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:03.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:03 smithi183 bash[20824]: cephadm 2024-02-13T03:21:02.595421+0000 mgr.y (mgr.14146) 261 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi161=foo.a;count:1 2024-02-13T03:21:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:03 smithi183 bash[20824]: audit 2024-02-13T03:21:02.602360+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:03.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:03 smithi161 bash[19769]: audit 2024-02-13T03:21:02.591546+0000 mgr.y (mgr.14146) 260 : audit [DBG] from='client.24340 v1:172.21.15.183:0/2360278415' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi161=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:03.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:03 smithi161 bash[19769]: cephadm 2024-02-13T03:21:02.595421+0000 mgr.y (mgr.14146) 261 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi161=foo.a;count:1 2024-02-13T03:21:03.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:03 smithi161 bash[19769]: audit 2024-02-13T03:21:02.602360+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:03 smithi161 bash[27879]: audit 2024-02-13T03:21:02.591546+0000 mgr.y (mgr.14146) 260 : audit [DBG] from='client.24340 v1:172.21.15.183:0/2360278415' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi161=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:03 smithi161 bash[27879]: cephadm 2024-02-13T03:21:02.595421+0000 mgr.y (mgr.14146) 261 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi161=foo.a;count:1 2024-02-13T03:21:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:03 smithi161 bash[27879]: audit 2024-02-13T03:21:02.602360+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: cluster 2024-02-13T03:21:03.030205+0000 mgr.y (mgr.14146) 262 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.853116+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.855165+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.856765+0000 mon.a (mon.0) 536 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.858215+0000 mon.a (mon.0) 537 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.859619+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.866875+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.868591+0000 mon.a (mon.0) 540 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.870100+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:21:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.878013+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.887552+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.888568+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-13T03:21:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.895245+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-13T03:21:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.903126+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[27879]: audit 2024-02-13T03:21:03.904511+0000 mon.a (mon.0) 547 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:04.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: cluster 2024-02-13T03:21:03.030205+0000 mgr.y (mgr.14146) 262 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:04.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.853116+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.855165+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:04.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.856765+0000 mon.a (mon.0) 536 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:04.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.858215+0000 mon.a (mon.0) 537 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:04.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.859619+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:04.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.866875+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.868591+0000 mon.a (mon.0) 540 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:04.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.870100+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:21:04.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.878013+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.887552+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.888568+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-13T03:21:04.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.895245+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-13T03:21:04.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.903126+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:04 smithi161 bash[19769]: audit 2024-02-13T03:21:03.904511+0000 mon.a (mon.0) 547 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:04.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: cluster 2024-02-13T03:21:03.030205+0000 mgr.y (mgr.14146) 262 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:04.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.853116+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.855165+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:04.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.856765+0000 mon.a (mon.0) 536 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:04.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.858215+0000 mon.a (mon.0) 537 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:04.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.859619+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:04.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.866875+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.868591+0000 mon.a (mon.0) 540 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:04.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.870100+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:21:04.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.878013+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.887552+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.888568+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-13T03:21:04.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.895245+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-13T03:21:04.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.903126+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:04.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:04 smithi183 bash[20824]: audit 2024-02-13T03:21:03.904511+0000 mon.a (mon.0) 547 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:05 smithi161 bash[19769]: cephadm 2024-02-13T03:21:03.842429+0000 mgr.y (mgr.14146) 263 : cephadm [INF] Detected new or changed devices on smithi183 2024-02-13T03:21:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:05 smithi161 bash[19769]: cephadm 2024-02-13T03:21:03.860783+0000 mgr.y (mgr.14146) 264 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 4329M 2024-02-13T03:21:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:05 smithi161 bash[19769]: cephadm 2024-02-13T03:21:03.882056+0000 mgr.y (mgr.14146) 265 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi161=foo.a;count:1 2024-02-13T03:21:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:05 smithi161 bash[19769]: cephadm 2024-02-13T03:21:03.905771+0000 mgr.y (mgr.14146) 266 : cephadm [INF] Deploying daemon rgw.foo.a on smithi161 2024-02-13T03:21:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:05 smithi161 bash[27879]: cephadm 2024-02-13T03:21:03.842429+0000 mgr.y (mgr.14146) 263 : cephadm [INF] Detected new or changed devices on smithi183 2024-02-13T03:21:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:05 smithi161 bash[27879]: cephadm 2024-02-13T03:21:03.860783+0000 mgr.y (mgr.14146) 264 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 4329M 2024-02-13T03:21:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:05 smithi161 bash[27879]: cephadm 2024-02-13T03:21:03.882056+0000 mgr.y (mgr.14146) 265 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi161=foo.a;count:1 2024-02-13T03:21:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:05 smithi161 bash[27879]: cephadm 2024-02-13T03:21:03.905771+0000 mgr.y (mgr.14146) 266 : cephadm [INF] Deploying daemon rgw.foo.a on smithi161 2024-02-13T03:21:05.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:05 smithi183 bash[20824]: cephadm 2024-02-13T03:21:03.842429+0000 mgr.y (mgr.14146) 263 : cephadm [INF] Detected new or changed devices on smithi183 2024-02-13T03:21:05.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:05 smithi183 bash[20824]: cephadm 2024-02-13T03:21:03.860783+0000 mgr.y (mgr.14146) 264 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 4329M 2024-02-13T03:21:05.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:05 smithi183 bash[20824]: cephadm 2024-02-13T03:21:03.882056+0000 mgr.y (mgr.14146) 265 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi161=foo.a;count:1 2024-02-13T03:21:05.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:05 smithi183 bash[20824]: cephadm 2024-02-13T03:21:03.905771+0000 mgr.y (mgr.14146) 266 : cephadm [INF] Deploying daemon rgw.foo.a on smithi161 2024-02-13T03:21:06.439 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.437+0000 7f54648a7700 1 -- 172.21.15.183:0/2814548369 <== mon.1 v1:172.21.15.183:6789/0 5 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5458003360 con 0x7f54600fe8d0 2024-02-13T03:21:06.439 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.437+0000 7f5467b0d700 1 -- 172.21.15.183:0/2814548369 >> v1:172.21.15.183:6789/0 conn(0x7f54600fe8d0 legacy=0x7f5460100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:06.439 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.437+0000 7f5467b0d700 1 -- 172.21.15.183:0/2814548369 shutdown_connections 2024-02-13T03:21:06.440 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.437+0000 7f5467b0d700 1 -- 172.21.15.183:0/2814548369 >> 172.21.15.183:0/2814548369 conn(0x7f54600f1f50 msgr2=0x7f54600f4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:06.440 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.437+0000 7f5467b0d700 1 -- 172.21.15.183:0/2814548369 shutdown_connections 2024-02-13T03:21:06.440 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.437+0000 7f5467b0d700 1 -- 172.21.15.183:0/2814548369 wait complete. 2024-02-13T03:21:06.441 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.437+0000 7f5467b0d700 1 Processor -- start 2024-02-13T03:21:06.441 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.437+0000 7f5467b0d700 1 -- start start 2024-02-13T03:21:06.441 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5467b0d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f54600fc520 con 0x7f54600f67d0 2024-02-13T03:21:06.441 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5467b0d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f54600fc6b0 con 0x7f54600faca0 2024-02-13T03:21:06.441 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5467b0d700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f54600fc880 con 0x7f54600fe8d0 2024-02-13T03:21:06.441 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f54650a8700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f54600faca0 0x7f5460110540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:44550/0 (socket says 172.21.15.183:44550) 2024-02-13T03:21:06.441 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f54660aa700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f54600fe8d0 0x7f5460113c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.183:35894/0 (socket says 172.21.15.183:35894) 2024-02-13T03:21:06.441 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f54658a9700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f54600f67d0 0x7f54600fbd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.183:55046/0 (socket says 172.21.15.183:55046) 2024-02-13T03:21:06.441 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f54650a8700 1 -- 172.21.15.183:0/3464483321 learned_addr learned my addr 172.21.15.183:0/3464483321 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:21:06.443 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417685770 0 0) 0x7f54600fc520 con 0x7f54600f67d0 2024-02-13T03:21:06.443 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f544c0033f0 con 0x7f54600f67d0 2024-02-13T03:21:06.443 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987261917 0 0) 0x7f54600fc880 con 0x7f54600fe8d0 2024-02-13T03:21:06.443 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54600fc520 con 0x7f54600fe8d0 2024-02-13T03:21:06.443 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532383049 0 0) 0x7f54600fc6b0 con 0x7f54600faca0 2024-02-13T03:21:06.443 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54600fc880 con 0x7f54600faca0 2024-02-13T03:21:06.443 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 163358243 0 0) 0x7f54600fc520 con 0x7f54600fe8d0 2024-02-13T03:21:06.443 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f54600fc6b0 con 0x7f54600fe8d0 2024-02-13T03:21:06.443 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2497701422 0 0) 0x7f54600fc880 con 0x7f54600faca0 2024-02-13T03:21:06.443 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f54600fc520 con 0x7f54600faca0 2024-02-13T03:21:06.443 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5458002240 con 0x7f54600fe8d0 2024-02-13T03:21:06.444 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f545c0028b0 con 0x7f54600faca0 2024-02-13T03:21:06.444 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 343734191 0 0) 0x7f544c0033f0 con 0x7f54600f67d0 2024-02-13T03:21:06.444 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f54600fc880 con 0x7f54600f67d0 2024-02-13T03:21:06.444 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5454001a50 con 0x7f54600f67d0 2024-02-13T03:21:06.444 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2338952662 0 0) 0x7f54600fc6b0 con 0x7f54600fe8d0 2024-02-13T03:21:06.444 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 >> v1:172.21.15.183:6789/0 conn(0x7f54600faca0 legacy=0x7f5460110540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:06.444 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 >> v1:172.21.15.161:6789/0 conn(0x7f54600f67d0 legacy=0x7f54600fbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:06.444 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54601183c0 con 0x7f54600fe8d0 2024-02-13T03:21:06.444 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5467b0d700 1 -- 172.21.15.183:0/3464483321 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5460115390 con 0x7f54600fe8d0 2024-02-13T03:21:06.444 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5467b0d700 1 -- 172.21.15.183:0/3464483321 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5460115880 con 0x7f54600fe8d0 2024-02-13T03:21:06.444 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f54580030f0 con 0x7f54600fe8d0 2024-02-13T03:21:06.444 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5458003b40 con 0x7f54600fe8d0 2024-02-13T03:21:06.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5467b0d700 1 -- 172.21.15.183:0/3464483321 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5444004f20 con 0x7f54600fe8d0 2024-02-13T03:21:06.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.441+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 1965425884 0 0) 0x7f5458003ca0 con 0x7f54600fe8d0 2024-02-13T03:21:06.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.445+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4054+0+0 (unknown 3714854458 0 0) 0x7f54580780c0 con 0x7f54600fe8d0 2024-02-13T03:21:06.458 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.457+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f54580489a0 con 0x7f54600fe8d0 2024-02-13T03:21:06.641 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:06.641+0000 7f5467b0d700 1 -- 172.21.15.183:0/3464483321 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7f5444005750 con 0x7f54600fe8d0 2024-02-13T03:21:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:06 smithi161 bash[27879]: cluster 2024-02-13T03:21:05.030809+0000 mgr.y (mgr.14146) 267 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:06 smithi161 bash[27879]: audit 2024-02-13T03:21:06.418765+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:06 smithi161 bash[27879]: audit 2024-02-13T03:21:06.420760+0000 mon.a (mon.0) 549 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:21:06.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:06 smithi161 bash[19769]: cluster 2024-02-13T03:21:05.030809+0000 mgr.y (mgr.14146) 267 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:06.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:06 smithi161 bash[19769]: audit 2024-02-13T03:21:06.418765+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:06.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:06 smithi161 bash[19769]: audit 2024-02-13T03:21:06.420760+0000 mon.a (mon.0) 549 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:21:06.677 INFO:journalctl@ceph.rgw.foo.a.smithi161.stdout:Feb 13 03:21:06 smithi161 systemd[1]: Started Ceph rgw.foo.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:21:06.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:06 smithi183 bash[20824]: cluster 2024-02-13T03:21:05.030809+0000 mgr.y (mgr.14146) 267 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:06.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:06 smithi183 bash[20824]: audit 2024-02-13T03:21:06.418765+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:06.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:06 smithi183 bash[20824]: audit 2024-02-13T03:21:06.420760+0000 mon.a (mon.0) 549 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:21:07.637 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:07.637+0000 7f5452ffd700 1 -- 172.21.15.183:0/3464483321 <== mon.2 v1:172.21.15.161:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'iscsi' created v51) v1 ==== 154+0+0 (unknown 2828387674 0 0) 0x7f545804a8d0 con 0x7f54600fe8d0 2024-02-13T03:21:07.641 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:07.641+0000 7f5467b0d700 1 -- 172.21.15.183:0/3464483321 >> v1:172.21.15.161:6800/1187535987 conn(0x7f544c0622a0 legacy=0x7f544c064760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:07.641 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:07.641+0000 7f5467b0d700 1 -- 172.21.15.183:0/3464483321 >> v1:172.21.15.161:6790/0 conn(0x7f54600fe8d0 legacy=0x7f5460113c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:07.641 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:07.641+0000 7f5467b0d700 1 -- 172.21.15.183:0/3464483321 shutdown_connections 2024-02-13T03:21:07.641 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:07.641+0000 7f5467b0d700 1 -- 172.21.15.183:0/3464483321 >> 172.21.15.183:0/3464483321 conn(0x7f54600f1f50 msgr2=0x7f5460101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:07.641 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:07.641+0000 7f5467b0d700 1 -- 172.21.15.183:0/3464483321 shutdown_connections 2024-02-13T03:21:07.641 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:07.641+0000 7f5467b0d700 1 -- 172.21.15.183:0/3464483321 wait complete. 2024-02-13T03:21:07.642 INFO:teuthology.orchestra.run.smithi183.stderr:pool 'iscsi' created 2024-02-13T03:21:07.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:07 smithi161 bash[27879]: audit 2024-02-13T03:21:06.640511+0000 mon.c (mon.2) 5 : audit [INF] from='client.? v1:172.21.15.183:0/3464483321' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-13T03:21:07.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:07 smithi161 bash[27879]: audit 2024-02-13T03:21:06.641154+0000 mon.a (mon.0) 550 : audit [INF] from='client.24347 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-13T03:21:07.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:07 smithi161 bash[19769]: audit 2024-02-13T03:21:06.640511+0000 mon.c (mon.2) 5 : audit [INF] from='client.? v1:172.21.15.183:0/3464483321' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-13T03:21:07.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:07 smithi161 bash[19769]: audit 2024-02-13T03:21:06.641154+0000 mon.a (mon.0) 550 : audit [INF] from='client.24347 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-13T03:21:07.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:07 smithi183 bash[20824]: audit 2024-02-13T03:21:06.640511+0000 mon.c (mon.2) 5 : audit [INF] from='client.? v1:172.21.15.183:0/3464483321' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-13T03:21:07.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:07 smithi183 bash[20824]: audit 2024-02-13T03:21:06.641154+0000 mon.a (mon.0) 550 : audit [INF] from='client.24347 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-13T03:21:08.285 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd pool application enable iscsi rbd 2024-02-13T03:21:08.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:08 smithi161 bash[19769]: cluster 2024-02-13T03:21:07.031398+0000 mgr.y (mgr.14146) 268 : cluster [DBG] pgmap v252: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:08.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:08 smithi161 bash[19769]: audit 2024-02-13T03:21:07.635125+0000 mon.a (mon.0) 551 : audit [INF] from='client.24347 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-13T03:21:08.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:08 smithi161 bash[19769]: cluster 2024-02-13T03:21:07.635237+0000 mon.a (mon.0) 552 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-02-13T03:21:08.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:08 smithi161 bash[27879]: cluster 2024-02-13T03:21:07.031398+0000 mgr.y (mgr.14146) 268 : cluster [DBG] pgmap v252: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:08.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:08 smithi161 bash[27879]: audit 2024-02-13T03:21:07.635125+0000 mon.a (mon.0) 551 : audit [INF] from='client.24347 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-13T03:21:08.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:08 smithi161 bash[27879]: cluster 2024-02-13T03:21:07.635237+0000 mon.a (mon.0) 552 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-02-13T03:21:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:08 smithi183 bash[20824]: cluster 2024-02-13T03:21:07.031398+0000 mgr.y (mgr.14146) 268 : cluster [DBG] pgmap v252: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:08 smithi183 bash[20824]: audit 2024-02-13T03:21:07.635125+0000 mon.a (mon.0) 551 : audit [INF] from='client.24347 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-13T03:21:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:08 smithi183 bash[20824]: cluster 2024-02-13T03:21:07.635237+0000 mon.a (mon.0) 552 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-02-13T03:21:09.673 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:09 smithi183 bash[20824]: cluster 2024-02-13T03:21:08.653158+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-02-13T03:21:09.673 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:09 smithi183 bash[20824]: audit 2024-02-13T03:21:08.658646+0000 mon.a (mon.0) 554 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-13T03:21:09.674 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:09 smithi183 bash[20824]: cluster 2024-02-13T03:21:09.032009+0000 mgr.y (mgr.14146) 269 : cluster [DBG] pgmap v255: 36 pgs: 35 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:09.674 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:09 smithi183 bash[20824]: audit 2024-02-13T03:21:09.150483+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:09.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:09 smithi161 bash[19769]: cluster 2024-02-13T03:21:08.653158+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-02-13T03:21:09.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:09 smithi161 bash[19769]: audit 2024-02-13T03:21:08.658646+0000 mon.a (mon.0) 554 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-13T03:21:09.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:09 smithi161 bash[19769]: cluster 2024-02-13T03:21:09.032009+0000 mgr.y (mgr.14146) 269 : cluster [DBG] pgmap v255: 36 pgs: 35 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:09.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:09 smithi161 bash[19769]: audit 2024-02-13T03:21:09.150483+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:09 smithi161 bash[27879]: cluster 2024-02-13T03:21:08.653158+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-02-13T03:21:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:09 smithi161 bash[27879]: audit 2024-02-13T03:21:08.658646+0000 mon.a (mon.0) 554 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-13T03:21:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:09 smithi161 bash[27879]: cluster 2024-02-13T03:21:09.032009+0000 mgr.y (mgr.14146) 269 : cluster [DBG] pgmap v255: 36 pgs: 35 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:09 smithi161 bash[27879]: audit 2024-02-13T03:21:09.150483+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:10.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:10 smithi161 bash[19769]: audit 2024-02-13T03:21:09.657589+0000 mon.a (mon.0) 556 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-13T03:21:10.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:10 smithi161 bash[19769]: cluster 2024-02-13T03:21:09.657709+0000 mon.a (mon.0) 557 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-13T03:21:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:10 smithi161 bash[27879]: audit 2024-02-13T03:21:09.657589+0000 mon.a (mon.0) 556 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-13T03:21:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:10 smithi161 bash[27879]: cluster 2024-02-13T03:21:09.657709+0000 mon.a (mon.0) 557 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-13T03:21:10.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:10 smithi183 bash[20824]: audit 2024-02-13T03:21:09.657589+0000 mon.a (mon.0) 556 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-13T03:21:10.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:10 smithi183 bash[20824]: cluster 2024-02-13T03:21:09.657709+0000 mon.a (mon.0) 557 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-13T03:21:11.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.281+0000 7ff9bb38a700 1 -- 172.21.15.183:0/1347050655 >> v1:172.21.15.183:6789/0 conn(0x7ff9b40fac90 legacy=0x7ff9b40fd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:11.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.281+0000 7ff9bb38a700 1 -- 172.21.15.183:0/1347050655 shutdown_connections 2024-02-13T03:21:11.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.281+0000 7ff9bb38a700 1 -- 172.21.15.183:0/1347050655 >> 172.21.15.183:0/1347050655 conn(0x7ff9b40f1f20 msgr2=0x7ff9b40f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:11.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.281+0000 7ff9bb38a700 1 -- 172.21.15.183:0/1347050655 shutdown_connections 2024-02-13T03:21:11.284 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.281+0000 7ff9bb38a700 1 -- 172.21.15.183:0/1347050655 wait complete. 2024-02-13T03:21:11.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.281+0000 7ff9bb38a700 1 Processor -- start 2024-02-13T03:21:11.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.281+0000 7ff9bb38a700 1 -- start start 2024-02-13T03:21:11.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9bb38a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9b4107550 con 0x7ff9b40f67c0 2024-02-13T03:21:11.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9bb38a700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9b4112db0 con 0x7ff9b40fac90 2024-02-13T03:21:11.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9bb38a700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9b4113f90 con 0x7ff9b40fe8c0 2024-02-13T03:21:11.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9b8925700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7ff9b40fac90 0x7ff9b410cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:56368/0 (socket says 172.21.15.183:56368) 2024-02-13T03:21:11.285 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9b8925700 1 -- 172.21.15.183:0/4021478822 learned_addr learned my addr 172.21.15.183:0/4021478822 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:21:11.286 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293313196 0 0) 0x7ff9b4112db0 con 0x7ff9b40fac90 2024-02-13T03:21:11.286 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9a0003030 con 0x7ff9b40fac90 2024-02-13T03:21:11.286 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738863558 0 0) 0x7ff9b4107550 con 0x7ff9b40f67c0 2024-02-13T03:21:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b4112db0 con 0x7ff9b40f67c0 2024-02-13T03:21:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3975622862 0 0) 0x7ff9a0003030 con 0x7ff9b40fac90 2024-02-13T03:21:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9b4107550 con 0x7ff9b40fac90 2024-02-13T03:21:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 73223263 0 0) 0x7ff9b4113f90 con 0x7ff9b40fe8c0 2024-02-13T03:21:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9a0003030 con 0x7ff9b40fe8c0 2024-02-13T03:21:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7ff9b00015d0 con 0x7ff9b40fac90 2024-02-13T03:21:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 260696030 0 0) 0x7ff9b4107550 con 0x7ff9b40fac90 2024-02-13T03:21:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 >> v1:172.21.15.161:6790/0 conn(0x7ff9b40fe8c0 legacy=0x7ff9b4110680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 >> v1:172.21.15.161:6789/0 conn(0x7ff9b40f67c0 legacy=0x7ff9b4106980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b4115170 con 0x7ff9b40fac90 2024-02-13T03:21:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9bb38a700 1 -- 172.21.15.183:0/4021478822 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b4111e00 con 0x7ff9b40fac90 2024-02-13T03:21:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9bb38a700 1 -- 172.21.15.183:0/4021478822 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b4112270 con 0x7ff9b40fac90 2024-02-13T03:21:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7ff9b0001160 con 0x7ff9b40fac90 2024-02-13T03:21:11.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.285+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7ff9b00036b0 con 0x7ff9b40fac90 2024-02-13T03:21:11.288 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.289+0000 7ff9bb38a700 1 -- 172.21.15.183:0/4021478822 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff998004f50 con 0x7ff9b40fac90 2024-02-13T03:21:11.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.289+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 1965425884 0 0) 0x7ff9b0001bf0 con 0x7ff9b40fac90 2024-02-13T03:21:11.289 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.289+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(54..54 src has 1..54) v4 ==== 5111+0+0 (unknown 953110299 0 0) 0x7ff9b00782e0 con 0x7ff9b40fac90 2024-02-13T03:21:11.295 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.293+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7ff9b0048800 con 0x7ff9b40fac90 2024-02-13T03:21:11.481 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.480+0000 7ff9bb38a700 1 -- 172.21.15.183:0/4021478822 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"} v 0) v1 -- 0x7ff998005780 con 0x7ff9b40fac90 2024-02-13T03:21:11.669 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.668+0000 7ff9a67fc700 1 -- 172.21.15.183:0/4021478822 <== mon.1 v1:172.21.15.183:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]=0 enabled application 'rbd' on pool 'iscsi' v55) v1 ==== 147+0+0 (unknown 3078387298 0 0) 0x7ff9b004a730 con 0x7ff9b40fac90 2024-02-13T03:21:11.678 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.676+0000 7ff9bb38a700 1 -- 172.21.15.183:0/4021478822 >> v1:172.21.15.161:6800/1187535987 conn(0x7ff9a0062210 legacy=0x7ff9a00646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:11.678 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.676+0000 7ff9bb38a700 1 -- 172.21.15.183:0/4021478822 >> v1:172.21.15.183:6789/0 conn(0x7ff9b40fac90 legacy=0x7ff9b410cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:11.679 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.676+0000 7ff9bb38a700 1 -- 172.21.15.183:0/4021478822 shutdown_connections 2024-02-13T03:21:11.680 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.676+0000 7ff9bb38a700 1 -- 172.21.15.183:0/4021478822 >> 172.21.15.183:0/4021478822 conn(0x7ff9b40f1f20 msgr2=0x7ff9b40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:11.681 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.680+0000 7ff9bb38a700 1 -- 172.21.15.183:0/4021478822 shutdown_connections 2024-02-13T03:21:11.682 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:11.680+0000 7ff9bb38a700 1 -- 172.21.15.183:0/4021478822 wait complete. 2024-02-13T03:21:11.683 INFO:teuthology.orchestra.run.smithi183.stderr:enabled application 'rbd' on pool 'iscsi' 2024-02-13T03:21:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:11 smithi183 bash[20824]: cluster 2024-02-13T03:21:10.666272+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-13T03:21:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:11 smithi183 bash[20824]: audit 2024-02-13T03:21:10.668472+0000 mon.a (mon.0) 559 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T03:21:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:11 smithi183 bash[20824]: cluster 2024-02-13T03:21:11.032664+0000 mgr.y (mgr.14146) 270 : cluster [DBG] pgmap v258: 68 pgs: 16 active+clean, 1 creating+activating, 2 creating+peering, 49 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:11 smithi183 bash[20824]: audit 2024-02-13T03:21:11.481216+0000 mon.a (mon.0) 560 : audit [INF] from='client.24361 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-13T03:21:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:11 smithi183 bash[20824]: audit 2024-02-13T03:21:11.483496+0000 mon.b (mon.1) 19 : audit [INF] from='client.? v1:172.21.15.183:0/4021478822' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-13T03:21:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:11 smithi183 bash[20824]: audit 2024-02-13T03:21:11.664270+0000 mon.a (mon.0) 561 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T03:21:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:11 smithi183 bash[20824]: audit 2024-02-13T03:21:11.664589+0000 mon.a (mon.0) 562 : audit [INF] from='client.24361 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-13T03:21:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:11 smithi183 bash[20824]: cluster 2024-02-13T03:21:11.664697+0000 mon.a (mon.0) 563 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-13T03:21:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[27879]: cluster 2024-02-13T03:21:10.666272+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-13T03:21:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[27879]: audit 2024-02-13T03:21:10.668472+0000 mon.a (mon.0) 559 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T03:21:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[27879]: cluster 2024-02-13T03:21:11.032664+0000 mgr.y (mgr.14146) 270 : cluster [DBG] pgmap v258: 68 pgs: 16 active+clean, 1 creating+activating, 2 creating+peering, 49 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[27879]: audit 2024-02-13T03:21:11.481216+0000 mon.a (mon.0) 560 : audit [INF] from='client.24361 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-13T03:21:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[27879]: audit 2024-02-13T03:21:11.483496+0000 mon.b (mon.1) 19 : audit [INF] from='client.? v1:172.21.15.183:0/4021478822' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-13T03:21:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[27879]: audit 2024-02-13T03:21:11.664270+0000 mon.a (mon.0) 561 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T03:21:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[27879]: audit 2024-02-13T03:21:11.664589+0000 mon.a (mon.0) 562 : audit [INF] from='client.24361 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-13T03:21:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[27879]: cluster 2024-02-13T03:21:11.664697+0000 mon.a (mon.0) 563 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-13T03:21:12.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[19769]: cluster 2024-02-13T03:21:10.666272+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-13T03:21:12.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[19769]: audit 2024-02-13T03:21:10.668472+0000 mon.a (mon.0) 559 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T03:21:12.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[19769]: cluster 2024-02-13T03:21:11.032664+0000 mgr.y (mgr.14146) 270 : cluster [DBG] pgmap v258: 68 pgs: 16 active+clean, 1 creating+activating, 2 creating+peering, 49 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:12.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[19769]: audit 2024-02-13T03:21:11.481216+0000 mon.a (mon.0) 560 : audit [INF] from='client.24361 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-13T03:21:12.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[19769]: audit 2024-02-13T03:21:11.483496+0000 mon.b (mon.1) 19 : audit [INF] from='client.? v1:172.21.15.183:0/4021478822' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-13T03:21:12.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[19769]: audit 2024-02-13T03:21:11.664270+0000 mon.a (mon.0) 561 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T03:21:12.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[19769]: audit 2024-02-13T03:21:11.664589+0000 mon.a (mon.0) 562 : audit [INF] from='client.24361 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-13T03:21:12.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:11 smithi161 bash[19769]: cluster 2024-02-13T03:21:11.664697+0000 mon.a (mon.0) 563 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-13T03:21:12.379 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch apply iscsi iscsi user password --placement '1;smithi183=iscsi.a' 2024-02-13T03:21:13.789 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:13 smithi183 bash[20824]: cluster 2024-02-13T03:21:12.665239+0000 mon.a (mon.0) 564 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-13T03:21:13.789 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:13 smithi183 bash[20824]: audit 2024-02-13T03:21:12.667795+0000 mon.a (mon.0) 565 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T03:21:13.789 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:13 smithi183 bash[20824]: audit 2024-02-13T03:21:12.836628+0000 mon.a (mon.0) 566 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:13.790 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:13 smithi183 bash[20824]: audit 2024-02-13T03:21:12.838513+0000 mon.a (mon.0) 567 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:13.790 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:13 smithi183 bash[20824]: audit 2024-02-13T03:21:12.840229+0000 mon.a (mon.0) 568 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:21:13.790 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:13 smithi183 bash[20824]: cephadm 2024-02-13T03:21:12.852374+0000 mgr.y (mgr.14146) 271 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-13T03:21:13.790 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:13 smithi183 bash[20824]: cluster 2024-02-13T03:21:13.033389+0000 mgr.y (mgr.14146) 272 : cluster [DBG] pgmap v261: 100 pgs: 41 active+clean, 1 creating+activating, 17 creating+peering, 41 unknown; 1.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 745 B/s wr, 2 op/s 2024-02-13T03:21:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[19769]: cluster 2024-02-13T03:21:12.665239+0000 mon.a (mon.0) 564 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-13T03:21:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[19769]: audit 2024-02-13T03:21:12.667795+0000 mon.a (mon.0) 565 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T03:21:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[19769]: audit 2024-02-13T03:21:12.836628+0000 mon.a (mon.0) 566 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[19769]: audit 2024-02-13T03:21:12.838513+0000 mon.a (mon.0) 567 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[19769]: audit 2024-02-13T03:21:12.840229+0000 mon.a (mon.0) 568 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:21:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[19769]: cephadm 2024-02-13T03:21:12.852374+0000 mgr.y (mgr.14146) 271 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-13T03:21:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[19769]: cluster 2024-02-13T03:21:13.033389+0000 mgr.y (mgr.14146) 272 : cluster [DBG] pgmap v261: 100 pgs: 41 active+clean, 1 creating+activating, 17 creating+peering, 41 unknown; 1.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 745 B/s wr, 2 op/s 2024-02-13T03:21:13.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[27879]: cluster 2024-02-13T03:21:12.665239+0000 mon.a (mon.0) 564 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-13T03:21:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[27879]: audit 2024-02-13T03:21:12.667795+0000 mon.a (mon.0) 565 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T03:21:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[27879]: audit 2024-02-13T03:21:12.836628+0000 mon.a (mon.0) 566 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[27879]: audit 2024-02-13T03:21:12.838513+0000 mon.a (mon.0) 567 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[27879]: audit 2024-02-13T03:21:12.840229+0000 mon.a (mon.0) 568 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:21:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[27879]: cephadm 2024-02-13T03:21:12.852374+0000 mgr.y (mgr.14146) 271 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-13T03:21:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:13 smithi161 bash[27879]: cluster 2024-02-13T03:21:13.033389+0000 mgr.y (mgr.14146) 272 : cluster [DBG] pgmap v261: 100 pgs: 41 active+clean, 1 creating+activating, 17 creating+peering, 41 unknown; 1.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 745 B/s wr, 2 op/s 2024-02-13T03:21:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:14 smithi183 bash[20824]: audit 2024-02-13T03:21:13.678518+0000 mon.a (mon.0) 569 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T03:21:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:14 smithi183 bash[20824]: cluster 2024-02-13T03:21:13.678778+0000 mon.a (mon.0) 570 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-13T03:21:15.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:14 smithi161 bash[19769]: audit 2024-02-13T03:21:13.678518+0000 mon.a (mon.0) 569 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T03:21:15.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:14 smithi161 bash[19769]: cluster 2024-02-13T03:21:13.678778+0000 mon.a (mon.0) 570 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-13T03:21:15.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:14 smithi161 bash[27879]: audit 2024-02-13T03:21:13.678518+0000 mon.a (mon.0) 569 : audit [INF] from='client.? v1:172.21.15.161:0/2747509086' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T03:21:15.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:14 smithi161 bash[27879]: cluster 2024-02-13T03:21:13.678778+0000 mon.a (mon.0) 570 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-13T03:21:15.625 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7cb499e700 1 -- 172.21.15.183:0/1694715667 >> v1:172.21.15.183:6789/0 conn(0x7f7cb00f7560 legacy=0x7f7cb01040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:15.626 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7cb499e700 1 -- 172.21.15.183:0/1694715667 shutdown_connections 2024-02-13T03:21:15.626 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7cb499e700 1 -- 172.21.15.183:0/1694715667 >> 172.21.15.183:0/1694715667 conn(0x7f7cb00f16b0 msgr2=0x7f7cb00f3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:15.626 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7cb499e700 1 -- 172.21.15.183:0/1694715667 shutdown_connections 2024-02-13T03:21:15.626 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7cb499e700 1 -- 172.21.15.183:0/1694715667 wait complete. 2024-02-13T03:21:15.626 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7cb499e700 1 Processor -- start 2024-02-13T03:21:15.626 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7cb499e700 1 -- start start 2024-02-13T03:21:15.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7cb499e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7cb010a830 con 0x7f7cb00f69c0 2024-02-13T03:21:15.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7cb499e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7cb0116080 con 0x7f7cb00f7d80 2024-02-13T03:21:15.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7cb499e700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7cb0117220 con 0x7f7cb00f7560 2024-02-13T03:21:15.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7caed9d700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f7cb00f7d80 0x7f7cb0113950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:56398/0 (socket says 172.21.15.183:56398) 2024-02-13T03:21:15.627 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7caed9d700 1 -- 172.21.15.183:0/3239285817 learned_addr learned my addr 172.21.15.183:0/3239285817 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:21:15.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395182119 0 0) 0x7f7cb0116080 con 0x7f7cb00f7d80 2024-02-13T03:21:15.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c9c003030 con 0x7f7cb00f7d80 2024-02-13T03:21:15.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192486207 0 0) 0x7f7cb010a830 con 0x7f7cb00f69c0 2024-02-13T03:21:15.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cb0116080 con 0x7f7cb00f69c0 2024-02-13T03:21:15.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498770803 0 0) 0x7f7cb0117220 con 0x7f7cb00f7560 2024-02-13T03:21:15.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cb010a830 con 0x7f7cb00f7560 2024-02-13T03:21:15.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 889020824 0 0) 0x7f7c9c003030 con 0x7f7cb00f7d80 2024-02-13T03:21:15.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7cb0117220 con 0x7f7cb00f7d80 2024-02-13T03:21:15.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f7ca80015d0 con 0x7f7cb00f7d80 2024-02-13T03:21:15.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1761933454 0 0) 0x7f7cb0116080 con 0x7f7cb00f69c0 2024-02-13T03:21:15.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7c9c003030 con 0x7f7cb00f69c0 2024-02-13T03:21:15.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f7ca0001c00 con 0x7f7cb00f69c0 2024-02-13T03:21:15.629 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 461736261 0 0) 0x7f7cb010a830 con 0x7f7cb00f7560 2024-02-13T03:21:15.630 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7cb0116080 con 0x7f7cb00f7560 2024-02-13T03:21:15.630 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.624+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3490676871 0 0) 0x7f7cb0117220 con 0x7f7cb00f7d80 2024-02-13T03:21:15.630 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.628+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 >> v1:172.21.15.161:6790/0 conn(0x7f7cb00f7560 legacy=0x7f7cb0110220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:15.630 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.628+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 >> v1:172.21.15.161:6789/0 conn(0x7f7cb00f69c0 legacy=0x7f7cb0109c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:15.630 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.628+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb0118420 con 0x7f7cb00f7d80 2024-02-13T03:21:15.630 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.628+0000 7f7cb499e700 1 -- 172.21.15.183:0/3239285817 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb01150d0 con 0x7f7cb00f7d80 2024-02-13T03:21:15.630 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.628+0000 7f7cb499e700 1 -- 172.21.15.183:0/3239285817 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb01155c0 con 0x7f7cb00f7d80 2024-02-13T03:21:15.630 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.628+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f7ca8001bf0 con 0x7f7cb00f7d80 2024-02-13T03:21:15.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.628+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f7ca8003b40 con 0x7f7cb00f7d80 2024-02-13T03:21:15.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.628+0000 7f7cb499e700 1 -- 172.21.15.183:0/3239285817 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb0060710 con 0x7f7cb00f7d80 2024-02-13T03:21:15.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.628+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 1965425884 0 0) 0x7f7ca8002160 con 0x7f7cb00f7d80 2024-02-13T03:21:15.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.632+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(58..58 src has 1..58) v4 ==== 5857+0+0 (unknown 1933609235 0 0) 0x7f7ca8078620 con 0x7f7cb00f7d80 2024-02-13T03:21:15.640 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.636+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f7cb0060710 con 0x7f7cb00f7d80 2024-02-13T03:21:15.706 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:15 smithi183 bash[20824]: cluster 2024-02-13T03:21:14.696038+0000 mon.a (mon.0) 571 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-13T03:21:15.706 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:15 smithi183 bash[20824]: audit 2024-02-13T03:21:14.701137+0000 mon.a (mon.0) 572 : audit [INF] from='client.? v1:172.21.15.161:0/3068980097' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T03:21:15.706 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:15 smithi183 bash[20824]: audit 2024-02-13T03:21:14.705635+0000 mon.c (mon.2) 6 : audit [INF] from='client.? v1:172.21.15.161:0/629238266' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T03:21:15.706 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:15 smithi183 bash[20824]: audit 2024-02-13T03:21:14.706017+0000 mon.a (mon.0) 573 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T03:21:15.706 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:15 smithi183 bash[20824]: cluster 2024-02-13T03:21:15.034279+0000 mgr.y (mgr.14146) 273 : cluster [DBG] pgmap v264: 132 pgs: 54 active+clean, 21 creating+peering, 57 unknown; 1.4 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-13T03:21:15.847 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.844+0000 7f7cb499e700 1 -- 172.21.15.183:0/3239285817 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi183=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cb01159e0 con 0x7f7c9c062280 2024-02-13T03:21:15.862 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.860+0000 7f7ca77fe700 1 -- 172.21.15.183:0/3239285817 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (unknown 0 0 3282307126) 0x7f7cb01159e0 con 0x7f7c9c062280 2024-02-13T03:21:15.862 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled iscsi.iscsi update... 2024-02-13T03:21:15.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.864+0000 7f7cb499e700 1 -- 172.21.15.183:0/3239285817 >> v1:172.21.15.161:6800/1187535987 conn(0x7f7c9c062280 legacy=0x7f7c9c064740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:15.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.864+0000 7f7cb499e700 1 -- 172.21.15.183:0/3239285817 >> v1:172.21.15.183:6789/0 conn(0x7f7cb00f7d80 legacy=0x7f7cb0113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:15.865 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.864+0000 7f7cb499e700 1 -- 172.21.15.183:0/3239285817 shutdown_connections 2024-02-13T03:21:15.866 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.864+0000 7f7cb499e700 1 -- 172.21.15.183:0/3239285817 >> 172.21.15.183:0/3239285817 conn(0x7f7cb00f16b0 msgr2=0x7f7cb00f3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:15.866 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.864+0000 7f7cb499e700 1 -- 172.21.15.183:0/3239285817 shutdown_connections 2024-02-13T03:21:15.866 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:15.864+0000 7f7cb499e700 1 -- 172.21.15.183:0/3239285817 wait complete. 2024-02-13T03:21:16.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:15 smithi161 bash[19769]: cluster 2024-02-13T03:21:14.696038+0000 mon.a (mon.0) 571 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-13T03:21:16.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:15 smithi161 bash[19769]: audit 2024-02-13T03:21:14.701137+0000 mon.a (mon.0) 572 : audit [INF] from='client.? v1:172.21.15.161:0/3068980097' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T03:21:16.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:15 smithi161 bash[19769]: audit 2024-02-13T03:21:14.705635+0000 mon.c (mon.2) 6 : audit [INF] from='client.? v1:172.21.15.161:0/629238266' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T03:21:16.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:15 smithi161 bash[19769]: audit 2024-02-13T03:21:14.706017+0000 mon.a (mon.0) 573 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T03:21:16.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:15 smithi161 bash[19769]: cluster 2024-02-13T03:21:15.034279+0000 mgr.y (mgr.14146) 273 : cluster [DBG] pgmap v264: 132 pgs: 54 active+clean, 21 creating+peering, 57 unknown; 1.4 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-13T03:21:16.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:15 smithi161 bash[27879]: cluster 2024-02-13T03:21:14.696038+0000 mon.a (mon.0) 571 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-13T03:21:16.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:15 smithi161 bash[27879]: audit 2024-02-13T03:21:14.701137+0000 mon.a (mon.0) 572 : audit [INF] from='client.? v1:172.21.15.161:0/3068980097' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T03:21:16.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:15 smithi161 bash[27879]: audit 2024-02-13T03:21:14.705635+0000 mon.c (mon.2) 6 : audit [INF] from='client.? v1:172.21.15.161:0/629238266' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T03:21:16.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:15 smithi161 bash[27879]: audit 2024-02-13T03:21:14.706017+0000 mon.a (mon.0) 573 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T03:21:16.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:15 smithi161 bash[27879]: cluster 2024-02-13T03:21:15.034279+0000 mgr.y (mgr.14146) 273 : cluster [DBG] pgmap v264: 132 pgs: 54 active+clean, 21 creating+peering, 57 unknown; 1.4 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-13T03:21:16.535 DEBUG:teuthology.orchestra.run.smithi183:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@iscsi.iscsi.a.service 2024-02-13T03:21:16.538 INFO:tasks.cephadm:Adding prometheus.a on smithi183 2024-02-13T03:21:16.538 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch apply prometheus '1;smithi183=a' 2024-02-13T03:21:16.722 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:21:16.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:16 smithi183 bash[20824]: audit 2024-02-13T03:21:15.696784+0000 mon.a (mon.0) 574 : audit [INF] from='client.? v1:172.21.15.161:0/3068980097' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T03:21:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:16 smithi183 bash[20824]: audit 2024-02-13T03:21:15.697059+0000 mon.a (mon.0) 575 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T03:21:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:16 smithi183 bash[20824]: cluster 2024-02-13T03:21:15.697157+0000 mon.a (mon.0) 576 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-13T03:21:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:16 smithi183 bash[20824]: audit 2024-02-13T03:21:15.699827+0000 mon.a (mon.0) 577 : audit [INF] from='client.? v1:172.21.15.161:0/3068980097' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T03:21:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:16 smithi183 bash[20824]: audit 2024-02-13T03:21:15.709047+0000 mon.c (mon.2) 7 : audit [INF] from='client.? v1:172.21.15.161:0/629238266' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T03:21:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:16 smithi183 bash[20824]: audit 2024-02-13T03:21:15.709690+0000 mon.a (mon.0) 578 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T03:21:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:16 smithi183 bash[20824]: audit 2024-02-13T03:21:15.847548+0000 mgr.y (mgr.14146) 274 : audit [DBG] from='client.24403 v1:172.21.15.183:0/3239285817' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi183=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:16 smithi183 bash[20824]: cephadm 2024-02-13T03:21:15.853928+0000 mgr.y (mgr.14146) 275 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi183=iscsi.a;count:1 2024-02-13T03:21:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:16 smithi183 bash[20824]: audit 2024-02-13T03:21:15.859544+0000 mon.a (mon.0) 579 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:16 smithi183 bash[20824]: audit 2024-02-13T03:21:16.699015+0000 mon.a (mon.0) 580 : audit [INF] from='client.? v1:172.21.15.161:0/3068980097' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T03:21:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:16 smithi183 bash[20824]: audit 2024-02-13T03:21:16.699213+0000 mon.a (mon.0) 581 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T03:21:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:16 smithi183 bash[20824]: cluster 2024-02-13T03:21:16.699384+0000 mon.a (mon.0) 582 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-13T03:21:17.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[27879]: audit 2024-02-13T03:21:15.696784+0000 mon.a (mon.0) 574 : audit [INF] from='client.? v1:172.21.15.161:0/3068980097' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T03:21:17.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[27879]: audit 2024-02-13T03:21:15.697059+0000 mon.a (mon.0) 575 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T03:21:17.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[27879]: cluster 2024-02-13T03:21:15.697157+0000 mon.a (mon.0) 576 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-13T03:21:17.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[27879]: audit 2024-02-13T03:21:15.699827+0000 mon.a (mon.0) 577 : audit [INF] from='client.? v1:172.21.15.161:0/3068980097' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T03:21:17.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[27879]: audit 2024-02-13T03:21:15.709047+0000 mon.c (mon.2) 7 : audit [INF] from='client.? v1:172.21.15.161:0/629238266' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T03:21:17.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[27879]: audit 2024-02-13T03:21:15.709690+0000 mon.a (mon.0) 578 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T03:21:17.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[27879]: audit 2024-02-13T03:21:15.847548+0000 mgr.y (mgr.14146) 274 : audit [DBG] from='client.24403 v1:172.21.15.183:0/3239285817' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi183=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:17.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[27879]: cephadm 2024-02-13T03:21:15.853928+0000 mgr.y (mgr.14146) 275 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi183=iscsi.a;count:1 2024-02-13T03:21:17.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[27879]: audit 2024-02-13T03:21:15.859544+0000 mon.a (mon.0) 579 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:17.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[27879]: audit 2024-02-13T03:21:16.699015+0000 mon.a (mon.0) 580 : audit [INF] from='client.? v1:172.21.15.161:0/3068980097' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T03:21:17.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[27879]: audit 2024-02-13T03:21:16.699213+0000 mon.a (mon.0) 581 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T03:21:17.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[27879]: cluster 2024-02-13T03:21:16.699384+0000 mon.a (mon.0) 582 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-13T03:21:17.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[19769]: audit 2024-02-13T03:21:15.696784+0000 mon.a (mon.0) 574 : audit [INF] from='client.? v1:172.21.15.161:0/3068980097' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T03:21:17.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[19769]: audit 2024-02-13T03:21:15.697059+0000 mon.a (mon.0) 575 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T03:21:17.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[19769]: cluster 2024-02-13T03:21:15.697157+0000 mon.a (mon.0) 576 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-13T03:21:17.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[19769]: audit 2024-02-13T03:21:15.699827+0000 mon.a (mon.0) 577 : audit [INF] from='client.? v1:172.21.15.161:0/3068980097' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T03:21:17.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[19769]: audit 2024-02-13T03:21:15.709047+0000 mon.c (mon.2) 7 : audit [INF] from='client.? v1:172.21.15.161:0/629238266' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T03:21:17.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[19769]: audit 2024-02-13T03:21:15.709690+0000 mon.a (mon.0) 578 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T03:21:17.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[19769]: audit 2024-02-13T03:21:15.847548+0000 mgr.y (mgr.14146) 274 : audit [DBG] from='client.24403 v1:172.21.15.183:0/3239285817' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi183=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:17.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[19769]: cephadm 2024-02-13T03:21:15.853928+0000 mgr.y (mgr.14146) 275 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi183=iscsi.a;count:1 2024-02-13T03:21:17.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[19769]: audit 2024-02-13T03:21:15.859544+0000 mon.a (mon.0) 579 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:17.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[19769]: audit 2024-02-13T03:21:16.699015+0000 mon.a (mon.0) 580 : audit [INF] from='client.? v1:172.21.15.161:0/3068980097' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T03:21:17.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[19769]: audit 2024-02-13T03:21:16.699213+0000 mon.a (mon.0) 581 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T03:21:17.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:16 smithi161 bash[19769]: cluster 2024-02-13T03:21:16.699384+0000 mon.a (mon.0) 582 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-13T03:21:18.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[19769]: audit 2024-02-13T03:21:16.897426+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:18.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[19769]: audit 2024-02-13T03:21:16.905335+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:18.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[19769]: audit 2024-02-13T03:21:16.913364+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:18.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[19769]: cephadm 2024-02-13T03:21:16.920269+0000 mgr.y (mgr.14146) 276 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-13T03:21:18.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[19769]: audit 2024-02-13T03:21:16.931242+0000 mon.a (mon.0) 586 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-13T03:21:18.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[19769]: audit 2024-02-13T03:21:16.935553+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-13T03:21:18.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[19769]: audit 2024-02-13T03:21:16.948690+0000 mon.a (mon.0) 588 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:18.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[19769]: cephadm 2024-02-13T03:21:16.949959+0000 mgr.y (mgr.14146) 277 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi183 2024-02-13T03:21:18.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[19769]: cluster 2024-02-13T03:21:17.034836+0000 mgr.y (mgr.14146) 278 : cluster [DBG] pgmap v267: 132 pgs: 107 active+clean, 18 creating+peering, 7 unknown; 1.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-02-13T03:21:18.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[27879]: audit 2024-02-13T03:21:16.897426+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:18.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[27879]: audit 2024-02-13T03:21:16.905335+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:18.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[27879]: audit 2024-02-13T03:21:16.913364+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:18.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[27879]: cephadm 2024-02-13T03:21:16.920269+0000 mgr.y (mgr.14146) 276 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-13T03:21:18.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[27879]: audit 2024-02-13T03:21:16.931242+0000 mon.a (mon.0) 586 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-13T03:21:18.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[27879]: audit 2024-02-13T03:21:16.935553+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-13T03:21:18.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[27879]: audit 2024-02-13T03:21:16.948690+0000 mon.a (mon.0) 588 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:18.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[27879]: cephadm 2024-02-13T03:21:16.949959+0000 mgr.y (mgr.14146) 277 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi183 2024-02-13T03:21:18.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:17 smithi161 bash[27879]: cluster 2024-02-13T03:21:17.034836+0000 mgr.y (mgr.14146) 278 : cluster [DBG] pgmap v267: 132 pgs: 107 active+clean, 18 creating+peering, 7 unknown; 1.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-02-13T03:21:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:17 smithi183 bash[20824]: audit 2024-02-13T03:21:16.897426+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:17 smithi183 bash[20824]: audit 2024-02-13T03:21:16.905335+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:17 smithi183 bash[20824]: audit 2024-02-13T03:21:16.913364+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:17 smithi183 bash[20824]: cephadm 2024-02-13T03:21:16.920269+0000 mgr.y (mgr.14146) 276 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-13T03:21:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:17 smithi183 bash[20824]: audit 2024-02-13T03:21:16.931242+0000 mon.a (mon.0) 586 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-13T03:21:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:17 smithi183 bash[20824]: audit 2024-02-13T03:21:16.935553+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-13T03:21:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:17 smithi183 bash[20824]: audit 2024-02-13T03:21:16.948690+0000 mon.a (mon.0) 588 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:17 smithi183 bash[20824]: cephadm 2024-02-13T03:21:16.949959+0000 mgr.y (mgr.14146) 277 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi183 2024-02-13T03:21:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:17 smithi183 bash[20824]: cluster 2024-02-13T03:21:17.034836+0000 mgr.y (mgr.14146) 278 : cluster [DBG] pgmap v267: 132 pgs: 107 active+clean, 18 creating+peering, 7 unknown; 1.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-02-13T03:21:19.665 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:19 smithi183 bash[20824]: cluster 2024-02-13T03:21:18.258129+0000 mon.a (mon.0) 589 : cluster [DBG] mgrmap e15: y(active, since 7m), standbys: x 2024-02-13T03:21:19.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:19 smithi161 bash[19769]: cluster 2024-02-13T03:21:18.258129+0000 mon.a (mon.0) 589 : cluster [DBG] mgrmap e15: y(active, since 7m), standbys: x 2024-02-13T03:21:19.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:19 smithi161 bash[27879]: cluster 2024-02-13T03:21:18.258129+0000 mon.a (mon.0) 589 : cluster [DBG] mgrmap e15: y(active, since 7m), standbys: x 2024-02-13T03:21:20.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:21:19 smithi183 systemd[1]: Started Ceph iscsi.iscsi.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:21:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:20 smithi161 bash[27879]: cluster 2024-02-13T03:21:19.035751+0000 mgr.y (mgr.14146) 279 : cluster [DBG] pgmap v268: 132 pgs: 118 active+clean, 14 creating+peering; 2.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 38 KiB/s rd, 2.1 KiB/s wr, 78 op/s 2024-02-13T03:21:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:20 smithi161 bash[27879]: audit 2024-02-13T03:21:19.998938+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:20 smithi161 bash[27879]: audit 2024-02-13T03:21:20.002028+0000 mon.a (mon.0) 591 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:21:20.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:20 smithi161 bash[19769]: cluster 2024-02-13T03:21:19.035751+0000 mgr.y (mgr.14146) 279 : cluster [DBG] pgmap v268: 132 pgs: 118 active+clean, 14 creating+peering; 2.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 38 KiB/s rd, 2.1 KiB/s wr, 78 op/s 2024-02-13T03:21:20.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:20 smithi161 bash[19769]: audit 2024-02-13T03:21:19.998938+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:20.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:20 smithi161 bash[19769]: audit 2024-02-13T03:21:20.002028+0000 mon.a (mon.0) 591 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:21:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:20 smithi183 bash[20824]: cluster 2024-02-13T03:21:19.035751+0000 mgr.y (mgr.14146) 279 : cluster [DBG] pgmap v268: 132 pgs: 118 active+clean, 14 creating+peering; 2.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 38 KiB/s rd, 2.1 KiB/s wr, 78 op/s 2024-02-13T03:21:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:20 smithi183 bash[20824]: audit 2024-02-13T03:21:19.998938+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:20 smithi183 bash[20824]: audit 2024-02-13T03:21:20.002028+0000 mon.a (mon.0) 591 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:21:21.381 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03ceba3700 1 -- 172.21.15.183:0/1565013088 >> v1:172.21.15.183:6789/0 conn(0x7f03c80fac90 legacy=0x7f03c80fd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:21.381 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03ceba3700 1 -- 172.21.15.183:0/1565013088 shutdown_connections 2024-02-13T03:21:21.381 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03ceba3700 1 -- 172.21.15.183:0/1565013088 >> 172.21.15.183:0/1565013088 conn(0x7f03c80f1f00 msgr2=0x7f03c80f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:21.381 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03ceba3700 1 -- 172.21.15.183:0/1565013088 shutdown_connections 2024-02-13T03:21:21.381 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03ceba3700 1 -- 172.21.15.183:0/1565013088 wait complete. 2024-02-13T03:21:21.383 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03ceba3700 1 Processor -- start 2024-02-13T03:21:21.383 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03ceba3700 1 -- start start 2024-02-13T03:21:21.383 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03ceba3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f03c8114de0 con 0x7f03c80fac90 2024-02-13T03:21:21.383 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03ceba3700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f03c8115fc0 con 0x7f03c80f67c0 2024-02-13T03:21:21.383 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03ceba3700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f03c81171c0 con 0x7f03c80fe8c0 2024-02-13T03:21:21.383 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03cc93f700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f03c80f67c0 0x7f03c8100810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:51606/0 (socket says 172.21.15.183:51606) 2024-02-13T03:21:21.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03cc93f700 1 -- 172.21.15.183:0/2761718404 learned_addr learned my addr 172.21.15.183:0/2761718404 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:21:21.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886436102 0 0) 0x7f03c8115fc0 con 0x7f03c80f67c0 2024-02-13T03:21:21.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a8003030 con 0x7f03c80f67c0 2024-02-13T03:21:21.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2155280927 0 0) 0x7f03a8003030 con 0x7f03c80f67c0 2024-02-13T03:21:21.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f03c8115fc0 con 0x7f03c80f67c0 2024-02-13T03:21:21.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321806304 0 0) 0x7f03c8114de0 con 0x7f03c80fac90 2024-02-13T03:21:21.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a8003030 con 0x7f03c80fac90 2024-02-13T03:21:21.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f03b8001950 con 0x7f03c80f67c0 2024-02-13T03:21:21.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509297381 0 0) 0x7f03c81171c0 con 0x7f03c80fe8c0 2024-02-13T03:21:21.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03c8114de0 con 0x7f03c80fe8c0 2024-02-13T03:21:21.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2662111056 0 0) 0x7f03c8115fc0 con 0x7f03c80f67c0 2024-02-13T03:21:21.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 >> v1:172.21.15.161:6790/0 conn(0x7f03c80fe8c0 legacy=0x7f03c81134a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:21.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 >> v1:172.21.15.161:6789/0 conn(0x7f03c80fac90 legacy=0x7f03c810fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:21.385 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c81183c0 con 0x7f03c80f67c0 2024-02-13T03:21:21.385 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03ceba3700 1 -- 172.21.15.183:0/2761718404 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c8114fd0 con 0x7f03c80f67c0 2024-02-13T03:21:21.385 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03ceba3700 1 -- 172.21.15.183:0/2761718404 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c81154c0 con 0x7f03c80f67c0 2024-02-13T03:21:21.385 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f03b8001b80 con 0x7f03c80f67c0 2024-02-13T03:21:21.385 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f03b8005bb0 con 0x7f03c80f67c0 2024-02-13T03:21:21.385 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.380+0000 7f03ceba3700 1 -- 172.21.15.183:0/2761718404 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac004f50 con 0x7f03c80f67c0 2024-02-13T03:21:21.385 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.384+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 15) v1 ==== 77081+0+0 (unknown 3266367394 0 0) 0x7f03b8002230 con 0x7f03c80f67c0 2024-02-13T03:21:21.386 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.384+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5884+0+0 (unknown 1000442856 0 0) 0x7f03b80786b0 con 0x7f03c80f67c0 2024-02-13T03:21:21.393 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.392+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f03b8048660 con 0x7f03c80f67c0 2024-02-13T03:21:21.600 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.596+0000 7f03ceba3700 1 -- 172.21.15.183:0/2761718404 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi183=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f03ac000bc0 con 0x7f03a80621f0 2024-02-13T03:21:21.617 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.616+0000 7f03c5ffb700 1 -- 172.21.15.183:0/2761718404 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (unknown 0 0 1342662408) 0x7f03ac000bc0 con 0x7f03a80621f0 2024-02-13T03:21:21.617 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled prometheus update... 2024-02-13T03:21:21.620 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.616+0000 7f03ceba3700 1 -- 172.21.15.183:0/2761718404 >> v1:172.21.15.161:6800/1187535987 conn(0x7f03a80621f0 legacy=0x7f03a80646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:21.621 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.616+0000 7f03ceba3700 1 -- 172.21.15.183:0/2761718404 >> v1:172.21.15.183:6789/0 conn(0x7f03c80f67c0 legacy=0x7f03c8100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:21.621 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.616+0000 7f03ceba3700 1 -- 172.21.15.183:0/2761718404 shutdown_connections 2024-02-13T03:21:21.621 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.616+0000 7f03ceba3700 1 -- 172.21.15.183:0/2761718404 >> 172.21.15.183:0/2761718404 conn(0x7f03c80f1f00 msgr2=0x7f03c80605b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:21.621 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.616+0000 7f03ceba3700 1 -- 172.21.15.183:0/2761718404 shutdown_connections 2024-02-13T03:21:21.621 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:21.620+0000 7f03ceba3700 1 -- 172.21.15.183:0/2761718404 wait complete. 2024-02-13T03:21:21.938 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:21:21 smithi183 bash[43494]: log file path now is '/var/log/tcmu-runner.log' 2024-02-13T03:21:22.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:21:21 smithi183 bash[43494]: debug Started the configuration object watcher 2024-02-13T03:21:22.237 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:21:21 smithi183 bash[43494]: debug Checking for config object changes every 1s 2024-02-13T03:21:22.237 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:21:21 smithi183 bash[43494]: debug Processing osd blocklist entries for this node 2024-02-13T03:21:22.421 DEBUG:teuthology.orchestra.run.smithi183:prometheus.a> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@prometheus.a.service 2024-02-13T03:21:22.424 INFO:tasks.cephadm:Adding node-exporter.a on smithi161 2024-02-13T03:21:22.424 INFO:tasks.cephadm:Adding node-exporter.b on smithi183 2024-02-13T03:21:22.424 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch apply node-exporter '2;smithi161=a;smithi183=b' 2024-02-13T03:21:22.442 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:21:22.616 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:21:22 smithi183 bash[43494]: debug Reading the configuration object to update local LIO configuration 2024-02-13T03:21:22.617 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:21:22 smithi183 bash[43494]: debug Configuration does not have an entry for this host(smithi183.front.sepia.ceph.com) - nothing to define to LIO 2024-02-13T03:21:22.617 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:21:22 smithi183 bash[43494]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-02-13T03:21:22.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:22 smithi161 bash[19769]: cluster 2024-02-13T03:21:21.036937+0000 mgr.y (mgr.14146) 280 : cluster [DBG] pgmap v269: 132 pgs: 132 active+clean; 4.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 75 KiB/s rd, 4.3 KiB/s wr, 162 op/s 2024-02-13T03:21:22.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:22 smithi161 bash[19769]: audit 2024-02-13T03:21:21.601041+0000 mgr.y (mgr.14146) 281 : audit [DBG] from='client.24427 v1:172.21.15.183:0/2761718404' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi183=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:22.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:22 smithi161 bash[19769]: cephadm 2024-02-13T03:21:21.605186+0000 mgr.y (mgr.14146) 282 : cephadm [INF] Saving service prometheus spec with placement smithi183=a;count:1 2024-02-13T03:21:22.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:22 smithi161 bash[19769]: audit 2024-02-13T03:21:21.614067+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:22.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:22 smithi161 bash[19769]: audit 2024-02-13T03:21:22.265659+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? v1:172.21.15.183:0/3784367185' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-13T03:21:22.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:22 smithi161 bash[27879]: cluster 2024-02-13T03:21:21.036937+0000 mgr.y (mgr.14146) 280 : cluster [DBG] pgmap v269: 132 pgs: 132 active+clean; 4.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 75 KiB/s rd, 4.3 KiB/s wr, 162 op/s 2024-02-13T03:21:22.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:22 smithi161 bash[27879]: audit 2024-02-13T03:21:21.601041+0000 mgr.y (mgr.14146) 281 : audit [DBG] from='client.24427 v1:172.21.15.183:0/2761718404' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi183=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:22.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:22 smithi161 bash[27879]: cephadm 2024-02-13T03:21:21.605186+0000 mgr.y (mgr.14146) 282 : cephadm [INF] Saving service prometheus spec with placement smithi183=a;count:1 2024-02-13T03:21:22.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:22 smithi161 bash[27879]: audit 2024-02-13T03:21:21.614067+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:22.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:22 smithi161 bash[27879]: audit 2024-02-13T03:21:22.265659+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? v1:172.21.15.183:0/3784367185' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-13T03:21:22.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:22 smithi183 bash[20824]: cluster 2024-02-13T03:21:21.036937+0000 mgr.y (mgr.14146) 280 : cluster [DBG] pgmap v269: 132 pgs: 132 active+clean; 4.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 75 KiB/s rd, 4.3 KiB/s wr, 162 op/s 2024-02-13T03:21:22.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:22 smithi183 bash[20824]: audit 2024-02-13T03:21:21.601041+0000 mgr.y (mgr.14146) 281 : audit [DBG] from='client.24427 v1:172.21.15.183:0/2761718404' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi183=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:22.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:22 smithi183 bash[20824]: cephadm 2024-02-13T03:21:21.605186+0000 mgr.y (mgr.14146) 282 : cephadm [INF] Saving service prometheus spec with placement smithi183=a;count:1 2024-02-13T03:21:22.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:22 smithi183 bash[20824]: audit 2024-02-13T03:21:21.614067+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:22.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:22 smithi183 bash[20824]: audit 2024-02-13T03:21:22.265659+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? v1:172.21.15.183:0/3784367185' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-13T03:21:24.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:24 smithi161 bash[19769]: cluster 2024-02-13T03:21:23.038135+0000 mgr.y (mgr.14146) 283 : cluster [DBG] pgmap v270: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 92 KiB/s rd, 4.4 KiB/s wr, 197 op/s 2024-02-13T03:21:24.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:24 smithi161 bash[19769]: audit 2024-02-13T03:21:24.176908+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:24.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:24 smithi161 bash[27879]: cluster 2024-02-13T03:21:23.038135+0000 mgr.y (mgr.14146) 283 : cluster [DBG] pgmap v270: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 92 KiB/s rd, 4.4 KiB/s wr, 197 op/s 2024-02-13T03:21:24.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:24 smithi161 bash[27879]: audit 2024-02-13T03:21:24.176908+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:24.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:24 smithi183 bash[20824]: cluster 2024-02-13T03:21:23.038135+0000 mgr.y (mgr.14146) 283 : cluster [DBG] pgmap v270: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 92 KiB/s rd, 4.4 KiB/s wr, 197 op/s 2024-02-13T03:21:24.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:24 smithi183 bash[20824]: audit 2024-02-13T03:21:24.176908+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:25.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff0a4b68700 1 -- 172.21.15.183:0/3987741649 >> v1:172.21.15.183:6789/0 conn(0x7ff0a00f78c0 legacy=0x7ff0a00f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:25.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff0a4b68700 1 -- 172.21.15.183:0/3987741649 shutdown_connections 2024-02-13T03:21:25.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff0a4b68700 1 -- 172.21.15.183:0/3987741649 >> 172.21.15.183:0/3987741649 conn(0x7ff0a00f3020 msgr2=0x7ff0a00f5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:25.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff0a4b68700 1 -- 172.21.15.183:0/3987741649 shutdown_connections 2024-02-13T03:21:25.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff0a4b68700 1 -- 172.21.15.183:0/3987741649 wait complete. 2024-02-13T03:21:25.562 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff0a4b68700 1 Processor -- start 2024-02-13T03:21:25.563 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff0a4b68700 1 -- start start 2024-02-13T03:21:25.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff0a4b68700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff0a0108670 con 0x7ff0a00fbd90 2024-02-13T03:21:25.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff0a4b68700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff0a0113ed0 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff0a4b68700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff0a01150b0 con 0x7ff0a00f78c0 2024-02-13T03:21:25.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff09ed9d700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7ff0a00ff9c0 0x7ff0a01117a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:51680/0 (socket says 172.21.15.183:51680) 2024-02-13T03:21:25.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff09ed9d700 1 -- 172.21.15.183:0/4224411467 learned_addr learned my addr 172.21.15.183:0/4224411467 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:21:25.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806025500 0 0) 0x7ff0a0113ed0 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.564 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff08c003030 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3064744168 0 0) 0x7ff08c003030 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff0a0113ed0 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7ff098001830 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395027620 0 0) 0x7ff0a01150b0 con 0x7ff0a00f78c0 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff08c003030 con 0x7ff0a00f78c0 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256913985 0 0) 0x7ff0a0108670 con 0x7ff0a00fbd90 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0a01150b0 con 0x7ff0a00fbd90 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1292399789 0 0) 0x7ff0a0113ed0 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 >> v1:172.21.15.161:6790/0 conn(0x7ff0a00f78c0 legacy=0x7ff0a0107aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 >> v1:172.21.15.161:6789/0 conn(0x7ff0a00fbd90 legacy=0x7ff0a010e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a0116290 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff0a4b68700 1 -- 172.21.15.183:0/4224411467 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a0112f20 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff0a4b68700 1 -- 172.21.15.183:0/4224411467 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a0113410 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.565 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7ff0980024a0 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.566 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.560+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7ff098005a70 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.566 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.564+0000 7ff0a4b68700 1 -- 172.21.15.183:0/4224411467 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff080004f50 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.566 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.564+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 15) v1 ==== 77081+0+0 (unknown 3266367394 0 0) 0x7ff098002000 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.566 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.564+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5884+0+0 (unknown 1000442856 0 0) 0x7ff098078550 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.576 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.572+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7ff098048500 con 0x7ff0a00ff9c0 2024-02-13T03:21:25.771 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.768+0000 7ff0a4b68700 1 -- 172.21.15.183:0/4224411467 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi161=a;smithi183=b", "target": ["mon-mgr", ""]}) v1 -- 0x7ff080000bc0 con 0x7ff08c0621a0 2024-02-13T03:21:25.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.784+0000 7ff097fff700 1 -- 172.21.15.183:0/4224411467 <== mgr.14146 v1:172.21.15.161:6800/1187535987 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (unknown 0 0 240551134) 0x7ff080000bc0 con 0x7ff08c0621a0 2024-02-13T03:21:25.785 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled node-exporter update... 2024-02-13T03:21:25.787 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.784+0000 7ff0a4b68700 1 -- 172.21.15.183:0/4224411467 >> v1:172.21.15.161:6800/1187535987 conn(0x7ff08c0621a0 legacy=0x7ff08c064660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.784+0000 7ff0a4b68700 1 -- 172.21.15.183:0/4224411467 >> v1:172.21.15.183:6789/0 conn(0x7ff0a00ff9c0 legacy=0x7ff0a01117a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.784+0000 7ff0a4b68700 1 -- 172.21.15.183:0/4224411467 shutdown_connections 2024-02-13T03:21:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.784+0000 7ff0a4b68700 1 -- 172.21.15.183:0/4224411467 >> 172.21.15.183:0/4224411467 conn(0x7ff0a00f3020 msgr2=0x7ff0a00ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.784+0000 7ff0a4b68700 1 -- 172.21.15.183:0/4224411467 shutdown_connections 2024-02-13T03:21:25.788 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:25.784+0000 7ff0a4b68700 1 -- 172.21.15.183:0/4224411467 wait complete. 2024-02-13T03:21:26.420 DEBUG:teuthology.orchestra.run.smithi161:node-exporter.a> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@node-exporter.a.service 2024-02-13T03:21:26.422 DEBUG:teuthology.orchestra.run.smithi183:node-exporter.b> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@node-exporter.b.service 2024-02-13T03:21:26.424 INFO:tasks.cephadm:Adding alertmanager.a on smithi161 2024-02-13T03:21:26.424 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch apply alertmanager '1;smithi161=a' 2024-02-13T03:21:26.533 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:21:26.597 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:21:26.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:26 smithi183 bash[20824]: cluster 2024-02-13T03:21:25.039043+0000 mgr.y (mgr.14146) 284 : cluster [DBG] pgmap v271: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 80 KiB/s rd, 3.7 KiB/s wr, 170 op/s 2024-02-13T03:21:26.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:26 smithi183 bash[20824]: audit 2024-02-13T03:21:25.782708+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:26 smithi161 bash[19769]: cluster 2024-02-13T03:21:25.039043+0000 mgr.y (mgr.14146) 284 : cluster [DBG] pgmap v271: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 80 KiB/s rd, 3.7 KiB/s wr, 170 op/s 2024-02-13T03:21:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:26 smithi161 bash[19769]: audit 2024-02-13T03:21:25.782708+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:26 smithi161 bash[27879]: cluster 2024-02-13T03:21:25.039043+0000 mgr.y (mgr.14146) 284 : cluster [DBG] pgmap v271: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 80 KiB/s rd, 3.7 KiB/s wr, 170 op/s 2024-02-13T03:21:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:26 smithi161 bash[27879]: audit 2024-02-13T03:21:25.782708+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:27.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:27 smithi183 bash[20824]: audit 2024-02-13T03:21:25.772036+0000 mgr.y (mgr.14146) 285 : audit [DBG] from='client.24457 v1:172.21.15.183:0/4224411467' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi161=a;smithi183=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:27.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:27 smithi183 bash[20824]: cephadm 2024-02-13T03:21:25.775741+0000 mgr.y (mgr.14146) 286 : cephadm [INF] Saving service node-exporter spec with placement smithi161=a;smithi183=b;count:2 2024-02-13T03:21:27.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:27 smithi183 bash[20824]: audit 2024-02-13T03:21:26.857999+0000 mon.a (mon.0) 595 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:27.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:27 smithi183 bash[20824]: audit 2024-02-13T03:21:27.137958+0000 mon.a (mon.0) 596 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:27.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:27 smithi183 bash[20824]: audit 2024-02-13T03:21:27.139833+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:21:27.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:27 smithi183 bash[20824]: audit 2024-02-13T03:21:27.148993+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:27.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:27 smithi183 bash[20824]: audit 2024-02-13T03:21:27.159833+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-13T03:21:27.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[19769]: audit 2024-02-13T03:21:25.772036+0000 mgr.y (mgr.14146) 285 : audit [DBG] from='client.24457 v1:172.21.15.183:0/4224411467' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi161=a;smithi183=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:27.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[19769]: cephadm 2024-02-13T03:21:25.775741+0000 mgr.y (mgr.14146) 286 : cephadm [INF] Saving service node-exporter spec with placement smithi161=a;smithi183=b;count:2 2024-02-13T03:21:27.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[19769]: audit 2024-02-13T03:21:26.857999+0000 mon.a (mon.0) 595 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:27.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[19769]: audit 2024-02-13T03:21:27.137958+0000 mon.a (mon.0) 596 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:27.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[19769]: audit 2024-02-13T03:21:27.139833+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:21:27.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[19769]: audit 2024-02-13T03:21:27.148993+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:27.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[19769]: audit 2024-02-13T03:21:27.159833+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-13T03:21:27.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[27879]: audit 2024-02-13T03:21:25.772036+0000 mgr.y (mgr.14146) 285 : audit [DBG] from='client.24457 v1:172.21.15.183:0/4224411467' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi161=a;smithi183=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:27.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[27879]: cephadm 2024-02-13T03:21:25.775741+0000 mgr.y (mgr.14146) 286 : cephadm [INF] Saving service node-exporter spec with placement smithi161=a;smithi183=b;count:2 2024-02-13T03:21:27.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[27879]: audit 2024-02-13T03:21:26.857999+0000 mon.a (mon.0) 595 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:27.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[27879]: audit 2024-02-13T03:21:27.137958+0000 mon.a (mon.0) 596 : audit [DBG] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:27.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[27879]: audit 2024-02-13T03:21:27.139833+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:21:27.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[27879]: audit 2024-02-13T03:21:27.148993+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' 2024-02-13T03:21:27.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:27 smithi161 bash[27879]: audit 2024-02-13T03:21:27.159833+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-13T03:21:28.459 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:28 smithi161 bash[20075]: ignoring --setuser ceph since I am not root 2024-02-13T03:21:28.459 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:28 smithi161 bash[20075]: ignoring --setgroup ceph since I am not root 2024-02-13T03:21:28.459 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:28 smithi161 bash[20075]: debug 2024-02-13T03:21:28.350+0000 7f018a50b680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T03:21:28.464 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:28 smithi183 bash[22114]: ignoring --setuser ceph since I am not root 2024-02-13T03:21:28.464 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:28 smithi183 bash[22114]: ignoring --setgroup ceph since I am not root 2024-02-13T03:21:28.464 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:28 smithi183 bash[22114]: debug 2024-02-13T03:21:28.200+0000 7f5f1d89c700 1 -- 172.21.15.183:0/2970424960 <== mon.1 v1:172.21.15.183:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 3758403927 0 0) 0x56295117ad00 con 0x56295126f000 2024-02-13T03:21:28.464 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:28 smithi183 bash[22114]: debug 2024-02-13T03:21:28.356+0000 7f5f2da7a680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T03:21:28.464 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:28 smithi183 bash[22114]: debug 2024-02-13T03:21:28.460+0000 7f5f2da7a680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T03:21:28.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:28 smithi183 bash[20824]: cluster 2024-02-13T03:21:27.040255+0000 mgr.y (mgr.14146) 287 : cluster [DBG] pgmap v272: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 79 KiB/s rd, 3.4 KiB/s wr, 164 op/s 2024-02-13T03:21:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:28 smithi183 bash[20824]: audit 2024-02-13T03:21:28.159711+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-13T03:21:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:28 smithi183 bash[20824]: cluster 2024-02-13T03:21:28.159803+0000 mon.a (mon.0) 601 : cluster [DBG] mgrmap e16: y(active, since 7m), standbys: x 2024-02-13T03:21:28.889 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:28 smithi161 bash[19769]: cluster 2024-02-13T03:21:27.040255+0000 mgr.y (mgr.14146) 287 : cluster [DBG] pgmap v272: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 79 KiB/s rd, 3.4 KiB/s wr, 164 op/s 2024-02-13T03:21:28.889 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:28 smithi161 bash[19769]: audit 2024-02-13T03:21:28.159711+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-13T03:21:28.889 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:28 smithi161 bash[19769]: cluster 2024-02-13T03:21:28.159803+0000 mon.a (mon.0) 601 : cluster [DBG] mgrmap e16: y(active, since 7m), standbys: x 2024-02-13T03:21:28.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:28 smithi161 bash[27879]: cluster 2024-02-13T03:21:27.040255+0000 mgr.y (mgr.14146) 287 : cluster [DBG] pgmap v272: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 79 KiB/s rd, 3.4 KiB/s wr, 164 op/s 2024-02-13T03:21:28.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:28 smithi161 bash[27879]: audit 2024-02-13T03:21:28.159711+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14146 v1:172.21.15.161:0/1992797389' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-13T03:21:28.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:28 smithi161 bash[27879]: cluster 2024-02-13T03:21:28.159803+0000 mon.a (mon.0) 601 : cluster [DBG] mgrmap e16: y(active, since 7m), standbys: x 2024-02-13T03:21:28.890 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:28 smithi161 bash[20075]: debug 2024-02-13T03:21:28.450+0000 7f018a50b680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T03:21:28.890 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:28 smithi161 bash[20075]: debug 2024-02-13T03:21:28.882+0000 7f018a50b680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T03:21:29.236 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:28 smithi183 bash[22114]: debug 2024-02-13T03:21:28.896+0000 7f5f2da7a680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T03:21:29.685 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb128248700 1 -- 172.21.15.183:0/440544089 >> v1:172.21.15.183:6789/0 conn(0x7fb1200f7d80 legacy=0x7fb1200f81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:29.685 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb128248700 1 -- 172.21.15.183:0/440544089 shutdown_connections 2024-02-13T03:21:29.685 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb128248700 1 -- 172.21.15.183:0/440544089 >> 172.21.15.183:0/440544089 conn(0x7fb1200f16b0 msgr2=0x7fb1200f3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:29.686 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb128248700 1 -- 172.21.15.183:0/440544089 shutdown_connections 2024-02-13T03:21:29.686 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb128248700 1 -- 172.21.15.183:0/440544089 wait complete. 2024-02-13T03:21:29.686 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb128248700 1 Processor -- start 2024-02-13T03:21:29.686 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb128248700 1 -- start start 2024-02-13T03:21:29.687 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb128248700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb12010a670 con 0x7fb1200f7d80 2024-02-13T03:21:29.687 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb128248700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb120115ed0 con 0x7fb1200f7560 2024-02-13T03:21:29.687 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb128248700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb1201170b0 con 0x7fb1200f69c0 2024-02-13T03:21:29.687 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb1257e3700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fb1200f7560 0x7fb120110070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:41386/0 (socket says 172.21.15.183:41386) 2024-02-13T03:21:29.687 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb1257e3700 1 -- 172.21.15.183:0/3337956399 learned_addr learned my addr 172.21.15.183:0/3337956399 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:21:29.688 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542722065 0 0) 0x7fb120115ed0 con 0x7fb1200f7560 2024-02-13T03:21:29.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb10c003630 con 0x7fb1200f7560 2024-02-13T03:21:29.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222572379 0 0) 0x7fb1201170b0 con 0x7fb1200f69c0 2024-02-13T03:21:29.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb120115ed0 con 0x7fb1200f69c0 2024-02-13T03:21:29.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845810328 0 0) 0x7fb12010a670 con 0x7fb1200f7d80 2024-02-13T03:21:29.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1201170b0 con 0x7fb1200f7d80 2024-02-13T03:21:29.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3860095648 0 0) 0x7fb10c003630 con 0x7fb1200f7560 2024-02-13T03:21:29.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb12010a670 con 0x7fb1200f7560 2024-02-13T03:21:29.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.684+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb110001920 con 0x7fb1200f7560 2024-02-13T03:21:29.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1713902685 0 0) 0x7fb120115ed0 con 0x7fb1200f69c0 2024-02-13T03:21:29.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb10c003630 con 0x7fb1200f69c0 2024-02-13T03:21:29.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb11c002ec0 con 0x7fb1200f69c0 2024-02-13T03:21:29.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2425532455 0 0) 0x7fb12010a670 con 0x7fb1200f7560 2024-02-13T03:21:29.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 >> v1:172.21.15.161:6790/0 conn(0x7fb1200f69c0 legacy=0x7fb120109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:29.690 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 >> v1:172.21.15.161:6789/0 conn(0x7fb1200f7d80 legacy=0x7fb1201137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:29.690 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb120118290 con 0x7fb1200f7560 2024-02-13T03:21:29.690 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb128248700 1 -- 172.21.15.183:0/3337956399 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1201172a0 con 0x7fb1200f7560 2024-02-13T03:21:29.690 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb128248700 1 -- 172.21.15.183:0/3337956399 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb120117790 con 0x7fb1200f7560 2024-02-13T03:21:29.690 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fb110001cb0 con 0x7fb1200f7560 2024-02-13T03:21:29.691 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb110005bc0 con 0x7fb1200f7560 2024-02-13T03:21:29.691 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb128248700 1 -- 172.21.15.183:0/3337956399 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb104004f20 con 0x7fb1200f7560 2024-02-13T03:21:29.691 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 16) v1 ==== 77095+0+0 (unknown 2914212039 0 0) 0x7fb110002270 con 0x7fb1200f7560 2024-02-13T03:21:29.692 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5884+0+0 (unknown 1000442856 0 0) 0x7fb1100785b0 con 0x7fb1200f7560 2024-02-13T03:21:29.692 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.688+0000 7fb125fe4700 1 -- 172.21.15.183:0/3337956399 >> v1:172.21.15.161:6800/1187535987 conn(0x7fb10c0620d0 legacy=0x7fb10c064590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/1187535987 2024-02-13T03:21:29.698 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.696+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fb1100485c0 con 0x7fb1200f7560 2024-02-13T03:21:29.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.892+0000 7fb125fe4700 1 -- 172.21.15.183:0/3337956399 >> v1:172.21.15.161:6800/1187535987 conn(0x7fb10c0620d0 legacy=0x7fb10c064590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/1187535987 2024-02-13T03:21:29.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:29.940+0000 7fb128248700 1 -- 172.21.15.183:0/3337956399 --> v1:172.21.15.161:6800/1187535987 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi161=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fb104000b90 con 0x7fb10c0620d0 2024-02-13T03:21:30.121 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:29 smithi161 bash[20075]: debug 2024-02-13T03:21:29.778+0000 7f018a50b680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T03:21:30.121 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:29 smithi161 bash[20075]: debug 2024-02-13T03:21:29.870+0000 7f018a50b680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T03:21:30.151 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:29 smithi183 bash[22114]: debug 2024-02-13T03:21:29.808+0000 7f5f2da7a680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T03:21:30.151 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:29 smithi183 bash[22114]: debug 2024-02-13T03:21:29.904+0000 7f5f2da7a680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T03:21:30.297 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:30.296+0000 7fb125fe4700 1 -- 172.21.15.183:0/3337956399 >> v1:172.21.15.161:6800/1187535987 conn(0x7fb10c0620d0 legacy=0x7fb10c064590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/1187535987 2024-02-13T03:21:30.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:30 smithi161 bash[20075]: debug 2024-02-13T03:21:30.114+0000 7f018a50b680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T03:21:30.486 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:30 smithi183 bash[22114]: debug 2024-02-13T03:21:30.148+0000 7f5f2da7a680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T03:21:31.101 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:31.100+0000 7fb125fe4700 1 -- 172.21.15.183:0/3337956399 >> v1:172.21.15.161:6800/1187535987 conn(0x7fb10c0620d0 legacy=0x7fb10c064590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/1187535987 2024-02-13T03:21:31.477 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:31 smithi161 bash[20075]: debug 2024-02-13T03:21:31.214+0000 7f018a50b680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T03:21:31.477 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:31 smithi161 bash[20075]: debug 2024-02-13T03:21:31.374+0000 7f018a50b680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T03:21:31.477 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:31 smithi161 bash[20075]: debug 2024-02-13T03:21:31.470+0000 7f018a50b680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T03:21:31.496 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:31 smithi183 bash[22114]: debug 2024-02-13T03:21:31.240+0000 7f5f2da7a680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T03:21:31.497 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:31 smithi183 bash[22114]: debug 2024-02-13T03:21:31.396+0000 7f5f2da7a680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T03:21:31.497 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:31 smithi183 bash[22114]: debug 2024-02-13T03:21:31.492+0000 7f5f2da7a680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T03:21:31.808 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:31 smithi161 bash[20075]: debug 2024-02-13T03:21:31.578+0000 7f018a50b680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T03:21:31.809 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:31 smithi161 bash[20075]: debug 2024-02-13T03:21:31.802+0000 7f018a50b680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T03:21:31.824 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:31 smithi183 bash[22114]: debug 2024-02-13T03:21:31.600+0000 7f5f2da7a680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T03:21:31.824 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:31 smithi183 bash[22114]: debug 2024-02-13T03:21:31.820+0000 7f5f2da7a680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T03:21:32.167 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:32 smithi161 bash[20075]: debug 2024-02-13T03:21:32.158+0000 7f018a50b680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T03:21:32.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:32 smithi161 bash[20075]: debug 2024-02-13T03:21:32.278+0000 7f018a50b680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T03:21:32.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:32 smithi183 bash[22114]: debug 2024-02-13T03:21:32.180+0000 7f5f2da7a680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T03:21:32.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:32 smithi183 bash[22114]: debug 2024-02-13T03:21:32.300+0000 7f5f2da7a680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T03:21:32.705 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:32.704+0000 7fb125fe4700 1 -- 172.21.15.183:0/3337956399 >> v1:172.21.15.161:6800/1187535987 conn(0x7fb10c0620d0 legacy=0x7fb10c064590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.161:6800/1187535987 2024-02-13T03:21:33.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:33 smithi161 bash[20075]: debug 2024-02-13T03:21:33.122+0000 7f018a50b680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T03:21:33.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:33 smithi161 bash[20075]: debug 2024-02-13T03:21:33.222+0000 7f018a50b680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T03:21:33.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:33 smithi161 bash[20075]: debug 2024-02-13T03:21:33.338+0000 7f018a50b680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T03:21:33.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:33 smithi183 bash[22114]: debug 2024-02-13T03:21:33.144+0000 7f5f2da7a680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T03:21:33.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:33 smithi183 bash[22114]: debug 2024-02-13T03:21:33.244+0000 7f5f2da7a680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T03:21:33.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:33 smithi183 bash[22114]: debug 2024-02-13T03:21:33.356+0000 7f5f2da7a680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T03:21:33.850 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:33 smithi161 bash[20075]: debug 2024-02-13T03:21:33.554+0000 7f018a50b680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T03:21:33.851 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:33 smithi161 bash[20075]: debug 2024-02-13T03:21:33.842+0000 7f018a50b680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T03:21:33.858 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:33 smithi183 bash[22114]: debug 2024-02-13T03:21:33.568+0000 7f5f2da7a680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T03:21:34.177 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[20075]: debug 2024-02-13T03:21:34.006+0000 7f018a50b680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T03:21:34.187 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:33 smithi183 bash[22114]: debug 2024-02-13T03:21:33.856+0000 7f5f2da7a680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T03:21:34.187 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[22114]: debug 2024-02-13T03:21:34.016+0000 7f5f2da7a680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T03:21:34.187 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[22114]: debug 2024-02-13T03:21:34.184+0000 7f5f2da7a680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T03:21:34.486 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[22114]: debug 2024-02-13T03:21:34.392+0000 7f5f2da7a680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T03:21:34.496 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[20075]: debug 2024-02-13T03:21:34.174+0000 7f018a50b680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T03:21:34.496 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[20075]: debug 2024-02-13T03:21:34.382+0000 7f018a50b680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T03:21:34.497 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[20075]: debug 2024-02-13T03:21:34.490+0000 7f018a50b680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T03:21:34.534 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:34.532+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.1 v1:172.21.15.183:6789/0 10 ==== mgrmap(e 17) v1 ==== 76859+0+0 (unknown 622989856 0 0) 0x7fb1100762d0 con 0x7fb1200f7560 2024-02-13T03:21:34.534 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:34.532+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 >> v1:172.21.15.161:6800/1187535987 conn(0x7fb10c0620d0 legacy=0x7fb10c064590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-13T03:21:34.749 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: cluster 2024-02-13T03:21:34.516375+0000 mon.a (mon.0) 602 : cluster [INF] Active manager daemon y restarted 2024-02-13T03:21:34.749 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: cluster 2024-02-13T03:21:34.516769+0000 mon.a (mon.0) 603 : cluster [INF] Activating manager daemon y 2024-02-13T03:21:34.749 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: cluster 2024-02-13T03:21:34.531237+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-13T03:21:34.750 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: cluster 2024-02-13T03:21:34.531934+0000 mon.a (mon.0) 605 : cluster [DBG] Standby manager daemon x restarted 2024-02-13T03:21:34.750 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: cluster 2024-02-13T03:21:34.531986+0000 mon.a (mon.0) 606 : cluster [DBG] Standby manager daemon x started 2024-02-13T03:21:34.750 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: cluster 2024-02-13T03:21:34.532164+0000 mon.a (mon.0) 607 : cluster [DBG] mgrmap e17: y(active, starting, since 0.0155228s), standbys: x 2024-02-13T03:21:34.750 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.533523+0000 mon.c (mon.2) 8 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T03:21:34.750 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.533875+0000 mon.c (mon.2) 9 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:21:34.750 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.534202+0000 mon.c (mon.2) 10 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:21:34.750 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.535175+0000 mon.c (mon.2) 11 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T03:21:34.750 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.535553+0000 mon.c (mon.2) 12 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T03:21:34.750 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.535869+0000 mon.c (mon.2) 13 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:21:34.750 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.536256+0000 mon.c (mon.2) 14 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:21:34.750 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.536646+0000 mon.c (mon.2) 15 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:21:34.750 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.537011+0000 mon.c (mon.2) 16 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:21:34.750 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.537339+0000 mon.c (mon.2) 17 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:21:34.751 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.537701+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:21:34.751 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.538035+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:21:34.751 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.538200+0000 mon.b (mon.1) 21 : audit [DBG] from='mgr.? v1:172.21.15.183:0/1910626557' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T03:21:34.751 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.538373+0000 mon.c (mon.2) 20 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:21:34.751 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: cluster 2024-02-13T03:21:34.516375+0000 mon.a (mon.0) 602 : cluster [INF] Active manager daemon y restarted 2024-02-13T03:21:34.751 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: cluster 2024-02-13T03:21:34.516769+0000 mon.a (mon.0) 603 : cluster [INF] Activating manager daemon y 2024-02-13T03:21:34.751 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: cluster 2024-02-13T03:21:34.531237+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-13T03:21:34.751 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: cluster 2024-02-13T03:21:34.531934+0000 mon.a (mon.0) 605 : cluster [DBG] Standby manager daemon x restarted 2024-02-13T03:21:34.751 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: cluster 2024-02-13T03:21:34.531986+0000 mon.a (mon.0) 606 : cluster [DBG] Standby manager daemon x started 2024-02-13T03:21:34.751 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: cluster 2024-02-13T03:21:34.532164+0000 mon.a (mon.0) 607 : cluster [DBG] mgrmap e17: y(active, starting, since 0.0155228s), standbys: x 2024-02-13T03:21:34.752 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.533523+0000 mon.c (mon.2) 8 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T03:21:34.752 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.533875+0000 mon.c (mon.2) 9 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:21:34.752 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.534202+0000 mon.c (mon.2) 10 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:21:34.752 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.535175+0000 mon.c (mon.2) 11 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T03:21:34.752 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.535553+0000 mon.c (mon.2) 12 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T03:21:34.752 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.535869+0000 mon.c (mon.2) 13 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:21:34.752 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.536256+0000 mon.c (mon.2) 14 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:21:34.752 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.536646+0000 mon.c (mon.2) 15 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:21:34.752 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.537011+0000 mon.c (mon.2) 16 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:21:34.752 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.537339+0000 mon.c (mon.2) 17 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:21:34.752 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.537701+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:21:34.752 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.538035+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:21:34.753 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.538200+0000 mon.b (mon.1) 21 : audit [DBG] from='mgr.? v1:172.21.15.183:0/1910626557' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T03:21:34.753 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.538373+0000 mon.c (mon.2) 20 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:21:34.753 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.538729+0000 mon.c (mon.2) 21 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T03:21:34.753 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.538769+0000 mon.b (mon.1) 22 : audit [DBG] from='mgr.? v1:172.21.15.183:0/1910626557' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T03:21:34.753 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.539007+0000 mon.c (mon.2) 22 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T03:21:34.753 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.539638+0000 mon.c (mon.2) 23 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T03:21:34.753 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.539869+0000 mon.b (mon.1) 23 : audit [DBG] from='mgr.? v1:172.21.15.183:0/1910626557' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T03:21:34.753 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: audit 2024-02-13T03:21:34.540296+0000 mon.b (mon.1) 24 : audit [DBG] from='mgr.? v1:172.21.15.183:0/1910626557' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T03:21:34.753 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[27879]: cluster 2024-02-13T03:21:34.543387+0000 mon.a (mon.0) 608 : cluster [INF] Manager daemon y is now available 2024-02-13T03:21:34.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.538729+0000 mon.c (mon.2) 21 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T03:21:34.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.538769+0000 mon.b (mon.1) 22 : audit [DBG] from='mgr.? v1:172.21.15.183:0/1910626557' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T03:21:34.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.539007+0000 mon.c (mon.2) 22 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T03:21:34.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.539638+0000 mon.c (mon.2) 23 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T03:21:34.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.539869+0000 mon.b (mon.1) 23 : audit [DBG] from='mgr.? v1:172.21.15.183:0/1910626557' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T03:21:34.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: audit 2024-02-13T03:21:34.540296+0000 mon.b (mon.1) 24 : audit [DBG] from='mgr.? v1:172.21.15.183:0/1910626557' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T03:21:34.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[19769]: cluster 2024-02-13T03:21:34.543387+0000 mon.a (mon.0) 608 : cluster [INF] Manager daemon y is now available 2024-02-13T03:21:34.805 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: cluster 2024-02-13T03:21:34.516375+0000 mon.a (mon.0) 602 : cluster [INF] Active manager daemon y restarted 2024-02-13T03:21:34.805 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: cluster 2024-02-13T03:21:34.516769+0000 mon.a (mon.0) 603 : cluster [INF] Activating manager daemon y 2024-02-13T03:21:34.805 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: cluster 2024-02-13T03:21:34.531237+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-13T03:21:34.805 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: cluster 2024-02-13T03:21:34.531934+0000 mon.a (mon.0) 605 : cluster [DBG] Standby manager daemon x restarted 2024-02-13T03:21:34.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: cluster 2024-02-13T03:21:34.531986+0000 mon.a (mon.0) 606 : cluster [DBG] Standby manager daemon x started 2024-02-13T03:21:34.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: cluster 2024-02-13T03:21:34.532164+0000 mon.a (mon.0) 607 : cluster [DBG] mgrmap e17: y(active, starting, since 0.0155228s), standbys: x 2024-02-13T03:21:34.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.533523+0000 mon.c (mon.2) 8 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T03:21:34.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.533875+0000 mon.c (mon.2) 9 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T03:21:34.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.534202+0000 mon.c (mon.2) 10 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T03:21:34.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.535175+0000 mon.c (mon.2) 11 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T03:21:34.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.535553+0000 mon.c (mon.2) 12 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T03:21:34.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.535869+0000 mon.c (mon.2) 13 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T03:21:34.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.536256+0000 mon.c (mon.2) 14 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T03:21:34.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.536646+0000 mon.c (mon.2) 15 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T03:21:34.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.537011+0000 mon.c (mon.2) 16 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T03:21:34.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.537339+0000 mon.c (mon.2) 17 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T03:21:34.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.537701+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T03:21:34.807 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.538035+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T03:21:34.807 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.538200+0000 mon.b (mon.1) 21 : audit [DBG] from='mgr.? v1:172.21.15.183:0/1910626557' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T03:21:34.807 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.538373+0000 mon.c (mon.2) 20 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T03:21:34.807 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[22114]: debug 2024-02-13T03:21:34.496+0000 7f5f2da7a680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T03:21:34.807 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[22114]: [13/Feb/2024:03:21:34] ENGINE Bus STARTING 2024-02-13T03:21:34.807 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[22114]: CherryPy Checker: 2024-02-13T03:21:34.807 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[22114]: The Application mounted at '' has an empty config. 2024-02-13T03:21:34.807 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[22114]: [13/Feb/2024:03:21:34] ENGINE Serving on http://:::9283 2024-02-13T03:21:34.807 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[22114]: [13/Feb/2024:03:21:34] ENGINE Bus STARTED 2024-02-13T03:21:34.808 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.538729+0000 mon.c (mon.2) 21 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T03:21:34.808 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.538769+0000 mon.b (mon.1) 22 : audit [DBG] from='mgr.? v1:172.21.15.183:0/1910626557' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T03:21:34.808 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.539007+0000 mon.c (mon.2) 22 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T03:21:34.808 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.539638+0000 mon.c (mon.2) 23 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T03:21:34.808 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.539869+0000 mon.b (mon.1) 23 : audit [DBG] from='mgr.? v1:172.21.15.183:0/1910626557' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T03:21:34.808 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: audit 2024-02-13T03:21:34.540296+0000 mon.b (mon.1) 24 : audit [DBG] from='mgr.? v1:172.21.15.183:0/1910626557' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T03:21:34.808 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:34 smithi183 bash[20824]: cluster 2024-02-13T03:21:34.543387+0000 mon.a (mon.0) 608 : cluster [INF] Manager daemon y is now available 2024-02-13T03:21:35.177 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[20075]: Warning: Permanently added '172.21.15.161' (ECDSA) to the list of known hosts. 2024-02-13T03:21:35.177 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[20075]: Warning: Permanently added '172.21.15.183' (ECDSA) to the list of known hosts. 2024-02-13T03:21:35.177 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[20075]: [13/Feb/2024:03:21:34] ENGINE Bus STARTING 2024-02-13T03:21:35.177 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[20075]: CherryPy Checker: 2024-02-13T03:21:35.177 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[20075]: The Application mounted at '' has an empty config. 2024-02-13T03:21:35.177 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[20075]: [13/Feb/2024:03:21:34] ENGINE Serving on http://:::9283 2024-02-13T03:21:35.177 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:21:34 smithi161 bash[20075]: [13/Feb/2024:03:21:34] ENGINE Bus STARTED 2024-02-13T03:21:35.542 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:35.540+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mon.1 v1:172.21.15.183:6789/0 11 ==== mgrmap(e 18) v1 ==== 76894+0+0 (unknown 2367926810 0 0) 0x7fb110076650 con 0x7fb1200f7560 2024-02-13T03:21:35.543 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:35.540+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 --> v1:172.21.15.161:6800/4114337139 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi161=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fb104000b90 con 0x7fb10c06c2e0 2024-02-13T03:21:35.554 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:35.552+0000 7fb116ffd700 1 -- 172.21.15.183:0/3337956399 <== mgr.24473 v1:172.21.15.161:6800/4114337139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (unknown 0 0 1850065467) 0x7fb104000b90 con 0x7fb10c06c2e0 2024-02-13T03:21:35.554 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled alertmanager update... 2024-02-13T03:21:35.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:35.556+0000 7fb128248700 1 -- 172.21.15.183:0/3337956399 >> v1:172.21.15.161:6800/4114337139 conn(0x7fb10c06c2e0 legacy=0x7fb10c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:35.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:35.556+0000 7fb128248700 1 -- 172.21.15.183:0/3337956399 >> v1:172.21.15.183:6789/0 conn(0x7fb1200f7560 legacy=0x7fb120110070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:35.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:35.556+0000 7fb1267e5700 1 -- 172.21.15.183:0/3337956399 reap_dead start 2024-02-13T03:21:35.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:35.556+0000 7fb128248700 1 -- 172.21.15.183:0/3337956399 shutdown_connections 2024-02-13T03:21:35.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:35.556+0000 7fb128248700 1 -- 172.21.15.183:0/3337956399 >> 172.21.15.183:0/3337956399 conn(0x7fb1200f16b0 msgr2=0x7fb1200f3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:35.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:35.556+0000 7fb128248700 1 -- 172.21.15.183:0/3337956399 shutdown_connections 2024-02-13T03:21:35.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:35.556+0000 7fb128248700 1 -- 172.21.15.183:0/3337956399 wait complete. 2024-02-13T03:21:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[27879]: audit 2024-02-13T03:21:34.685574+0000 mon.c (mon.2) 24 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:21:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[27879]: audit 2024-02-13T03:21:34.688200+0000 mon.c (mon.2) 25 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:21:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[27879]: audit 2024-02-13T03:21:34.751806+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[27879]: audit 2024-02-13T03:21:34.770359+0000 mon.c (mon.2) 26 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T03:21:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[27879]: audit 2024-02-13T03:21:34.770819+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T03:21:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[27879]: audit 2024-02-13T03:21:34.808225+0000 mon.c (mon.2) 27 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T03:21:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[27879]: audit 2024-02-13T03:21:34.810070+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T03:21:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[27879]: cluster 2024-02-13T03:21:35.539319+0000 mon.a (mon.0) 612 : cluster [DBG] mgrmap e18: y(active, since 1.02266s), standbys: x 2024-02-13T03:21:35.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[27879]: audit 2024-02-13T03:21:35.551834+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[19769]: audit 2024-02-13T03:21:34.685574+0000 mon.c (mon.2) 24 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:21:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[19769]: audit 2024-02-13T03:21:34.688200+0000 mon.c (mon.2) 25 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:21:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[19769]: audit 2024-02-13T03:21:34.751806+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[19769]: audit 2024-02-13T03:21:34.770359+0000 mon.c (mon.2) 26 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T03:21:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[19769]: audit 2024-02-13T03:21:34.770819+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T03:21:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[19769]: audit 2024-02-13T03:21:34.808225+0000 mon.c (mon.2) 27 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T03:21:35.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[19769]: audit 2024-02-13T03:21:34.810070+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T03:21:35.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[19769]: cluster 2024-02-13T03:21:35.539319+0000 mon.a (mon.0) 612 : cluster [DBG] mgrmap e18: y(active, since 1.02266s), standbys: x 2024-02-13T03:21:35.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:35 smithi161 bash[19769]: audit 2024-02-13T03:21:35.551834+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:35.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:21:35 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:21:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:35 smithi183 bash[20824]: audit 2024-02-13T03:21:34.685574+0000 mon.c (mon.2) 24 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:21:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:35 smithi183 bash[20824]: audit 2024-02-13T03:21:34.688200+0000 mon.c (mon.2) 25 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:21:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:35 smithi183 bash[20824]: audit 2024-02-13T03:21:34.751806+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:35 smithi183 bash[20824]: audit 2024-02-13T03:21:34.770359+0000 mon.c (mon.2) 26 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T03:21:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:35 smithi183 bash[20824]: audit 2024-02-13T03:21:34.770819+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T03:21:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:35 smithi183 bash[20824]: audit 2024-02-13T03:21:34.808225+0000 mon.c (mon.2) 27 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T03:21:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:35 smithi183 bash[20824]: audit 2024-02-13T03:21:34.810070+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T03:21:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:35 smithi183 bash[20824]: cluster 2024-02-13T03:21:35.539319+0000 mon.a (mon.0) 612 : cluster [DBG] mgrmap e18: y(active, since 1.02266s), standbys: x 2024-02-13T03:21:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:35 smithi183 bash[20824]: audit 2024-02-13T03:21:35.551834+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:36.370 DEBUG:teuthology.orchestra.run.smithi161:alertmanager.a> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@alertmanager.a.service 2024-02-13T03:21:36.372 INFO:tasks.cephadm:Adding grafana.a on smithi183 2024-02-13T03:21:36.372 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph orch apply grafana '1;smithi183=a' 2024-02-13T03:21:36.494 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:21:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:36 smithi161 bash[19769]: cephadm 2024-02-13T03:21:35.547737+0000 mgr.y (mgr.24473) 4 : cephadm [INF] Saving service alertmanager spec with placement smithi161=a;count:1 2024-02-13T03:21:36.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:36 smithi161 bash[27879]: cephadm 2024-02-13T03:21:35.547737+0000 mgr.y (mgr.24473) 4 : cephadm [INF] Saving service alertmanager spec with placement smithi161=a;count:1 2024-02-13T03:21:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:36 smithi183 bash[20824]: cephadm 2024-02-13T03:21:35.547737+0000 mgr.y (mgr.24473) 4 : cephadm [INF] Saving service alertmanager spec with placement smithi161=a;count:1 2024-02-13T03:21:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:37 smithi183 bash[20824]: cluster 2024-02-13T03:21:36.537155+0000 mgr.y (mgr.24473) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:37 smithi183 bash[20824]: cluster 2024-02-13T03:21:37.550462+0000 mon.a (mon.0) 614 : cluster [DBG] mgrmap e19: y(active, since 3s), standbys: x 2024-02-13T03:21:37.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:37 smithi161 bash[27879]: cluster 2024-02-13T03:21:36.537155+0000 mgr.y (mgr.24473) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:37.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:37 smithi161 bash[27879]: cluster 2024-02-13T03:21:37.550462+0000 mon.a (mon.0) 614 : cluster [DBG] mgrmap e19: y(active, since 3s), standbys: x 2024-02-13T03:21:37.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:37 smithi161 bash[19769]: cluster 2024-02-13T03:21:36.537155+0000 mgr.y (mgr.24473) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:37.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:37 smithi161 bash[19769]: cluster 2024-02-13T03:21:37.550462+0000 mon.a (mon.0) 614 : cluster [DBG] mgrmap e19: y(active, since 3s), standbys: x 2024-02-13T03:21:39.179 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.176+0000 7f6244297700 1 -- 172.21.15.183:0/1095844534 >> v1:172.21.15.183:6789/0 conn(0x7f623c061760 legacy=0x7f623c0fa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:39.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.176+0000 7f6244297700 1 -- 172.21.15.183:0/1095844534 shutdown_connections 2024-02-13T03:21:39.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.176+0000 7f6244297700 1 -- 172.21.15.183:0/1095844534 >> 172.21.15.183:0/1095844534 conn(0x7f623c0f1f50 msgr2=0x7f623c0f4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:39.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.176+0000 7f6244297700 1 -- 172.21.15.183:0/1095844534 shutdown_connections 2024-02-13T03:21:39.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.176+0000 7f6244297700 1 -- 172.21.15.183:0/1095844534 wait complete. 2024-02-13T03:21:39.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f6244297700 1 Processor -- start 2024-02-13T03:21:39.180 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f6244297700 1 -- start start 2024-02-13T03:21:39.181 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f6244297700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f623c108660 con 0x7f623c0fbcc0 2024-02-13T03:21:39.181 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f6244297700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f623c113ec0 con 0x7f623c061760 2024-02-13T03:21:39.181 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f6244297700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f623c1150a0 con 0x7f623c0ff9d0 2024-02-13T03:21:39.181 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f6242033700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f623c061760 0x7f623c107a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:41418/0 (socket says 172.21.15.183:41418) 2024-02-13T03:21:39.181 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f6242033700 1 -- 172.21.15.183:0/1358676025 learned_addr learned my addr 172.21.15.183:0/1358676025 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:21:39.183 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942503991 0 0) 0x7f623c113ec0 con 0x7f623c061760 2024-02-13T03:21:39.183 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6224003030 con 0x7f623c061760 2024-02-13T03:21:39.183 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691951578 0 0) 0x7f623c1150a0 con 0x7f623c0ff9d0 2024-02-13T03:21:39.183 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f623c113ec0 con 0x7f623c0ff9d0 2024-02-13T03:21:39.183 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151306357 0 0) 0x7f623c108660 con 0x7f623c0fbcc0 2024-02-13T03:21:39.183 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f623c1150a0 con 0x7f623c0fbcc0 2024-02-13T03:21:39.183 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 92268649 0 0) 0x7f6224003030 con 0x7f623c061760 2024-02-13T03:21:39.183 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f623c108660 con 0x7f623c061760 2024-02-13T03:21:39.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f6234001430 con 0x7f623c061760 2024-02-13T03:21:39.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2933574992 0 0) 0x7f623c113ec0 con 0x7f623c0ff9d0 2024-02-13T03:21:39.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6224003030 con 0x7f623c0ff9d0 2024-02-13T03:21:39.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f622c001b80 con 0x7f623c0ff9d0 2024-02-13T03:21:39.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1925072596 0 0) 0x7f623c1150a0 con 0x7f623c0fbcc0 2024-02-13T03:21:39.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f623c113ec0 con 0x7f623c0fbcc0 2024-02-13T03:21:39.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4028376694 0 0) 0x7f623c108660 con 0x7f623c061760 2024-02-13T03:21:39.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 >> v1:172.21.15.161:6790/0 conn(0x7f623c0ff9d0 legacy=0x7f623c111790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:39.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 >> v1:172.21.15.161:6789/0 conn(0x7f623c0fbcc0 legacy=0x7f623c10e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:39.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c116280 con 0x7f623c061760 2024-02-13T03:21:39.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f6244297700 1 -- 172.21.15.183:0/1358676025 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c115290 con 0x7f623c061760 2024-02-13T03:21:39.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f6244297700 1 -- 172.21.15.183:0/1358676025 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c115780 con 0x7f623c061760 2024-02-13T03:21:39.184 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f62340011d0 con 0x7f623c061760 2024-02-13T03:21:39.185 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.180+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f6234003860 con 0x7f623c061760 2024-02-13T03:21:39.185 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.184+0000 7f6244297700 1 -- 172.21.15.183:0/1358676025 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6228004f20 con 0x7f623c061760 2024-02-13T03:21:39.185 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.184+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f6234003a60 con 0x7f623c061760 2024-02-13T03:21:39.186 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.184+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6056+0+0 (unknown 2145861735 0 0) 0x7f62340787a0 con 0x7f623c061760 2024-02-13T03:21:39.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.192+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f623404c430 con 0x7f623c061760 2024-02-13T03:21:39.393 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.392+0000 7f6244297700 1 -- 172.21.15.183:0/1358676025 --> v1:172.21.15.161:6800/4114337139 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi183=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f6228000b90 con 0x7f6224062400 2024-02-13T03:21:39.405 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.404+0000 7f623b779700 1 -- 172.21.15.183:0/1358676025 <== mgr.24473 v1:172.21.15.161:6800/4114337139 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (unknown 0 0 664801700) 0x7f6228000b90 con 0x7f6224062400 2024-02-13T03:21:39.405 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled grafana update... 2024-02-13T03:21:39.407 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.404+0000 7f6244297700 1 -- 172.21.15.183:0/1358676025 >> v1:172.21.15.161:6800/4114337139 conn(0x7f6224062400 legacy=0x7f62240648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:39.407 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.404+0000 7f6244297700 1 -- 172.21.15.183:0/1358676025 >> v1:172.21.15.183:6789/0 conn(0x7f623c061760 legacy=0x7f623c107a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:39.407 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.404+0000 7f6244297700 1 -- 172.21.15.183:0/1358676025 shutdown_connections 2024-02-13T03:21:39.408 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.404+0000 7f6244297700 1 -- 172.21.15.183:0/1358676025 >> 172.21.15.183:0/1358676025 conn(0x7f623c0f1f50 msgr2=0x7f623c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:39.408 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.404+0000 7f6244297700 1 -- 172.21.15.183:0/1358676025 shutdown_connections 2024-02-13T03:21:39.408 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:39.404+0000 7f6244297700 1 -- 172.21.15.183:0/1358676025 wait complete. 2024-02-13T03:21:39.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:39 smithi183 bash[20824]: cluster 2024-02-13T03:21:38.537939+0000 mgr.y (mgr.24473) 6 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:39.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:39 smithi183 bash[20824]: audit 2024-02-13T03:21:39.402304+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:39.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:39 smithi161 bash[27879]: cluster 2024-02-13T03:21:38.537939+0000 mgr.y (mgr.24473) 6 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:39.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:39 smithi161 bash[27879]: audit 2024-02-13T03:21:39.402304+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:39.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:39 smithi161 bash[19769]: cluster 2024-02-13T03:21:38.537939+0000 mgr.y (mgr.24473) 6 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:39.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:39 smithi161 bash[19769]: audit 2024-02-13T03:21:39.402304+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:40.016 DEBUG:teuthology.orchestra.run.smithi183:grafana.a> sudo journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@grafana.a.service 2024-02-13T03:21:40.019 INFO:tasks.cephadm:Setting up client nodes... 2024-02-13T03:21:40.020 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-13T03:21:40.123 INFO:journalctl@ceph.grafana.a.smithi183.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-13T03:21:40.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:40 smithi161 bash[19769]: audit 2024-02-13T03:21:39.393678+0000 mgr.y (mgr.24473) 7 : audit [DBG] from='client.24493 v1:172.21.15.183:0/1358676025' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi183=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:40.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:40 smithi161 bash[19769]: cephadm 2024-02-13T03:21:39.395992+0000 mgr.y (mgr.24473) 8 : cephadm [INF] Saving service grafana spec with placement smithi183=a;count:1 2024-02-13T03:21:40.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:40 smithi161 bash[19769]: audit 2024-02-13T03:21:40.544462+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:40.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:40 smithi161 bash[27879]: audit 2024-02-13T03:21:39.393678+0000 mgr.y (mgr.24473) 7 : audit [DBG] from='client.24493 v1:172.21.15.183:0/1358676025' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi183=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:40.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:40 smithi161 bash[27879]: cephadm 2024-02-13T03:21:39.395992+0000 mgr.y (mgr.24473) 8 : cephadm [INF] Saving service grafana spec with placement smithi183=a;count:1 2024-02-13T03:21:40.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:40 smithi161 bash[27879]: audit 2024-02-13T03:21:40.544462+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:40.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:40 smithi183 bash[20824]: audit 2024-02-13T03:21:39.393678+0000 mgr.y (mgr.24473) 7 : audit [DBG] from='client.24493 v1:172.21.15.183:0/1358676025' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi183=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T03:21:40.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:40 smithi183 bash[20824]: cephadm 2024-02-13T03:21:39.395992+0000 mgr.y (mgr.24473) 8 : cephadm [INF] Saving service grafana spec with placement smithi183=a;count:1 2024-02-13T03:21:40.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:40 smithi183 bash[20824]: audit 2024-02-13T03:21:40.544462+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: cluster 2024-02-13T03:21:40.538526+0000 mgr.y (mgr.24473) 9 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:40.832333+0000 mon.c (mon.2) 28 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:40.832932+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:40.834839+0000 mon.c (mon.2) 29 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:40.835412+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:40.836839+0000 mon.c (mon.2) 30 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:40.837443+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:40.838834+0000 mon.c (mon.2) 31 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:40.839438+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: cephadm 2024-02-13T03:21:40.840552+0000 mgr.y (mgr.24473) 10 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 3817M 2024-02-13T03:21:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:40.846603+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:41.269050+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:41.554942+0000 mon.c (mon.2) 32 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:41.555519+0000 mon.a (mon.0) 623 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:41.557215+0000 mon.c (mon.2) 33 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:41.557729+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:41.559316+0000 mon.c (mon.2) 34 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:41.559748+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:41.561194+0000 mon.c (mon.2) 35 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:41.561628+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:41.573786+0000 mon.a (mon.0) 627 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:41.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:41.577738+0000 mon.c (mon.2) 36 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:41.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[27879]: audit 2024-02-13T03:21:41.579045+0000 mon.c (mon.2) 37 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:21:41.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: cluster 2024-02-13T03:21:40.538526+0000 mgr.y (mgr.24473) 9 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:41.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:40.832333+0000 mon.c (mon.2) 28 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:40.832932+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:40.834839+0000 mon.c (mon.2) 29 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:40.835412+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:40.836839+0000 mon.c (mon.2) 30 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:40.837443+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:40.838834+0000 mon.c (mon.2) 31 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:40.839438+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: cephadm 2024-02-13T03:21:40.840552+0000 mgr.y (mgr.24473) 10 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 3817M 2024-02-13T03:21:41.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:40.846603+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:41.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:41.269050+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:41.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:41.554942+0000 mon.c (mon.2) 32 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:41.555519+0000 mon.a (mon.0) 623 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:41.557215+0000 mon.c (mon.2) 33 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:41.557729+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:41.559316+0000 mon.c (mon.2) 34 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:41.559748+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:41.561194+0000 mon.c (mon.2) 35 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:41.561628+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:41.573786+0000 mon.a (mon.0) 627 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:41.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:41.577738+0000 mon.c (mon.2) 36 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:41.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:41 smithi161 bash[19769]: audit 2024-02-13T03:21:41.579045+0000 mon.c (mon.2) 37 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:21:41.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: cluster 2024-02-13T03:21:40.538526+0000 mgr.y (mgr.24473) 9 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:21:41.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:40.832333+0000 mon.c (mon.2) 28 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:40.832932+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:40.834839+0000 mon.c (mon.2) 29 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:40.835412+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:40.836839+0000 mon.c (mon.2) 30 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:40.837443+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:40.838834+0000 mon.c (mon.2) 31 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:40.839438+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: cephadm 2024-02-13T03:21:40.840552+0000 mgr.y (mgr.24473) 10 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 3817M 2024-02-13T03:21:41.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:40.846603+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:41.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:41.269050+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:41.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:41.554942+0000 mon.c (mon.2) 32 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:41.555519+0000 mon.a (mon.0) 623 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:41.557215+0000 mon.c (mon.2) 33 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:41.557729+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:41.559316+0000 mon.c (mon.2) 34 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:41.559748+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:41.561194+0000 mon.c (mon.2) 35 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:41.561628+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:21:41.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:41.573786+0000 mon.a (mon.0) 627 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:41.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:41.577738+0000 mon.c (mon.2) 36 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:21:41.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:41 smithi183 bash[20824]: audit 2024-02-13T03:21:41.579045+0000 mon.c (mon.2) 37 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:21:42.256 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:42 smithi161 systemd[1]: Started Ceph node-exporter.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:21:42.646 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[19769]: cephadm 2024-02-13T03:21:41.562787+0000 mgr.y (mgr.24473) 11 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 4073M 2024-02-13T03:21:42.646 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[19769]: cephadm 2024-02-13T03:21:41.581274+0000 mgr.y (mgr.24473) 12 : cephadm [INF] Updating smithi161:/etc/ceph/ceph.conf 2024-02-13T03:21:42.647 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[19769]: cephadm 2024-02-13T03:21:41.581649+0000 mgr.y (mgr.24473) 13 : cephadm [INF] Updating smithi183:/etc/ceph/ceph.conf 2024-02-13T03:21:42.647 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[19769]: cephadm 2024-02-13T03:21:41.607527+0000 mgr.y (mgr.24473) 14 : cephadm [INF] Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-02-13T03:21:42.647 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[19769]: cephadm 2024-02-13T03:21:41.617353+0000 mgr.y (mgr.24473) 15 : cephadm [INF] Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-02-13T03:21:42.647 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[19769]: audit 2024-02-13T03:21:41.642711+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:42.647 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[19769]: audit 2024-02-13T03:21:41.666775+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:42.647 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[51269]: Unable to find image 'quay.io/prometheus/node-exporter:v1.3.1' locally 2024-02-13T03:21:42.770 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:42 smithi183 bash[20824]: cephadm 2024-02-13T03:21:41.562787+0000 mgr.y (mgr.24473) 11 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 4073M 2024-02-13T03:21:42.770 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:42 smithi183 bash[20824]: cephadm 2024-02-13T03:21:41.581274+0000 mgr.y (mgr.24473) 12 : cephadm [INF] Updating smithi161:/etc/ceph/ceph.conf 2024-02-13T03:21:42.770 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:42 smithi183 bash[20824]: cephadm 2024-02-13T03:21:41.581649+0000 mgr.y (mgr.24473) 13 : cephadm [INF] Updating smithi183:/etc/ceph/ceph.conf 2024-02-13T03:21:42.771 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:42 smithi183 bash[20824]: cephadm 2024-02-13T03:21:41.607527+0000 mgr.y (mgr.24473) 14 : cephadm [INF] Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-02-13T03:21:42.771 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:42 smithi183 bash[20824]: cephadm 2024-02-13T03:21:41.617353+0000 mgr.y (mgr.24473) 15 : cephadm [INF] Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-02-13T03:21:42.771 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:42 smithi183 bash[20824]: audit 2024-02-13T03:21:41.642711+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:42.771 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:42 smithi183 bash[20824]: audit 2024-02-13T03:21:41.666775+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:42.771 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:42 smithi183 bash[20824]: audit 2024-02-13T03:21:41.678182+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:42.771 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:42 smithi183 bash[20824]: cephadm 2024-02-13T03:21:41.682216+0000 mgr.y (mgr.24473) 16 : cephadm [INF] Deploying daemon node-exporter.a on smithi161 2024-02-13T03:21:42.771 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:42 smithi183 bash[20824]: audit 2024-02-13T03:21:42.223671+0000 mon.a (mon.0) 631 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:42.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[19769]: audit 2024-02-13T03:21:41.678182+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:42.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[19769]: cephadm 2024-02-13T03:21:41.682216+0000 mgr.y (mgr.24473) 16 : cephadm [INF] Deploying daemon node-exporter.a on smithi161 2024-02-13T03:21:42.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[19769]: audit 2024-02-13T03:21:42.223671+0000 mon.a (mon.0) 631 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[27879]: cephadm 2024-02-13T03:21:41.562787+0000 mgr.y (mgr.24473) 11 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 4073M 2024-02-13T03:21:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[27879]: cephadm 2024-02-13T03:21:41.581274+0000 mgr.y (mgr.24473) 12 : cephadm [INF] Updating smithi161:/etc/ceph/ceph.conf 2024-02-13T03:21:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[27879]: cephadm 2024-02-13T03:21:41.581649+0000 mgr.y (mgr.24473) 13 : cephadm [INF] Updating smithi183:/etc/ceph/ceph.conf 2024-02-13T03:21:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[27879]: cephadm 2024-02-13T03:21:41.607527+0000 mgr.y (mgr.24473) 14 : cephadm [INF] Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-02-13T03:21:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[27879]: cephadm 2024-02-13T03:21:41.617353+0000 mgr.y (mgr.24473) 15 : cephadm [INF] Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-02-13T03:21:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[27879]: audit 2024-02-13T03:21:41.642711+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[27879]: audit 2024-02-13T03:21:41.666775+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[27879]: audit 2024-02-13T03:21:41.678182+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[27879]: cephadm 2024-02-13T03:21:41.682216+0000 mgr.y (mgr.24473) 16 : cephadm [INF] Deploying daemon node-exporter.a on smithi161 2024-02-13T03:21:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:42 smithi161 bash[27879]: audit 2024-02-13T03:21:42.223671+0000 mon.a (mon.0) 631 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:43.239 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:42 smithi183 systemd[1]: Started Ceph node-exporter.b for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:21:43.240 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:42 smithi183 bash[45283]: Unable to find image 'quay.io/prometheus/node-exporter:v1.3.1' locally 2024-02-13T03:21:43.676 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[51269]: v1.3.1: Pulling from prometheus/node-exporter 2024-02-13T03:21:43.677 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[51269]: aa2a8d90b84c: Pulling fs layer 2024-02-13T03:21:43.677 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[51269]: b45d31ee2d7f: Pulling fs layer 2024-02-13T03:21:43.677 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[51269]: b5db1e299295: Pulling fs layer 2024-02-13T03:21:43.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.682+0000 7f0d3effd700 1 -- 172.21.15.161:0/3674182952 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f0d34002e60 con 0x7f0d40100730 2024-02-13T03:21:43.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.682+0000 7f0d466b8700 1 -- 172.21.15.161:0/3674182952 >> v1:172.21.15.161:6789/0 conn(0x7f0d40100730 legacy=0x7f0d40102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:43.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.682+0000 7f0d466b8700 1 -- 172.21.15.161:0/3674182952 shutdown_connections 2024-02-13T03:21:43.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.682+0000 7f0d466b8700 1 -- 172.21.15.161:0/3674182952 >> 172.21.15.161:0/3674182952 conn(0x7f0d4006c800 msgr2=0x7f0d4006cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:43.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.682+0000 7f0d466b8700 1 -- 172.21.15.161:0/3674182952 shutdown_connections 2024-02-13T03:21:43.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.682+0000 7f0d466b8700 1 -- 172.21.15.161:0/3674182952 wait complete. 2024-02-13T03:21:43.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.682+0000 7f0d466b8700 1 Processor -- start 2024-02-13T03:21:43.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.682+0000 7f0d466b8700 1 -- start start 2024-02-13T03:21:43.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.682+0000 7f0d466b8700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0d40117000 con 0x7f0d40104240 2024-02-13T03:21:43.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.682+0000 7f0d466b8700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0d401181e0 con 0x7f0d40100730 2024-02-13T03:21:43.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.682+0000 7f0d466b8700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0d401193e0 con 0x7f0d400fd970 2024-02-13T03:21:43.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.682+0000 7f0d3ffff700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f0d400fd970 0x7f0d400f95f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:60524/0 (socket says 172.21.15.161:60524) 2024-02-13T03:21:43.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.682+0000 7f0d3ffff700 1 -- 172.21.15.161:0/4257692010 learned_addr learned my addr 172.21.15.161:0/4257692010 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:21:43.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689874205 0 0) 0x7f0d40117000 con 0x7f0d40104240 2024-02-13T03:21:43.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d280033f0 con 0x7f0d40104240 2024-02-13T03:21:43.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606018137 0 0) 0x7f0d401193e0 con 0x7f0d400fd970 2024-02-13T03:21:43.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d40117000 con 0x7f0d400fd970 2024-02-13T03:21:43.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886012816 0 0) 0x7f0d401181e0 con 0x7f0d40100730 2024-02-13T03:21:43.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d401193e0 con 0x7f0d40100730 2024-02-13T03:21:43.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3802431494 0 0) 0x7f0d40117000 con 0x7f0d400fd970 2024-02-13T03:21:43.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0d401181e0 con 0x7f0d400fd970 2024-02-13T03:21:43.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f0d30001a50 con 0x7f0d400fd970 2024-02-13T03:21:43.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2091497377 0 0) 0x7f0d280033f0 con 0x7f0d40104240 2024-02-13T03:21:43.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0d40117000 con 0x7f0d40104240 2024-02-13T03:21:43.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f0d38001b40 con 0x7f0d40104240 2024-02-13T03:21:43.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 345099956 0 0) 0x7f0d401193e0 con 0x7f0d40100730 2024-02-13T03:21:43.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0d280033f0 con 0x7f0d40100730 2024-02-13T03:21:43.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2479615951 0 0) 0x7f0d401181e0 con 0x7f0d400fd970 2024-02-13T03:21:43.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 >> v1:172.21.15.183:6789/0 conn(0x7f0d40100730 legacy=0x7f0d400f9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:43.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 >> v1:172.21.15.161:6789/0 conn(0x7f0d40104240 legacy=0x7f0d401158e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:43.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4011a5e0 con 0x7f0d400fd970 2024-02-13T03:21:43.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d466b8700 1 -- 172.21.15.161:0/4257692010 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d401195d0 con 0x7f0d400fd970 2024-02-13T03:21:43.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d466b8700 1 -- 172.21.15.161:0/4257692010 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d40119ac0 con 0x7f0d400fd970 2024-02-13T03:21:43.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f0d300024b0 con 0x7f0d400fd970 2024-02-13T03:21:43.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f0d30005af0 con 0x7f0d400fd970 2024-02-13T03:21:43.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d466b8700 1 -- 172.21.15.161:0/4257692010 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2c004f20 con 0x7f0d400fd970 2024-02-13T03:21:43.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f0d30002010 con 0x7f0d400fd970 2024-02-13T03:21:43.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.686+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6056+0+0 (unknown 2145861735 0 0) 0x7f0d300783d0 con 0x7f0d400fd970 2024-02-13T03:21:43.701 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.694+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f0d3004bfa0 con 0x7f0d400fd970 2024-02-13T03:21:43.736 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:43 smithi183 bash[45283]: v1.3.1: Pulling from prometheus/node-exporter 2024-02-13T03:21:43.737 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:43 smithi183 bash[45283]: aa2a8d90b84c: Pulling fs layer 2024-02-13T03:21:43.737 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:43 smithi183 bash[45283]: b45d31ee2d7f: Pulling fs layer 2024-02-13T03:21:43.737 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:43 smithi183 bash[45283]: b5db1e299295: Pulling fs layer 2024-02-13T03:21:43.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.966+0000 7f0d466b8700 1 -- 172.21.15.161:0/4257692010 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f0d2c005750 con 0x7f0d400fd970 2024-02-13T03:21:43.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.974+0000 7f0d3d7fa700 1 -- 172.21.15.161:0/4257692010 <== mon.2 v1:172.21.15.161:6790/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (unknown 57899324 0 466188388) 0x7f0d3004a3b0 con 0x7f0d400fd970 2024-02-13T03:21:43.980 INFO:teuthology.orchestra.run.smithi161.stdout:[client.0] 2024-02-13T03:21:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQBH4Mplqz4VOhAAzLPMtc8NxBMm9T6skg1J2g== 2024-02-13T03:21:43.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.974+0000 7f0d466b8700 1 -- 172.21.15.161:0/4257692010 >> v1:172.21.15.161:6800/4114337139 conn(0x7f0d280622a0 legacy=0x7f0d28064760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:43.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.974+0000 7f0d466b8700 1 -- 172.21.15.161:0/4257692010 >> v1:172.21.15.161:6790/0 conn(0x7f0d400fd970 legacy=0x7f0d400f95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:43.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.974+0000 7f0d466b8700 1 -- 172.21.15.161:0/4257692010 shutdown_connections 2024-02-13T03:21:43.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.974+0000 7f0d466b8700 1 -- 172.21.15.161:0/4257692010 >> 172.21.15.161:0/4257692010 conn(0x7f0d4006c800 msgr2=0x7f0d4006d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:43.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.978+0000 7f0d466b8700 1 -- 172.21.15.161:0/4257692010 shutdown_connections 2024-02-13T03:21:43.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:21:43.978+0000 7f0d466b8700 1 -- 172.21.15.161:0/4257692010 wait complete. 2024-02-13T03:21:44.033 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[51269]: b45d31ee2d7f: Download complete 2024-02-13T03:21:44.033 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[51269]: aa2a8d90b84c: Verifying Checksum 2024-02-13T03:21:44.033 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[51269]: aa2a8d90b84c: Download complete 2024-02-13T03:21:44.033 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[51269]: b5db1e299295: Verifying Checksum 2024-02-13T03:21:44.033 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[51269]: b5db1e299295: Download complete 2024-02-13T03:21:44.033 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[51269]: aa2a8d90b84c: Pull complete 2024-02-13T03:21:44.033 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[27879]: cephadm 2024-02-13T03:21:42.225589+0000 mgr.y (mgr.24473) 17 : cephadm [INF] Deploying daemon node-exporter.b on smithi183 2024-02-13T03:21:44.034 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[27879]: cluster 2024-02-13T03:21:42.539932+0000 mgr.y (mgr.24473) 18 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 732 B/s rd, 0 op/s 2024-02-13T03:21:44.034 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[27879]: audit 2024-02-13T03:21:42.815400+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:44.034 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[27879]: cephadm 2024-02-13T03:21:42.842517+0000 mgr.y (mgr.24473) 19 : cephadm [INF] Deploying daemon prometheus.a on smithi183 2024-02-13T03:21:44.034 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[19769]: cephadm 2024-02-13T03:21:42.225589+0000 mgr.y (mgr.24473) 17 : cephadm [INF] Deploying daemon node-exporter.b on smithi183 2024-02-13T03:21:44.034 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[19769]: cluster 2024-02-13T03:21:42.539932+0000 mgr.y (mgr.24473) 18 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 732 B/s rd, 0 op/s 2024-02-13T03:21:44.034 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[19769]: audit 2024-02-13T03:21:42.815400+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:44.034 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:43 smithi161 bash[19769]: cephadm 2024-02-13T03:21:42.842517+0000 mgr.y (mgr.24473) 19 : cephadm [INF] Deploying daemon prometheus.a on smithi183 2024-02-13T03:21:44.082 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:43 smithi183 bash[20824]: cephadm 2024-02-13T03:21:42.225589+0000 mgr.y (mgr.24473) 17 : cephadm [INF] Deploying daemon node-exporter.b on smithi183 2024-02-13T03:21:44.082 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:43 smithi183 bash[20824]: cluster 2024-02-13T03:21:42.539932+0000 mgr.y (mgr.24473) 18 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 732 B/s rd, 0 op/s 2024-02-13T03:21:44.082 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:43 smithi183 bash[20824]: audit 2024-02-13T03:21:42.815400+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:44.082 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:43 smithi183 bash[20824]: cephadm 2024-02-13T03:21:42.842517+0000 mgr.y (mgr.24473) 19 : cephadm [INF] Deploying daemon prometheus.a on smithi183 2024-02-13T03:21:44.083 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:43 smithi183 bash[45283]: b45d31ee2d7f: Verifying Checksum 2024-02-13T03:21:44.083 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:43 smithi183 bash[45283]: b45d31ee2d7f: Download complete 2024-02-13T03:21:44.083 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:43 smithi183 bash[45283]: aa2a8d90b84c: Download complete 2024-02-13T03:21:44.083 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:43 smithi183 bash[45283]: b5db1e299295: Verifying Checksum 2024-02-13T03:21:44.083 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:43 smithi183 bash[45283]: b5db1e299295: Download complete 2024-02-13T03:21:44.486 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:44 smithi183 bash[45283]: aa2a8d90b84c: Pull complete 2024-02-13T03:21:44.935 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:21:44.936 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-13T03:21:44.936 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-13T03:21:44.953 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-13T03:21:44.962 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:44 smithi183 bash[45283]: b45d31ee2d7f: Pull complete 2024-02-13T03:21:44.963 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:44 smithi183 bash[20824]: audit 2024-02-13T03:21:43.973720+0000 mon.c (mon.2) 38 : audit [INF] from='client.? v1:172.21.15.161:0/4257692010' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T03:21:44.963 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:44 smithi183 bash[20824]: audit 2024-02-13T03:21:43.974316+0000 mon.a (mon.0) 633 : audit [INF] from='client.24503 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T03:21:44.963 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:44 smithi183 bash[20824]: audit 2024-02-13T03:21:43.978527+0000 mon.a (mon.0) 634 : audit [INF] from='client.24503 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T03:21:44.963 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:44 smithi183 bash[20824]: audit 2024-02-13T03:21:44.767132+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:45.177 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:44 smithi161 bash[51269]: b45d31ee2d7f: Pull complete 2024-02-13T03:21:45.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:44 smithi161 bash[27879]: audit 2024-02-13T03:21:43.973720+0000 mon.c (mon.2) 38 : audit [INF] from='client.? v1:172.21.15.161:0/4257692010' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T03:21:45.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:44 smithi161 bash[27879]: audit 2024-02-13T03:21:43.974316+0000 mon.a (mon.0) 633 : audit [INF] from='client.24503 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T03:21:45.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:44 smithi161 bash[27879]: audit 2024-02-13T03:21:43.978527+0000 mon.a (mon.0) 634 : audit [INF] from='client.24503 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T03:21:45.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:44 smithi161 bash[27879]: audit 2024-02-13T03:21:44.767132+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:45.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:44 smithi161 bash[19769]: audit 2024-02-13T03:21:43.973720+0000 mon.c (mon.2) 38 : audit [INF] from='client.? v1:172.21.15.161:0/4257692010' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T03:21:45.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:44 smithi161 bash[19769]: audit 2024-02-13T03:21:43.974316+0000 mon.a (mon.0) 633 : audit [INF] from='client.24503 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T03:21:45.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:44 smithi161 bash[19769]: audit 2024-02-13T03:21:43.978527+0000 mon.a (mon.0) 634 : audit [INF] from='client.24503 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T03:21:45.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:44 smithi161 bash[19769]: audit 2024-02-13T03:21:44.767132+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:45.236 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:45 smithi183 bash[45283]: b5db1e299295: Pull complete 2024-02-13T03:21:45.236 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:45 smithi183 bash[45283]: Digest: sha256:f2269e73124dd0f60a7d19a2ce1264d33d08a985aed0ee6b0b89d0be470592cd 2024-02-13T03:21:45.554 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:45 smithi183 bash[45283]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.3.1 2024-02-13T03:21:45.677 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:45 smithi161 bash[51269]: b5db1e299295: Pull complete 2024-02-13T03:21:45.677 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:45 smithi161 bash[51269]: Digest: sha256:f2269e73124dd0f60a7d19a2ce1264d33d08a985aed0ee6b0b89d0be470592cd 2024-02-13T03:21:45.677 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:45 smithi161 bash[51269]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.3.1 2024-02-13T03:21:45.983 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:21:45 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:21:46.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:45 smithi183 bash[20824]: cluster 2024-02-13T03:21:44.540720+0000 mgr.y (mgr.24473) 20 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 569 B/s rd, 0 op/s 2024-02-13T03:21:46.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:45 smithi161 bash[19769]: cluster 2024-02-13T03:21:44.540720+0000 mgr.y (mgr.24473) 20 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 569 B/s rd, 0 op/s 2024-02-13T03:21:46.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:45 smithi161 bash[27879]: cluster 2024-02-13T03:21:44.540720+0000 mgr.y (mgr.24473) 20 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 569 B/s rd, 0 op/s 2024-02-13T03:21:46.928 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.812Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-02-13T03:21:46.928 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.812Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-02-13T03:21:46.928 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.813Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-02-13T03:21:46.928 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.813Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-13T03:21:46.928 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.813Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-02-13T03:21:46.928 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.813Z caller=node_exporter.go:115 level=info collector=arp 2024-02-13T03:21:46.928 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.813Z caller=node_exporter.go:115 level=info collector=bcache 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.813Z caller=node_exporter.go:115 level=info collector=bonding 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.813Z caller=node_exporter.go:115 level=info collector=btrfs 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.813Z caller=node_exporter.go:115 level=info collector=conntrack 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.813Z caller=node_exporter.go:115 level=info collector=cpu 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.813Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.813Z caller=node_exporter.go:115 level=info collector=diskstats 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=dmi 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=edac 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=entropy 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=filefd 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=filesystem 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=hwmon 2024-02-13T03:21:46.929 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=infiniband 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=ipvs 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=loadavg 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=mdadm 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=meminfo 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=netclass 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=netdev 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=netstat 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=nfs 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=nfsd 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=nvme 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=os 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=pressure 2024-02-13T03:21:46.930 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=rapl 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=schedstat 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=sockstat 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=softnet 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=stat 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=tapestats 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=textfile 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=time 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=uname 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=vmstat 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=xfs 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:115 level=info collector=zfs 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-02-13T03:21:46.931 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[51269]: ts=2024-02-13T03:21:46.814Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-02-13T03:21:47.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[19769]: audit 2024-02-13T03:21:45.551641+0000 mgr.y (mgr.24473) 21 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:21:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:46 smithi161 bash[27879]: audit 2024-02-13T03:21:45.551641+0000 mgr.y (mgr.24473) 21 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:21:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:46 smithi183 bash[20824]: audit 2024-02-13T03:21:45.551641+0000 mgr.y (mgr.24473) 21 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:21:48.356 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:47 smithi183 bash[20824]: cluster 2024-02-13T03:21:46.542329+0000 mgr.y (mgr.24473) 22 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 931 B/s rd, 0 op/s 2024-02-13T03:21:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:47 smithi161 bash[19769]: cluster 2024-02-13T03:21:46.542329+0000 mgr.y (mgr.24473) 22 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 931 B/s rd, 0 op/s 2024-02-13T03:21:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:47 smithi161 bash[27879]: cluster 2024-02-13T03:21:46.542329+0000 mgr.y (mgr.24473) 22 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 931 B/s rd, 0 op/s 2024-02-13T03:21:48.685 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.582Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-02-13T03:21:48.685 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.582Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-02-13T03:21:48.685 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.582Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-02-13T03:21:48.685 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.582Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=arp 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=bcache 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=bonding 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=btrfs 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=conntrack 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=cpu 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=diskstats 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=dmi 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=edac 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=entropy 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-02-13T03:21:48.686 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=filefd 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=filesystem 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=hwmon 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=infiniband 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=ipvs 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=loadavg 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=mdadm 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=meminfo 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=netclass 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=netdev 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=netstat 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=nfs 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.584Z caller=node_exporter.go:115 level=info collector=nfsd 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=nvme 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=os 2024-02-13T03:21:48.687 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-02-13T03:21:48.688 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=pressure 2024-02-13T03:21:48.688 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=rapl 2024-02-13T03:21:48.688 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=schedstat 2024-02-13T03:21:48.688 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=sockstat 2024-02-13T03:21:48.688 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=softnet 2024-02-13T03:21:48.688 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=stat 2024-02-13T03:21:48.688 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=tapestats 2024-02-13T03:21:48.688 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=textfile 2024-02-13T03:21:48.688 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-02-13T03:21:48.688 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=time 2024-02-13T03:21:48.688 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-02-13T03:21:48.688 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=uname 2024-02-13T03:21:48.689 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=vmstat 2024-02-13T03:21:48.689 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=xfs 2024-02-13T03:21:48.689 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:115 level=info collector=zfs 2024-02-13T03:21:48.689 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-02-13T03:21:48.689 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:21:48 smithi183 bash[45283]: ts=2024-02-13T03:21:48.585Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-02-13T03:21:50.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:49 smithi161 bash[27879]: cluster 2024-02-13T03:21:48.543113+0000 mgr.y (mgr.24473) 23 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:21:50.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:49 smithi161 bash[19769]: cluster 2024-02-13T03:21:48.543113+0000 mgr.y (mgr.24473) 23 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:21:50.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:50 smithi183 bash[20824]: cluster 2024-02-13T03:21:48.543113+0000 mgr.y (mgr.24473) 23 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:21:51.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.411+0000 7fc91f354700 1 -- 172.21.15.183:0/2238610492 >> v1:172.21.15.183:6789/0 conn(0x7fc918104240 legacy=0x7fc918106700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:51.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.411+0000 7fc91f354700 1 -- 172.21.15.183:0/2238610492 shutdown_connections 2024-02-13T03:21:51.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.411+0000 7fc91f354700 1 -- 172.21.15.183:0/2238610492 >> 172.21.15.183:0/2238610492 conn(0x7fc91806c800 msgr2=0x7fc91806cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:51.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.411+0000 7fc91f354700 1 -- 172.21.15.183:0/2238610492 shutdown_connections 2024-02-13T03:21:51.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.411+0000 7fc91f354700 1 -- 172.21.15.183:0/2238610492 wait complete. 2024-02-13T03:21:51.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.411+0000 7fc91f354700 1 Processor -- start 2024-02-13T03:21:51.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.411+0000 7fc91f354700 1 -- start start 2024-02-13T03:21:51.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc91f354700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc9180f8530 con 0x7fc918100730 2024-02-13T03:21:51.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc91f354700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc9180f8700 con 0x7fc9180fd970 2024-02-13T03:21:51.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc91f354700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc9180f88f0 con 0x7fc918104240 2024-02-13T03:21:51.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc91d0f0700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fc9180fd970 0x7fc9180fc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:33956/0 (socket says 172.21.15.183:33956) 2024-02-13T03:21:51.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc91d0f0700 1 -- 172.21.15.183:0/1166534425 learned_addr learned my addr 172.21.15.183:0/1166534425 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-02-13T03:21:51.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031034604 0 0) 0x7fc9180f8700 con 0x7fc9180fd970 2024-02-13T03:21:51.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc904003030 con 0x7fc9180fd970 2024-02-13T03:21:51.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128559640 0 0) 0x7fc9180f88f0 con 0x7fc918104240 2024-02-13T03:21:51.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9180f8700 con 0x7fc918104240 2024-02-13T03:21:51.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588093840 0 0) 0x7fc9180f8530 con 0x7fc918100730 2024-02-13T03:21:51.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9180f88f0 con 0x7fc918100730 2024-02-13T03:21:51.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1430457140 0 0) 0x7fc904003030 con 0x7fc9180fd970 2024-02-13T03:21:51.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc9180f8530 con 0x7fc9180fd970 2024-02-13T03:21:51.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fc914001910 con 0x7fc9180fd970 2024-02-13T03:21:51.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 <== mon.1 v1:172.21.15.183:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2612873624 0 0) 0x7fc9180f8530 con 0x7fc9180fd970 2024-02-13T03:21:51.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 >> v1:172.21.15.161:6790/0 conn(0x7fc918104240 legacy=0x7fc9181183e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:51.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 >> v1:172.21.15.161:6789/0 conn(0x7fc918100730 legacy=0x7fc918114cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:51.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91811cb30 con 0x7fc9180fd970 2024-02-13T03:21:51.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc91f354700 1 -- 172.21.15.183:0/1166534425 --> v1:172.21.15.183:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc918119b60 con 0x7fc9180fd970 2024-02-13T03:21:51.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc91f354700 1 -- 172.21.15.183:0/1166534425 --> v1:172.21.15.183:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91811a050 con 0x7fc9180fd970 2024-02-13T03:21:51.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 <== mon.1 v1:172.21.15.183:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fc914002420 con 0x7fc9180fd970 2024-02-13T03:21:51.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 <== mon.1 v1:172.21.15.183:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fc914005a90 con 0x7fc9180fd970 2024-02-13T03:21:51.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc91f354700 1 -- 172.21.15.183:0/1166534425 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8fc004f50 con 0x7fc9180fd970 2024-02-13T03:21:51.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.415+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 <== mon.1 v1:172.21.15.183:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fc914006d60 con 0x7fc9180fd970 2024-02-13T03:21:51.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.419+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 <== mon.1 v1:172.21.15.183:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6056+0+0 (unknown 2145861735 0 0) 0x7fc914078630 con 0x7fc9180fd970 2024-02-13T03:21:51.427 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.423+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 <== mon.1 v1:172.21.15.183:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fc91404c260 con 0x7fc9180fd970 2024-02-13T03:21:51.677 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.675+0000 7fc91f354700 1 -- 172.21.15.183:0/1166534425 --> v1:172.21.15.183:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc8fc005780 con 0x7fc9180fd970 2024-02-13T03:21:51.687 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.683+0000 7fc90e7fc700 1 -- 172.21.15.183:0/1166534425 <== mon.1 v1:172.21.15.183:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v18) v1 ==== 170+0+59 (unknown 3508017507 0 3750871453) 0x7fc91404a670 con 0x7fc9180fd970 2024-02-13T03:21:51.687 INFO:teuthology.orchestra.run.smithi183.stdout:[client.1] 2024-02-13T03:21:51.687 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBP4Mplde5eKBAAVPdx6pFbjIFTK3DGIC5Hdg== 2024-02-13T03:21:51.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.687+0000 7fc91f354700 1 -- 172.21.15.183:0/1166534425 >> v1:172.21.15.161:6800/4114337139 conn(0x7fc904062200 legacy=0x7fc9040646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:51.689 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.687+0000 7fc91f354700 1 -- 172.21.15.183:0/1166534425 >> v1:172.21.15.183:6789/0 conn(0x7fc9180fd970 legacy=0x7fc9180fc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:21:51.690 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.687+0000 7fc91f354700 1 -- 172.21.15.183:0/1166534425 shutdown_connections 2024-02-13T03:21:51.690 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.687+0000 7fc91f354700 1 -- 172.21.15.183:0/1166534425 >> 172.21.15.183:0/1166534425 conn(0x7fc91806c800 msgr2=0x7fc91806d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:21:51.690 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.687+0000 7fc91f354700 1 -- 172.21.15.183:0/1166534425 shutdown_connections 2024-02-13T03:21:51.690 INFO:teuthology.orchestra.run.smithi183.stderr:2024-02-13T03:21:51.687+0000 7fc91f354700 1 -- 172.21.15.183:0/1166534425 wait complete. 2024-02-13T03:21:52.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:52 smithi161 bash[27879]: cluster 2024-02-13T03:21:50.544100+0000 mgr.y (mgr.24473) 24 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:21:52.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:52 smithi161 bash[27879]: audit 2024-02-13T03:21:51.676830+0000 mon.a (mon.0) 636 : audit [INF] from='client.24505 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T03:21:52.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:52 smithi161 bash[27879]: audit 2024-02-13T03:21:51.679236+0000 mon.b (mon.1) 25 : audit [INF] from='client.? v1:172.21.15.183:0/1166534425' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T03:21:52.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:52 smithi161 bash[27879]: audit 2024-02-13T03:21:51.683591+0000 mon.a (mon.0) 637 : audit [INF] from='client.24505 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T03:21:52.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:52 smithi161 bash[19769]: cluster 2024-02-13T03:21:50.544100+0000 mgr.y (mgr.24473) 24 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:21:52.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:52 smithi161 bash[19769]: audit 2024-02-13T03:21:51.676830+0000 mon.a (mon.0) 636 : audit [INF] from='client.24505 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T03:21:52.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:52 smithi161 bash[19769]: audit 2024-02-13T03:21:51.679236+0000 mon.b (mon.1) 25 : audit [INF] from='client.? v1:172.21.15.183:0/1166534425' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T03:21:52.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:52 smithi161 bash[19769]: audit 2024-02-13T03:21:51.683591+0000 mon.a (mon.0) 637 : audit [INF] from='client.24505 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T03:21:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:52 smithi183 bash[20824]: cluster 2024-02-13T03:21:50.544100+0000 mgr.y (mgr.24473) 24 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:21:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:52 smithi183 bash[20824]: audit 2024-02-13T03:21:51.676830+0000 mon.a (mon.0) 636 : audit [INF] from='client.24505 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T03:21:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:52 smithi183 bash[20824]: audit 2024-02-13T03:21:51.679236+0000 mon.b (mon.1) 25 : audit [INF] from='client.? v1:172.21.15.183:0/1166534425' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T03:21:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:52 smithi183 bash[20824]: audit 2024-02-13T03:21:51.683591+0000 mon.a (mon.0) 637 : audit [INF] from='client.24505 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T03:21:52.579 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-02-13T03:21:52.579 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-13T03:21:52.579 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-13T03:21:52.643 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-13T03:21:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:53 smithi161 bash[27879]: cluster 2024-02-13T03:21:52.545357+0000 mgr.y (mgr.24473) 25 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:21:53.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:53 smithi161 bash[19769]: cluster 2024-02-13T03:21:52.545357+0000 mgr.y (mgr.24473) 25 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:21:53.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:53 smithi183 bash[20824]: cluster 2024-02-13T03:21:52.545357+0000 mgr.y (mgr.24473) 25 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:21:55.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:55 smithi183 bash[20824]: cluster 2024-02-13T03:21:54.546121+0000 mgr.y (mgr.24473) 26 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:21:55.850 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:21:55 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:21:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:55 smithi161 bash[19769]: cluster 2024-02-13T03:21:54.546121+0000 mgr.y (mgr.24473) 26 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:21:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:55 smithi161 bash[27879]: cluster 2024-02-13T03:21:54.546121+0000 mgr.y (mgr.24473) 26 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:21:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:56 smithi183 bash[20824]: audit 2024-02-13T03:21:55.552972+0000 mgr.y (mgr.24473) 27 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:21:56.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:56 smithi161 bash[19769]: audit 2024-02-13T03:21:55.552972+0000 mgr.y (mgr.24473) 27 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:21:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:56 smithi161 bash[27879]: audit 2024-02-13T03:21:55.552972+0000 mgr.y (mgr.24473) 27 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:21:57.237 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:56 smithi183 systemd[1]: Started Ceph prometheus.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:21:57.465 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:21:57.953 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[20824]: cluster 2024-02-13T03:21:56.547611+0000 mgr.y (mgr.24473) 28 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:21:57.953 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[20824]: audit 2024-02-13T03:21:56.785440+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:57.953 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[20824]: cephadm 2024-02-13T03:21:56.800920+0000 mgr.y (mgr.24473) 29 : cephadm [INF] Deploying daemon alertmanager.a on smithi161 2024-02-13T03:21:57.953 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.922Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-02-13T03:21:57.953 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.922Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-02-13T03:21:57.953 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.922Z caller=main.go:518 level=info host_details="(Linux 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 smithi183 (none))" 2024-02-13T03:21:57.953 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.922Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-13T03:21:57.953 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.922Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-13T03:21:57.954 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.924Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-02-13T03:21:57.954 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.925Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-02-13T03:21:57.954 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.926Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-02-13T03:21:57.954 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.930Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-13T03:21:57.954 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.930Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.228µs 2024-02-13T03:21:57.954 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.930Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-13T03:21:57.954 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.931Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-13T03:21:57.954 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.931Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=56.661µs wal_replay_duration=992.908µs total_replay_duration=1.072459ms 2024-02-13T03:21:57.954 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.933Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-02-13T03:21:57.954 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.933Z caller=main.go:947 level=info msg="TSDB started" 2024-02-13T03:21:57.954 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.933Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-13T03:21:58.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:57 smithi161 bash[27879]: cluster 2024-02-13T03:21:56.547611+0000 mgr.y (mgr.24473) 28 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:21:58.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:57 smithi161 bash[27879]: audit 2024-02-13T03:21:56.785440+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:58.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:57 smithi161 bash[27879]: cephadm 2024-02-13T03:21:56.800920+0000 mgr.y (mgr.24473) 29 : cephadm [INF] Deploying daemon alertmanager.a on smithi161 2024-02-13T03:21:58.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:57 smithi161 bash[19769]: cluster 2024-02-13T03:21:56.547611+0000 mgr.y (mgr.24473) 28 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:21:58.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:57 smithi161 bash[19769]: audit 2024-02-13T03:21:56.785440+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:21:58.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:57 smithi161 bash[19769]: cephadm 2024-02-13T03:21:56.800920+0000 mgr.y (mgr.24473) 29 : cephadm [INF] Deploying daemon alertmanager.a on smithi161 2024-02-13T03:21:58.237 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.954Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=21.146409ms db_storage=842ns remote_storage=1.719µs web_handler=397ns query_engine=818ns scrape=320.038µs scrape_sd=32.365µs notify=1.087µs notify_sd=1.742µs rules=20.531003ms 2024-02-13T03:21:58.237 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:21:57 smithi183 bash[46136]: ts=2024-02-13T03:21:57.954Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-02-13T03:22:00.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:21:59 smithi161 bash[27879]: cluster 2024-02-13T03:21:58.548264+0000 mgr.y (mgr.24473) 30 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:21:59 smithi161 bash[19769]: cluster 2024-02-13T03:21:58.548264+0000 mgr.y (mgr.24473) 30 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:00.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:21:59 smithi183 bash[20824]: cluster 2024-02-13T03:21:58.548264+0000 mgr.y (mgr.24473) 30 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:01.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:00 smithi161 bash[27879]: audit 2024-02-13T03:21:59.800814+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:01.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:00 smithi161 bash[19769]: audit 2024-02-13T03:21:59.800814+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:01.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:00 smithi183 bash[20824]: audit 2024-02-13T03:21:59.800814+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:01.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.701+0000 7fb70009f700 1 -- 172.21.15.161:0/2505338058 >> v1:172.21.15.161:6790/0 conn(0x7fb6f80f9650 legacy=0x7fb6f8102be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:01.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.701+0000 7fb70009f700 1 -- 172.21.15.161:0/2505338058 shutdown_connections 2024-02-13T03:22:01.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.701+0000 7fb70009f700 1 -- 172.21.15.161:0/2505338058 >> 172.21.15.161:0/2505338058 conn(0x7fb6f80f2610 msgr2=0x7fb6f80f4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:01.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.701+0000 7fb70009f700 1 -- 172.21.15.161:0/2505338058 shutdown_connections 2024-02-13T03:22:01.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.701+0000 7fb70009f700 1 -- 172.21.15.161:0/2505338058 wait complete. 2024-02-13T03:22:01.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.701+0000 7fb70009f700 1 Processor -- start 2024-02-13T03:22:01.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.701+0000 7fb70009f700 1 -- start start 2024-02-13T03:22:01.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.701+0000 7fb70009f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb6f8102cc0 con 0x7fb6f8104430 2024-02-13T03:22:01.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.701+0000 7fb70009f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb6f8102e90 con 0x7fb6f80f8b60 2024-02-13T03:22:01.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.701+0000 7fb70009f700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb6f8103080 con 0x7fb6f80f9650 2024-02-13T03:22:01.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.701+0000 7fb6fe63c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb6f8104430 0x7fb6f81185b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58236/0 (socket says 172.21.15.161:58236) 2024-02-13T03:22:01.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.701+0000 7fb6fe63c700 1 -- 172.21.15.161:0/24391547 learned_addr learned my addr 172.21.15.161:0/24391547 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:22:01.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696673561 0 0) 0x7fb6f8102cc0 con 0x7fb6f8104430 2024-02-13T03:22:01.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6e40033f0 con 0x7fb6f8104430 2024-02-13T03:22:01.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644664460 0 0) 0x7fb6f8103080 con 0x7fb6f80f9650 2024-02-13T03:22:01.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6f8102cc0 con 0x7fb6f80f9650 2024-02-13T03:22:01.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674801146 0 0) 0x7fb6f8102e90 con 0x7fb6f80f8b60 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6f8103080 con 0x7fb6f80f8b60 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1607432279 0 0) 0x7fb6e40033f0 con 0x7fb6f8104430 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb6f8102e90 con 0x7fb6f8104430 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb6f0002cb0 con 0x7fb6f8104430 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1772954299 0 0) 0x7fb6f8103080 con 0x7fb6f80f8b60 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb6e40033f0 con 0x7fb6f80f8b60 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3798473671 0 0) 0x7fb6f8102cc0 con 0x7fb6f80f9650 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb6f8103080 con 0x7fb6f80f9650 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb6f4001840 con 0x7fb6f80f8b60 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb6e8001ec0 con 0x7fb6f80f9650 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2685273328 0 0) 0x7fb6f8102e90 con 0x7fb6f8104430 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 >> v1:172.21.15.161:6790/0 conn(0x7fb6f80f9650 legacy=0x7fb6f8114e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 >> v1:172.21.15.183:6789/0 conn(0x7fb6f80f8b60 legacy=0x7fb6f8102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f811cd00 con 0x7fb6f8104430 2024-02-13T03:22:01.713 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb70009f700 1 -- 172.21.15.161:0/24391547 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f8119d30 con 0x7fb6f8104430 2024-02-13T03:22:01.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb70009f700 1 -- 172.21.15.161:0/24391547 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f811a1a0 con 0x7fb6f8104430 2024-02-13T03:22:01.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fb6f0003020 con 0x7fb6f8104430 2024-02-13T03:22:01.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb6f0005ee0 con 0x7fb6f8104430 2024-02-13T03:22:01.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb70009f700 1 -- 172.21.15.161:0/24391547 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6dc004f20 con 0x7fb6f8104430 2024-02-13T03:22:01.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fb6f00019e0 con 0x7fb6f8104430 2024-02-13T03:22:01.714 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.705+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6056+0+0 (unknown 2145861735 0 0) 0x7fb6f00787f0 con 0x7fb6f8104430 2024-02-13T03:22:01.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.713+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fb6f004c420 con 0x7fb6f8104430 2024-02-13T03:22:01.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.897+0000 7fb70009f700 1 -- 172.21.15.161:0/24391547 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6dc0051d0 con 0x7fb6f8104430 2024-02-13T03:22:01.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.901+0000 7fb6eeffd700 1 -- 172.21.15.161:0/24391547 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+19055 (unknown 1877459060 0 3332835232) 0x7fb6f004a890 con 0x7fb6f8104430 2024-02-13T03:22:01.908 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-02-13T03:22:01.908 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":61,"fsid":"b9b3ef98-ca1d-11ee-95b9-87774f69a715","created":"2024-02-13T03:13:05.822934+0000","modified":"2024-02-13T03:21:34.516457+0000","last_up_change":"2024-02-13T03:20:54.783700+0000","last_in_change":"2024-02-13T03:20:29.453920+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-13T03:17:31.445899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-13T03:21:07.091767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-13T03:21:08.103932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-13T03:21:10.382753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-13T03:21:12.105682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-13T03:21:14.239505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"cf603e39-02f0-4508-b426-6fdd8402f1b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6801","nonce":2012328581}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6802","nonce":2012328581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6804","nonce":2012328581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6803","nonce":2012328581}]},"public_addr":"172.21.15.161:6801/2012328581","cluster_addr":"172.21.15.161:6802/2012328581","heartbeat_back_addr":"172.21.15.161:6804/2012328581","heartbeat_front_addr":"172.21.15.161:6803/2012328581","state":["exists","up"]},{"osd":1,"uuid":"3bda8b4b-0bd5-47f1-a699-93a180d0a278","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6805","nonce":72291598}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6806","nonce":72291598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6808","nonce":72291598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6807","nonce":72291598}]},"public_addr":"172.21.15.161:6805/72291598","cluster_addr":"172.21.15.161:6806/72291598","heartbeat_back_addr":"172.21.15.161:6808/72291598","heartbeat_front_addr":"172.21.15.161:6807/72291598","state":["exists","up"]},{"osd":2,"uuid":"c521de51-8f7c-4ec2-94b7-96ca43b52d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6809","nonce":1657207182}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6810","nonce":1657207182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6812","nonce":1657207182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6811","nonce":1657207182}]},"public_addr":"172.21.15.161:6809/1657207182","cluster_addr":"172.21.15.161:6810/1657207182","heartbeat_back_addr":"172.21.15.161:6812/1657207182","heartbeat_front_addr":"172.21.15.161:6811/1657207182","state":["exists","up"]},{"osd":3,"uuid":"871101d1-6af5-47da-9d56-085de6869077","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6813","nonce":2788336304}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6814","nonce":2788336304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6816","nonce":2788336304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6815","nonce":2788336304}]},"public_addr":"172.21.15.161:6813/2788336304","cluster_addr":"172.21.15.161:6814/2788336304","heartbeat_back_addr":"172.21.15.161:6816/2788336304","heartbeat_front_addr":"172.21.15.161:6815/2788336304","state":["exists","up"]},{"osd":4,"uuid":"02e5c987-89d4-490f-b6cb-be43e286cb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6800","nonce":384629816}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6801","nonce":384629816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6803","nonce":384629816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6802","nonce":384629816}]},"public_addr":"172.21.15.183:6800/384629816","cluster_addr":"172.21.15.183:6801/384629816","heartbeat_back_addr":"172.21.15.183:6803/384629816","heartbeat_front_addr":"172.21.15.183:6802/384629816","state":["exists","up"]},{"osd":5,"uuid":"68fc8d37-f86b-4f2e-8cfe-5e47161d27c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6804","nonce":2958722347}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6805","nonce":2958722347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6807","nonce":2958722347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6806","nonce":2958722347}]},"public_addr":"172.21.15.183:6804/2958722347","cluster_addr":"172.21.15.183:6805/2958722347","heartbeat_back_addr":"172.21.15.183:6807/2958722347","heartbeat_front_addr":"172.21.15.183:6806/2958722347","state":["exists","up"]},{"osd":6,"uuid":"aad4c7d8-2a9d-46f2-bf74-fb1bfa83a64c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6808","nonce":3716679598}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6809","nonce":3716679598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6811","nonce":3716679598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6810","nonce":3716679598}]},"public_addr":"172.21.15.183:6808/3716679598","cluster_addr":"172.21.15.183:6809/3716679598","heartbeat_back_addr":"172.21.15.183:6811/3716679598","heartbeat_front_addr":"172.21.15.183:6810/3716679598","state":["exists","up"]},{"osd":7,"uuid":"4dc175e5-44c6-4454-81e3-92a465398a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6812","nonce":1821141317}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6813","nonce":1821141317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6815","nonce":1821141317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6814","nonce":1821141317}]},"public_addr":"172.21.15.183:6812/1821141317","cluster_addr":"172.21.15.183:6813/1821141317","heartbeat_back_addr":"172.21.15.183:6815/1821141317","heartbeat_front_addr":"172.21.15.183:6814/1821141317","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:16:07.873972+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:16:47.940673+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:17:28.981004+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:18:10.307597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:18:49.391774+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:19:29.925899+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:20:11.948394+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:20:52.997527+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/2234015705":"2024-02-14T03:21:34.516409+0000","172.21.15.161:6800/1187535987":"2024-02-14T03:21:34.516409+0000","172.21.15.161:0/3385104551":"2024-02-14T03:14:02.911245+0000","172.21.15.161:0/2943775157":"2024-02-14T03:14:02.911245+0000","172.21.15.161:6800/2381271555":"2024-02-14T03:14:02.911245+0000","172.21.15.161:6800/1159674441":"2024-02-14T03:13:33.423519+0000","172.21.15.161:0/2907902373":"2024-02-14T03:21:34.516409+0000","172.21.15.161:0/3042112516":"2024-02-14T03:21:34.516409+0000","172.21.15.161:0/2926342541":"2024-02-14T03:13:33.423519+0000","172.21.15.161:0/869165781":"2024-02-14T03:13:33.423519+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-13T03:22:01.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.901+0000 7fb70009f700 1 -- 172.21.15.161:0/24391547 >> v1:172.21.15.161:6800/4114337139 conn(0x7fb6e40623b0 legacy=0x7fb6e4064870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:01.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.901+0000 7fb70009f700 1 -- 172.21.15.161:0/24391547 >> v1:172.21.15.161:6789/0 conn(0x7fb6f8104430 legacy=0x7fb6f81185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:01.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.901+0000 7fb70009f700 1 -- 172.21.15.161:0/24391547 shutdown_connections 2024-02-13T03:22:01.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.901+0000 7fb70009f700 1 -- 172.21.15.161:0/24391547 >> 172.21.15.161:0/24391547 conn(0x7fb6f80f2610 msgr2=0x7fb6f81078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:01.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.901+0000 7fb70009f700 1 -- 172.21.15.161:0/24391547 shutdown_connections 2024-02-13T03:22:01.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:01.901+0000 7fb70009f700 1 -- 172.21.15.161:0/24391547 wait complete. 2024-02-13T03:22:02.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:01 smithi161 bash[27879]: cluster 2024-02-13T03:22:00.549693+0000 mgr.y (mgr.24473) 31 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:02.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:01 smithi161 bash[19769]: cluster 2024-02-13T03:22:00.549693+0000 mgr.y (mgr.24473) 31 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:02.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:01 smithi183 bash[20824]: cluster 2024-02-13T03:22:00.549693+0000 mgr.y (mgr.24473) 31 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:02.907 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-13T03:17:31.445899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'iscsi', 'create_time': '2024-02-13T03:21:07.091767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-02-13T03:21:08.103932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-13T03:21:10.382753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-13T03:21:12.105682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-13T03:21:14.239505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}] 2024-02-13T03:22:02.907 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd pool get device_health_metrics pg_num 2024-02-13T03:22:03.067 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:02 smithi161 bash[27879]: audit 2024-02-13T03:22:01.905421+0000 mon.a (mon.0) 640 : audit [DBG] from='client.? v1:172.21.15.161:0/24391547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T03:22:03.068 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:02 smithi161 bash[19769]: audit 2024-02-13T03:22:01.905421+0000 mon.a (mon.0) 640 : audit [DBG] from='client.? v1:172.21.15.161:0/24391547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T03:22:03.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:02 smithi183 bash[20824]: audit 2024-02-13T03:22:01.905421+0000 mon.a (mon.0) 640 : audit [DBG] from='client.? v1:172.21.15.161:0/24391547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T03:22:04.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:03 smithi161 bash[19769]: cluster 2024-02-13T03:22:02.550553+0000 mgr.y (mgr.24473) 32 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:04.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:03 smithi161 bash[27879]: cluster 2024-02-13T03:22:02.550553+0000 mgr.y (mgr.24473) 32 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:04.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:03 smithi183 bash[20824]: cluster 2024-02-13T03:22:02.550553+0000 mgr.y (mgr.24473) 32 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:05.819 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:22:05 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:22:06.058 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:05 smithi161 bash[19769]: cluster 2024-02-13T03:22:04.551277+0000 mgr.y (mgr.24473) 33 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:06.059 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:05 smithi161 bash[27879]: cluster 2024-02-13T03:22:04.551277+0000 mgr.y (mgr.24473) 33 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:06.059 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[20075]: [13/Feb/2024:03:22:06] ENGINE Bus STOPPING 2024-02-13T03:22:06.060 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:05 smithi161 systemd[1]: Started Ceph alertmanager.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:22:06.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:05 smithi183 bash[20824]: cluster 2024-02-13T03:22:04.551277+0000 mgr.y (mgr.24473) 33 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:06.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[20075]: [13/Feb/2024:03:22:06] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T03:22:06.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[20075]: [13/Feb/2024:03:22:06] ENGINE Bus STOPPED 2024-02-13T03:22:06.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[20075]: [13/Feb/2024:03:22:06] ENGINE Bus STARTING 2024-02-13T03:22:06.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[20075]: [13/Feb/2024:03:22:06] ENGINE Serving on http://:::9283 2024-02-13T03:22:06.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[20075]: [13/Feb/2024:03:22:06] ENGINE Bus STARTED 2024-02-13T03:22:07.110 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[19769]: audit 2024-02-13T03:22:05.564223+0000 mgr.y (mgr.24473) 34 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:07.111 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[19769]: audit 2024-02-13T03:22:05.848324+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:07.111 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[19769]: audit 2024-02-13T03:22:06.002049+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:07.111 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[19769]: audit 2024-02-13T03:22:06.013359+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:07.111 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[19769]: audit 2024-02-13T03:22:06.018987+0000 mon.c (mon.2) 39 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T03:22:07.111 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[19769]: audit 2024-02-13T03:22:06.019674+0000 mgr.y (mgr.24473) 35 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T03:22:07.111 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[19769]: audit 2024-02-13T03:22:06.028564+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:07.111 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[19769]: cephadm 2024-02-13T03:22:06.046971+0000 mgr.y (mgr.24473) 36 : cephadm [INF] Deploying daemon grafana.a on smithi183 2024-02-13T03:22:07.111 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[27879]: audit 2024-02-13T03:22:05.564223+0000 mgr.y (mgr.24473) 34 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:07.111 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[27879]: audit 2024-02-13T03:22:05.848324+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:07.111 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[27879]: audit 2024-02-13T03:22:06.002049+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:07.112 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[27879]: audit 2024-02-13T03:22:06.013359+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:07.112 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[27879]: audit 2024-02-13T03:22:06.018987+0000 mon.c (mon.2) 39 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T03:22:07.112 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[27879]: audit 2024-02-13T03:22:06.019674+0000 mgr.y (mgr.24473) 35 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T03:22:07.112 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[27879]: audit 2024-02-13T03:22:06.028564+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:07.112 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:06 smithi161 bash[27879]: cephadm 2024-02-13T03:22:06.046971+0000 mgr.y (mgr.24473) 36 : cephadm [INF] Deploying daemon grafana.a on smithi183 2024-02-13T03:22:07.112 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:07 smithi161 bash[52200]: level=info ts=2024-02-13T03:22:07.048Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-02-13T03:22:07.112 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:07 smithi161 bash[52200]: level=info ts=2024-02-13T03:22:07.048Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-02-13T03:22:07.112 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:07 smithi161 bash[52200]: level=info ts=2024-02-13T03:22:07.051Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.161 port=9094 2024-02-13T03:22:07.112 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:07 smithi161 bash[52200]: level=info ts=2024-02-13T03:22:07.052Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-13T03:22:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:06 smithi183 bash[20824]: audit 2024-02-13T03:22:05.564223+0000 mgr.y (mgr.24473) 34 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:06 smithi183 bash[20824]: audit 2024-02-13T03:22:05.848324+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:06 smithi183 bash[20824]: audit 2024-02-13T03:22:06.002049+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:06 smithi183 bash[20824]: audit 2024-02-13T03:22:06.013359+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:06 smithi183 bash[20824]: audit 2024-02-13T03:22:06.018987+0000 mon.c (mon.2) 39 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T03:22:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:06 smithi183 bash[20824]: audit 2024-02-13T03:22:06.019674+0000 mgr.y (mgr.24473) 35 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T03:22:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:06 smithi183 bash[20824]: audit 2024-02-13T03:22:06.028564+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:06 smithi183 bash[20824]: cephadm 2024-02-13T03:22:06.046971+0000 mgr.y (mgr.24473) 36 : cephadm [INF] Deploying daemon grafana.a on smithi183 2024-02-13T03:22:07.427 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:07 smithi161 bash[52200]: level=info ts=2024-02-13T03:22:07.108Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-13T03:22:07.427 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:07 smithi161 bash[52200]: level=info ts=2024-02-13T03:22:07.108Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-13T03:22:07.427 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:07 smithi161 bash[52200]: level=info ts=2024-02-13T03:22:07.113Z caller=main.go:518 msg=Listening address=:9093 2024-02-13T03:22:07.427 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:07 smithi161 bash[52200]: level=info ts=2024-02-13T03:22:07.113Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-02-13T03:22:08.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:08 smithi161 bash[19769]: cluster 2024-02-13T03:22:06.552800+0000 mgr.y (mgr.24473) 37 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:08 smithi161 bash[27879]: cluster 2024-02-13T03:22:06.552800+0000 mgr.y (mgr.24473) 37 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:08.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:08 smithi183 bash[20824]: cluster 2024-02-13T03:22:06.552800+0000 mgr.y (mgr.24473) 37 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:08.751 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:22:09.427 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:09 smithi161 bash[52200]: level=info ts=2024-02-13T03:22:09.053Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000633608s 2024-02-13T03:22:09.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:22:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:22:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:22:10.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:10 smithi161 bash[27879]: cluster 2024-02-13T03:22:08.553555+0000 mgr.y (mgr.24473) 38 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:10.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:10 smithi161 bash[27879]: audit 2024-02-13T03:22:09.815104+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:10.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:10 smithi161 bash[19769]: cluster 2024-02-13T03:22:08.553555+0000 mgr.y (mgr.24473) 38 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:10.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:10 smithi161 bash[19769]: audit 2024-02-13T03:22:09.815104+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:10 smithi183 bash[20824]: cluster 2024-02-13T03:22:08.553555+0000 mgr.y (mgr.24473) 38 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:10 smithi183 bash[20824]: audit 2024-02-13T03:22:09.815104+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:11.389 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:22:11] "GET /metrics HTTP/1.1" 200 172124 "" "Prometheus/2.33.4" 2024-02-13T03:22:11.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.625+0000 7fb7860ee700 1 -- 172.21.15.161:0/2446600271 >> v1:172.21.15.161:6789/0 conn(0x7fb780103320 legacy=0x7fb7801057c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:11.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.625+0000 7fb7860ee700 1 -- 172.21.15.161:0/2446600271 shutdown_connections 2024-02-13T03:22:11.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.625+0000 7fb7860ee700 1 -- 172.21.15.161:0/2446600271 >> 172.21.15.161:0/2446600271 conn(0x7fb7800f2610 msgr2=0x7fb7800f4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:11.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.625+0000 7fb7860ee700 1 -- 172.21.15.161:0/2446600271 shutdown_connections 2024-02-13T03:22:11.630 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.625+0000 7fb7860ee700 1 -- 172.21.15.161:0/2446600271 wait complete. 2024-02-13T03:22:11.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.625+0000 7fb7860ee700 1 Processor -- start 2024-02-13T03:22:11.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.625+0000 7fb7860ee700 1 -- start start 2024-02-13T03:22:11.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb7860ee700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb780117310 con 0x7fb780103320 2024-02-13T03:22:11.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb7860ee700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb780118510 con 0x7fb7800fca30 2024-02-13T03:22:11.631 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb7860ee700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb780119710 con 0x7fb7800ff810 2024-02-13T03:22:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77ffff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb780103320 0x7fb7801159d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59232/0 (socket says 172.21.15.161:59232) 2024-02-13T03:22:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77ffff700 1 -- 172.21.15.161:0/1636754826 learned_addr learned my addr 172.21.15.161:0/1636754826 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:22:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908040332 0 0) 0x7fb780119710 con 0x7fb7800ff810 2024-02-13T03:22:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb760003030 con 0x7fb7800ff810 2024-02-13T03:22:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649590819 0 0) 0x7fb780117310 con 0x7fb780103320 2024-02-13T03:22:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb780119710 con 0x7fb780103320 2024-02-13T03:22:11.632 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635551629 0 0) 0x7fb780118510 con 0x7fb7800fca30 2024-02-13T03:22:11.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb780117310 con 0x7fb7800fca30 2024-02-13T03:22:11.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 834570699 0 0) 0x7fb760003030 con 0x7fb7800ff810 2024-02-13T03:22:11.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb780118510 con 0x7fb7800ff810 2024-02-13T03:22:11.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb7680018e0 con 0x7fb7800ff810 2024-02-13T03:22:11.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1498334529 0 0) 0x7fb780119710 con 0x7fb780103320 2024-02-13T03:22:11.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb760003030 con 0x7fb780103320 2024-02-13T03:22:11.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb778001930 con 0x7fb780103320 2024-02-13T03:22:11.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 202855529 0 0) 0x7fb780117310 con 0x7fb7800fca30 2024-02-13T03:22:11.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb780119710 con 0x7fb7800fca30 2024-02-13T03:22:11.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb770001840 con 0x7fb7800fca30 2024-02-13T03:22:11.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3113632716 0 0) 0x7fb780118510 con 0x7fb7800ff810 2024-02-13T03:22:11.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 >> v1:172.21.15.183:6789/0 conn(0x7fb7800fca30 legacy=0x7fb780060b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:11.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 >> v1:172.21.15.161:6789/0 conn(0x7fb780103320 legacy=0x7fb7801159d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:11.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78011a910 con 0x7fb7800ff810 2024-02-13T03:22:11.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb7860ee700 1 -- 172.21.15.161:0/1636754826 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb780119900 con 0x7fb7800ff810 2024-02-13T03:22:11.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb7860ee700 1 -- 172.21.15.161:0/1636754826 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb780119df0 con 0x7fb7800ff810 2024-02-13T03:22:11.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fb768002250 con 0x7fb7800ff810 2024-02-13T03:22:11.635 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.629+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb768003190 con 0x7fb7800ff810 2024-02-13T03:22:11.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.633+0000 7fb7860ee700 1 -- 172.21.15.161:0/1636754826 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb764004f20 con 0x7fb7800ff810 2024-02-13T03:22:11.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.633+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fb768001db0 con 0x7fb7800ff810 2024-02-13T03:22:11.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.633+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6056+0+0 (unknown 2145861735 0 0) 0x7fb768075b30 con 0x7fb7800ff810 2024-02-13T03:22:11.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.641+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fb768049760 con 0x7fb7800ff810 2024-02-13T03:22:11.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.821+0000 7fb7860ee700 1 -- 172.21.15.161:0/1636754826 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb764005750 con 0x7fb7800ff810 2024-02-13T03:22:11.825 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.821+0000 7fb77cff9700 1 -- 172.21.15.161:0/1636754826 <== mon.2 v1:172.21.15.161:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v61) v1 ==== 110+0+10 (unknown 1192997216 0 2170607528) 0x7fb768047b70 con 0x7fb7800ff810 2024-02-13T03:22:11.826 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 1 2024-02-13T03:22:11.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.825+0000 7fb7860ee700 1 -- 172.21.15.161:0/1636754826 >> v1:172.21.15.161:6800/4114337139 conn(0x7fb760062620 legacy=0x7fb760064ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:11.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.825+0000 7fb7860ee700 1 -- 172.21.15.161:0/1636754826 >> v1:172.21.15.161:6790/0 conn(0x7fb7800ff810 legacy=0x7fb780061210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:11.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.825+0000 7fb7860ee700 1 -- 172.21.15.161:0/1636754826 shutdown_connections 2024-02-13T03:22:11.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.825+0000 7fb7860ee700 1 -- 172.21.15.161:0/1636754826 >> 172.21.15.161:0/1636754826 conn(0x7fb7800f2610 msgr2=0x7fb7800f3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:11.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.825+0000 7fb7860ee700 1 -- 172.21.15.161:0/1636754826 shutdown_connections 2024-02-13T03:22:11.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:11.825+0000 7fb7860ee700 1 -- 172.21.15.161:0/1636754826 wait complete. 2024-02-13T03:22:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:12 smithi161 bash[27879]: cluster 2024-02-13T03:22:10.554977+0000 mgr.y (mgr.24473) 39 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:12 smithi161 bash[27879]: audit 2024-02-13T03:22:11.823605+0000 mon.c (mon.2) 40 : audit [DBG] from='client.? v1:172.21.15.161:0/1636754826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-13T03:22:12.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:12 smithi161 bash[19769]: cluster 2024-02-13T03:22:10.554977+0000 mgr.y (mgr.24473) 39 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:12.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:12 smithi161 bash[19769]: audit 2024-02-13T03:22:11.823605+0000 mon.c (mon.2) 40 : audit [DBG] from='client.? v1:172.21.15.161:0/1636754826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-13T03:22:12.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:12 smithi183 bash[20824]: cluster 2024-02-13T03:22:10.554977+0000 mgr.y (mgr.24473) 39 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:12.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:12 smithi183 bash[20824]: audit 2024-02-13T03:22:11.823605+0000 mon.c (mon.2) 40 : audit [DBG] from='client.? v1:172.21.15.161:0/1636754826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-13T03:22:12.497 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd pool get iscsi pg_num 2024-02-13T03:22:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:13 smithi161 bash[27879]: cluster 2024-02-13T03:22:12.555691+0000 mgr.y (mgr.24473) 40 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:13.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:13 smithi161 bash[19769]: cluster 2024-02-13T03:22:12.555691+0000 mgr.y (mgr.24473) 40 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:13 smithi183 bash[20824]: cluster 2024-02-13T03:22:12.555691+0000 mgr.y (mgr.24473) 40 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:15.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:15 smithi161 bash[27879]: cluster 2024-02-13T03:22:14.556480+0000 mgr.y (mgr.24473) 41 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:15.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:15 smithi161 bash[19769]: cluster 2024-02-13T03:22:14.556480+0000 mgr.y (mgr.24473) 41 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:15.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:15 smithi183 bash[20824]: cluster 2024-02-13T03:22:14.556480+0000 mgr.y (mgr.24473) 41 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:15.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:22:15 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:22:16.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:16 smithi161 bash[27879]: audit 2024-02-13T03:22:15.571638+0000 mgr.y (mgr.24473) 42 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:16 smithi161 bash[19769]: audit 2024-02-13T03:22:15.571638+0000 mgr.y (mgr.24473) 42 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:16.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:16 smithi183 bash[20824]: audit 2024-02-13T03:22:15.571638+0000 mgr.y (mgr.24473) 42 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:17.338 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:22:17.427 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:17 smithi161 bash[52200]: level=info ts=2024-02-13T03:22:17.054Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.001898471s 2024-02-13T03:22:17.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:17 smithi161 bash[27879]: cluster 2024-02-13T03:22:16.558002+0000 mgr.y (mgr.24473) 43 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:17.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:17 smithi161 bash[19769]: cluster 2024-02-13T03:22:16.558002+0000 mgr.y (mgr.24473) 43 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:17.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:17 smithi183 bash[20824]: cluster 2024-02-13T03:22:16.558002+0000 mgr.y (mgr.24473) 43 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:19.632 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:22:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:22:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:22:19.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:19 smithi161 bash[19769]: cluster 2024-02-13T03:22:18.558702+0000 mgr.y (mgr.24473) 44 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:19 smithi161 bash[27879]: cluster 2024-02-13T03:22:18.558702+0000 mgr.y (mgr.24473) 44 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:19.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:19 smithi183 bash[20824]: cluster 2024-02-13T03:22:18.558702+0000 mgr.y (mgr.24473) 44 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f452f144700 1 -- 172.21.15.161:0/1848963894 >> v1:172.21.15.161:6790/0 conn(0x7f4528103330 legacy=0x7f45281057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f452f144700 1 -- 172.21.15.161:0/1848963894 shutdown_connections 2024-02-13T03:22:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f452f144700 1 -- 172.21.15.161:0/1848963894 >> 172.21.15.161:0/1848963894 conn(0x7f45280f2620 msgr2=0x7f45280f4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f452f144700 1 -- 172.21.15.161:0/1848963894 shutdown_connections 2024-02-13T03:22:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f452f144700 1 -- 172.21.15.161:0/1848963894 wait complete. 2024-02-13T03:22:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f452f144700 1 Processor -- start 2024-02-13T03:22:20.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f452f144700 1 -- start start 2024-02-13T03:22:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f452f144700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4528116250 con 0x7f45280ff7f0 2024-02-13T03:22:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f452f144700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4528117430 con 0x7f4528103330 2024-02-13T03:22:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f452f144700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4528118630 con 0x7f45280fca50 2024-02-13T03:22:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f452cee0700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f45280fca50 0x7f45280f8830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:34794/0 (socket says 172.21.15.161:34794) 2024-02-13T03:22:20.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f452cee0700 1 -- 172.21.15.161:0/685522595 learned_addr learned my addr 172.21.15.161:0/685522595 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:22:20.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708732443 0 0) 0x7f4528118630 con 0x7f45280fca50 2024-02-13T03:22:20.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4514003110 con 0x7f45280fca50 2024-02-13T03:22:20.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903992910 0 0) 0x7f4528116250 con 0x7f45280ff7f0 2024-02-13T03:22:20.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4528118630 con 0x7f45280ff7f0 2024-02-13T03:22:20.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1170762523 0 0) 0x7f4528117430 con 0x7f4528103330 2024-02-13T03:22:20.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.169+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4528116250 con 0x7f4528103330 2024-02-13T03:22:20.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1636452666 0 0) 0x7f4514003110 con 0x7f45280fca50 2024-02-13T03:22:20.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4528117430 con 0x7f45280fca50 2024-02-13T03:22:20.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f4518001df0 con 0x7f45280fca50 2024-02-13T03:22:20.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 487252676 0 0) 0x7f4528118630 con 0x7f45280ff7f0 2024-02-13T03:22:20.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4514003110 con 0x7f45280ff7f0 2024-02-13T03:22:20.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f4510001880 con 0x7f45280ff7f0 2024-02-13T03:22:20.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 308280903 0 0) 0x7f4528116250 con 0x7f4528103330 2024-02-13T03:22:20.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4528118630 con 0x7f4528103330 2024-02-13T03:22:20.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f4520001360 con 0x7f4528103330 2024-02-13T03:22:20.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3310479310 0 0) 0x7f4514003110 con 0x7f45280ff7f0 2024-02-13T03:22:20.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 >> v1:172.21.15.161:6790/0 conn(0x7f45280fca50 legacy=0x7f45280f8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:20.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 >> v1:172.21.15.183:6789/0 conn(0x7f4528103330 legacy=0x7f45281149b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:20.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528119830 con 0x7f45280ff7f0 2024-02-13T03:22:20.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f452f144700 1 -- 172.21.15.161:0/685522595 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4528118820 con 0x7f45280ff7f0 2024-02-13T03:22:20.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f452f144700 1 -- 172.21.15.161:0/685522595 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4528118d10 con 0x7f45280ff7f0 2024-02-13T03:22:20.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f4510002d80 con 0x7f45280ff7f0 2024-02-13T03:22:20.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f45100031c0 con 0x7f45280ff7f0 2024-02-13T03:22:20.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f452f144700 1 -- 172.21.15.161:0/685522595 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4508004f20 con 0x7f45280ff7f0 2024-02-13T03:22:20.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.173+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f45100160d0 con 0x7f45280ff7f0 2024-02-13T03:22:20.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.177+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6056+0+0 (unknown 2145861735 0 0) 0x7f4510075b00 con 0x7f45280ff7f0 2024-02-13T03:22:20.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.185+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f4510049730 con 0x7f45280ff7f0 2024-02-13T03:22:20.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.365+0000 7f452f144700 1 -- 172.21.15.161:0/685522595 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"} v 0) v1 -- 0x7f4508005750 con 0x7f45280ff7f0 2024-02-13T03:22:20.369 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.365+0000 7f4525ffb700 1 -- 172.21.15.161:0/685522595 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]=0 v61) v1 ==== 94+0+10 (unknown 2183422609 0 2787501894) 0x7f4510047b40 con 0x7f45280ff7f0 2024-02-13T03:22:20.369 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 3 2024-02-13T03:22:20.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.369+0000 7f452f144700 1 -- 172.21.15.161:0/685522595 >> v1:172.21.15.161:6800/4114337139 conn(0x7f45140624e0 legacy=0x7f45140649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:20.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.369+0000 7f452f144700 1 -- 172.21.15.161:0/685522595 >> v1:172.21.15.161:6789/0 conn(0x7f45280ff7f0 legacy=0x7f45280f8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:20.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.369+0000 7f452f144700 1 -- 172.21.15.161:0/685522595 shutdown_connections 2024-02-13T03:22:20.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.369+0000 7f452f144700 1 -- 172.21.15.161:0/685522595 >> 172.21.15.161:0/685522595 conn(0x7f45280f2620 msgr2=0x7f45281067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:20.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.369+0000 7f452f144700 1 -- 172.21.15.161:0/685522595 shutdown_connections 2024-02-13T03:22:20.372 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:20.369+0000 7f452f144700 1 -- 172.21.15.161:0/685522595 wait complete. 2024-02-13T03:22:20.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:20 smithi161 bash[19769]: audit 2024-02-13T03:22:20.367479+0000 mon.a (mon.0) 646 : audit [DBG] from='client.? v1:172.21.15.161:0/685522595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-13T03:22:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:20 smithi161 bash[27879]: audit 2024-02-13T03:22:20.367479+0000 mon.a (mon.0) 646 : audit [DBG] from='client.? v1:172.21.15.161:0/685522595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-13T03:22:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:20 smithi183 bash[20824]: audit 2024-02-13T03:22:20.367479+0000 mon.a (mon.0) 646 : audit [DBG] from='client.? v1:172.21.15.161:0/685522595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-13T03:22:20.996 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd pool get .rgw.root pg_num 2024-02-13T03:22:21.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:22:21] "GET /metrics HTTP/1.1" 200 172125 "" "Prometheus/2.33.4" 2024-02-13T03:22:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:21 smithi161 bash[27879]: cluster 2024-02-13T03:22:20.560482+0000 mgr.y (mgr.24473) 45 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:21 smithi161 bash[19769]: cluster 2024-02-13T03:22:20.560482+0000 mgr.y (mgr.24473) 45 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:21.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:21 smithi183 bash[20824]: cluster 2024-02-13T03:22:20.560482+0000 mgr.y (mgr.24473) 45 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:23.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:23 smithi161 bash[19769]: cluster 2024-02-13T03:22:22.561276+0000 mgr.y (mgr.24473) 46 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:23 smithi161 bash[27879]: cluster 2024-02-13T03:22:22.561276+0000 mgr.y (mgr.24473) 46 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:23 smithi183 bash[20824]: cluster 2024-02-13T03:22:22.561276+0000 mgr.y (mgr.24473) 46 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:25.487 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:25 smithi183 systemd[1]: Started Ceph grafana.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:22:25.843 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:22:25.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:22:25 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:22:26.267 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[20824]: cluster 2024-02-13T03:22:24.562095+0000 mgr.y (mgr.24473) 47 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:26.267 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[20824]: audit 2024-02-13T03:22:25.175665+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:26.268 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[20824]: audit 2024-02-13T03:22:25.179673+0000 mon.c (mon.2) 41 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:22:26.268 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="The state of unified alerting is still not defined. The decision will be made during as we run the database migrations" logger=settings 2024-02-13T03:22:26.268 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=warn msg="falling back to legacy setting of 'min_interval_seconds'; please use the configuration option in the `unified_alerting` section if Grafana 8 alerts are enabled." logger=settings 2024-02-13T03:22:26.268 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini 2024-02-13T03:22:26.268 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini 2024-02-13T03:22:26.268 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-13T03:22:26.268 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-13T03:22:26.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-13T03:22:26.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-13T03:22:26.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Path Home" logger=settings path=/usr/share/grafana 2024-02-13T03:22:26.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Path Data" logger=settings path=/var/lib/grafana 2024-02-13T03:22:26.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Path Logs" logger=settings path=/var/log/grafana 2024-02-13T03:22:26.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Path Plugins" logger=settings path=/var/lib/grafana/plugins 2024-02-13T03:22:26.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Path Provisioning" logger=settings path=/etc/grafana/provisioning 2024-02-13T03:22:26.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="App mode production" logger=settings 2024-02-13T03:22:26.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3 2024-02-13T03:22:26.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=warn msg="SQLite database file has broader permissions than it should" logger=sqlstore path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-13T03:22:26.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Starting DB migrations" logger=migrator 2024-02-13T03:22:26.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="create migration_log table" 2024-02-13T03:22:26.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="create user table" 2024-02-13T03:22:26.512 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:26 smithi161 bash[27879]: cluster 2024-02-13T03:22:24.562095+0000 mgr.y (mgr.24473) 47 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:26.512 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:26 smithi161 bash[27879]: audit 2024-02-13T03:22:25.175665+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:26.512 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:26 smithi161 bash[27879]: audit 2024-02-13T03:22:25.179673+0000 mon.c (mon.2) 41 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:22:26.512 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:26 smithi161 bash[19769]: cluster 2024-02-13T03:22:24.562095+0000 mgr.y (mgr.24473) 47 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:26.512 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:26 smithi161 bash[19769]: audit 2024-02-13T03:22:25.175665+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:26.512 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:26 smithi161 bash[19769]: audit 2024-02-13T03:22:25.179673+0000 mon.c (mon.2) 41 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:22:26.528 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.login" 2024-02-13T03:22:26.529 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.email" 2024-02-13T03:22:26.529 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_login - v1" 2024-02-13T03:22:26.529 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_email - v1" 2024-02-13T03:22:26.529 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table user to user_v1 - v1" 2024-02-13T03:22:26.529 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="create user table v2" 2024-02-13T03:22:26.529 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_login - v2" 2024-02-13T03:22:26.529 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_email - v2" 2024-02-13T03:22:26.529 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-02-13T03:22:26.781 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table user_v1" 2024-02-13T03:22:26.781 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="Add column help_flags1 to user table" 2024-02-13T03:22:26.781 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="Update user table charset" 2024-02-13T03:22:26.781 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="Add last_seen_at column to user" 2024-02-13T03:22:26.781 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="Add missing user data" 2024-02-13T03:22:26.781 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_disabled column to user" 2024-02-13T03:22:26.781 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="Add index user.login/user.email" 2024-02-13T03:22:27.042 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_service_account column to user" 2024-02-13T03:22:27.042 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="create temp user table v1-7" 2024-02-13T03:22:27.042 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v1-7" 2024-02-13T03:22:27.042 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v1-7" 2024-02-13T03:22:27.042 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v1-7" 2024-02-13T03:22:27.042 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v1-7" 2024-02-13T03:22:27.042 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:26 smithi183 bash[46554]: t=2024-02-13T03:22:26+0000 lvl=info msg="Executing migration" logger=migrator id="Update temp_user table charset" 2024-02-13T03:22:27.042 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_email - v1" 2024-02-13T03:22:27.311 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[20824]: audit 2024-02-13T03:22:25.579447+0000 mgr.y (mgr.24473) 48 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:27.312 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_org_id - v1" 2024-02-13T03:22:27.312 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_code - v1" 2024-02-13T03:22:27.312 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_status - v1" 2024-02-13T03:22:27.312 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-13T03:22:27.312 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create temp_user v2" 2024-02-13T03:22:27.312 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v2" 2024-02-13T03:22:27.312 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v2" 2024-02-13T03:22:27.312 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v2" 2024-02-13T03:22:27.313 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v2" 2024-02-13T03:22:27.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:27 smithi161 bash[19769]: audit 2024-02-13T03:22:25.579447+0000 mgr.y (mgr.24473) 48 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:27 smithi161 bash[27879]: audit 2024-02-13T03:22:25.579447+0000 mgr.y (mgr.24473) 48 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:27.564 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="copy temp_user v1 to v2" 2024-02-13T03:22:27.564 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="drop temp_user_tmp_qwerty" 2024-02-13T03:22:27.564 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="Set created for temp users that will otherwise prematurely expire" 2024-02-13T03:22:27.564 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create star table" 2024-02-13T03:22:27.565 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index star.user_id_dashboard_id" 2024-02-13T03:22:27.565 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create org table v1" 2024-02-13T03:22:27.565 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_name - v1" 2024-02-13T03:22:27.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create org_user table v1" 2024-02-13T03:22:27.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_org_id - v1" 2024-02-13T03:22:27.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_user_org_id_user_id - v1" 2024-02-13T03:22:27.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_user_id - v1" 2024-02-13T03:22:27.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="Update org table charset" 2024-02-13T03:22:27.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="Update org_user table charset" 2024-02-13T03:22:27.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate all Read Only Viewers to Viewers" 2024-02-13T03:22:27.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard table" 2024-02-13T03:22:28.095 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard.account_id" 2024-02-13T03:22:28.095 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_account_id_slug" 2024-02-13T03:22:28.095 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_tag table" 2024-02-13T03:22:28.095 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_tag.dasboard_id_term" 2024-02-13T03:22:28.095 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-13T03:22:28.095 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard to dashboard_v1 - v1" 2024-02-13T03:22:28.095 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:27 smithi183 bash[46554]: t=2024-02-13T03:22:27+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard v2" 2024-02-13T03:22:28.095 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_org_id - v2" 2024-02-13T03:22:28.095 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_org_id_slug - v2" 2024-02-13T03:22:28.095 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard v1 to v2" 2024-02-13T03:22:28.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[20824]: cluster 2024-02-13T03:22:26.563567+0000 mgr.y (mgr.24473) 49 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:28.364 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_v1" 2024-02-13T03:22:28.365 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard.data to mediumtext v1" 2024-02-13T03:22:28.365 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column updated_by in dashboard - v2" 2024-02-13T03:22:28.365 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column created_by in dashboard - v2" 2024-02-13T03:22:28.365 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column gnetId in dashboard" 2024-02-13T03:22:28.365 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for gnetId in dashboard" 2024-02-13T03:22:28.365 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_id in dashboard" 2024-02-13T03:22:28.365 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for plugin_id in dashboard" 2024-02-13T03:22:28.365 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_id in dashboard_tag" 2024-02-13T03:22:28.447 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:28 smithi161 bash[19769]: cluster 2024-02-13T03:22:26.563567+0000 mgr.y (mgr.24473) 49 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:28.448 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:28 smithi161 bash[27879]: cluster 2024-02-13T03:22:26.563567+0000 mgr.y (mgr.24473) 49 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:28.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb86bef1700 1 -- 172.21.15.161:0/312436296 >> v1:172.21.15.161:6790/0 conn(0x7fb864104440 legacy=0x7fb8641068e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:28.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb86bef1700 1 -- 172.21.15.161:0/312436296 shutdown_connections 2024-02-13T03:22:28.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb86bef1700 1 -- 172.21.15.161:0/312436296 >> 172.21.15.161:0/312436296 conn(0x7fb8640f2610 msgr2=0x7fb8640f4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:28.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb86bef1700 1 -- 172.21.15.161:0/312436296 shutdown_connections 2024-02-13T03:22:28.600 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb86bef1700 1 -- 172.21.15.161:0/312436296 wait complete. 2024-02-13T03:22:28.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb86bef1700 1 Processor -- start 2024-02-13T03:22:28.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb86bef1700 1 -- start start 2024-02-13T03:22:28.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb86bef1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb864102db0 con 0x7fb8640f8b40 2024-02-13T03:22:28.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb86bef1700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb864102f80 con 0x7fb864104440 2024-02-13T03:22:28.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb86bef1700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb86411bcf0 con 0x7fb8640f9630 2024-02-13T03:22:28.601 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb86948c700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7fb8640f9630 0x7fb864114e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:60580/0 (socket says 172.21.15.161:60580) 2024-02-13T03:22:28.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb86948c700 1 -- 172.21.15.161:0/749251568 learned_addr learned my addr 172.21.15.161:0/749251568 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:22:28.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb869c8d700 1 --1- 172.21.15.161:0/749251568 >> v1:172.21.15.161:6789/0 conn(0x7fb8640f8b40 0x7fb864102610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:37060/0 (socket says 172.21.15.161:37060) 2024-02-13T03:22:28.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3168018887 0 0) 0x7fb864102db0 con 0x7fb8640f8b40 2024-02-13T03:22:28.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb84c0033f0 con 0x7fb8640f8b40 2024-02-13T03:22:28.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634252613 0 0) 0x7fb86411bcf0 con 0x7fb8640f9630 2024-02-13T03:22:28.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb864102db0 con 0x7fb8640f9630 2024-02-13T03:22:28.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937855728 0 0) 0x7fb864102f80 con 0x7fb864104440 2024-02-13T03:22:28.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb86411bcf0 con 0x7fb864104440 2024-02-13T03:22:28.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1794347009 0 0) 0x7fb84c0033f0 con 0x7fb8640f8b40 2024-02-13T03:22:28.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb864102f80 con 0x7fb8640f8b40 2024-02-13T03:22:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb85c002c20 con 0x7fb8640f8b40 2024-02-13T03:22:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 791789840 0 0) 0x7fb864102db0 con 0x7fb8640f9630 2024-02-13T03:22:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb84c0033f0 con 0x7fb8640f9630 2024-02-13T03:22:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3949483934 0 0) 0x7fb86411bcf0 con 0x7fb864104440 2024-02-13T03:22:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb864102db0 con 0x7fb864104440 2024-02-13T03:22:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb858002950 con 0x7fb8640f9630 2024-02-13T03:22:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb854001ec0 con 0x7fb864104440 2024-02-13T03:22:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 12908300 0 0) 0x7fb864102f80 con 0x7fb8640f8b40 2024-02-13T03:22:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.597+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 >> v1:172.21.15.161:6790/0 conn(0x7fb8640f9630 legacy=0x7fb864114e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.601+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 >> v1:172.21.15.183:6789/0 conn(0x7fb864104440 legacy=0x7fb8641185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.601+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86411ce90 con 0x7fb8640f8b40 2024-02-13T03:22:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.601+0000 7fb86bef1700 1 -- 172.21.15.161:0/749251568 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86411bee0 con 0x7fb8640f8b40 2024-02-13T03:22:28.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.601+0000 7fb86bef1700 1 -- 172.21.15.161:0/749251568 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86411c350 con 0x7fb8640f8b40 2024-02-13T03:22:28.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.601+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fb85c001b40 con 0x7fb8640f8b40 2024-02-13T03:22:28.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.601+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb85c005e80 con 0x7fb8640f8b40 2024-02-13T03:22:28.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.601+0000 7fb86bef1700 1 -- 172.21.15.161:0/749251568 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb850004f20 con 0x7fb8640f8b40 2024-02-13T03:22:28.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.601+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fb85c018db0 con 0x7fb8640f8b40 2024-02-13T03:22:28.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.601+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6056+0+0 (unknown 2145861735 0 0) 0x7fb85c078800 con 0x7fb8640f8b40 2024-02-13T03:22:28.613 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.609+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fb85c04c430 con 0x7fb8640f8b40 2024-02-13T03:22:28.625 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard table charset" 2024-02-13T03:22:28.625 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_tag table charset" 2024-02-13T03:22:28.625 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column folder_id in dashboard" 2024-02-13T03:22:28.625 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column isFolder in dashboard" 2024-02-13T03:22:28.625 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column has_acl in dashboard" 2024-02-13T03:22:28.625 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in dashboard" 2024-02-13T03:22:28.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.801+0000 7fb86bef1700 1 -- 172.21.15.161:0/749251568 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"} v 0) v1 -- 0x7fb850005750 con 0x7fb8640f8b40 2024-02-13T03:22:28.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.801+0000 7fb862ffd700 1 -- 172.21.15.161:0/749251568 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]=0 v61) v1 ==== 98+0+11 (unknown 611845801 0 3519293104) 0x7fb85c04a840 con 0x7fb8640f8b40 2024-02-13T03:22:28.805 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-02-13T03:22:28.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.801+0000 7fb86bef1700 1 -- 172.21.15.161:0/749251568 >> v1:172.21.15.161:6800/4114337139 conn(0x7fb84c062340 legacy=0x7fb84c064800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:28.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.801+0000 7fb86bef1700 1 -- 172.21.15.161:0/749251568 >> v1:172.21.15.161:6789/0 conn(0x7fb8640f8b40 legacy=0x7fb864102610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:28.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.805+0000 7fb86bef1700 1 -- 172.21.15.161:0/749251568 shutdown_connections 2024-02-13T03:22:28.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.805+0000 7fb86bef1700 1 -- 172.21.15.161:0/749251568 >> 172.21.15.161:0/749251568 conn(0x7fb8640f2610 msgr2=0x7fb8641078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:28.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.805+0000 7fb86bef1700 1 -- 172.21.15.161:0/749251568 shutdown_connections 2024-02-13T03:22:28.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:28.805+0000 7fb86bef1700 1 -- 172.21.15.161:0/749251568 wait complete. 2024-02-13T03:22:28.886 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in dashboard" 2024-02-13T03:22:28.886 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index dashboard_org_id_uid" 2024-02-13T03:22:28.886 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_slug" 2024-02-13T03:22:28.886 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard title length" 2024-02-13T03:22:28.886 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-13T03:22:28.886 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning" 2024-02-13T03:22:28.886 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-13T03:22:28.886 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning v2" 2024-02-13T03:22:28.887 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-13T03:22:28.887 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-13T03:22:29.138 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard_provisioning v1 to v2" 2024-02-13T03:22:29.138 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop dashboard_provisioning_tmp_qwerty" 2024-02-13T03:22:29.138 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:28 smithi183 bash[46554]: t=2024-02-13T03:22:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add check_sum column" 2024-02-13T03:22:29.139 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_title" 2024-02-13T03:22:29.139 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="delete tags for deleted dashboards" 2024-02-13T03:22:29.139 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="delete stars for deleted dashboards" 2024-02-13T03:22:29.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[20824]: audit 2024-02-13T03:22:28.802976+0000 mon.a (mon.0) 648 : audit [DBG] from='client.? v1:172.21.15.161:0/749251568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-13T03:22:29.400 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:22:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:22:29.400 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_is_folder" 2024-02-13T03:22:29.400 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table" 2024-02-13T03:22:29.400 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="add index data_source.account_id" 2024-02-13T03:22:29.400 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index data_source.account_id_name" 2024-02-13T03:22:29.400 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_data_source_account_id - v1" 2024-02-13T03:22:29.400 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_data_source_account_id_name - v1" 2024-02-13T03:22:29.401 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table data_source to data_source_v1 - v1" 2024-02-13T03:22:29.401 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table v2" 2024-02-13T03:22:29.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:29 smithi161 bash[27879]: audit 2024-02-13T03:22:28.802976+0000 mon.a (mon.0) 648 : audit [DBG] from='client.? v1:172.21.15.161:0/749251568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-13T03:22:29.472 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:29 smithi161 bash[19769]: audit 2024-02-13T03:22:28.802976+0000 mon.a (mon.0) 648 : audit [DBG] from='client.? v1:172.21.15.161:0/749251568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-13T03:22:29.473 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd pool get default.rgw.log pg_num 2024-02-13T03:22:29.668 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_data_source_org_id - v2" 2024-02-13T03:22:29.668 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_data_source_org_id_name - v2" 2024-02-13T03:22:29.669 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-02-13T03:22:29.669 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table data_source_v1 #2" 2024-02-13T03:22:29.669 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add column with_credentials" 2024-02-13T03:22:29.669 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add secure json data column" 2024-02-13T03:22:29.669 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Update data_source table charset" 2024-02-13T03:22:29.669 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Update initial version to 1" 2024-02-13T03:22:29.929 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add read_only data column" 2024-02-13T03:22:29.930 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate logging ds to loki ds" 2024-02-13T03:22:29.930 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Update json_data with nulls" 2024-02-13T03:22:29.930 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add uid column" 2024-02-13T03:22:29.930 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid value" 2024-02-13T03:22:29.930 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index datasource_org_id_uid" 2024-02-13T03:22:29.930 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index datasource_org_id_is_default" 2024-02-13T03:22:29.930 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table" 2024-02-13T03:22:30.182 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id" 2024-02-13T03:22:30.182 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.key" 2024-02-13T03:22:30.183 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:29 smithi183 bash[46554]: t=2024-02-13T03:22:29+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id_name" 2024-02-13T03:22:30.183 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_api_key_account_id - v1" 2024-02-13T03:22:30.183 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_key - v1" 2024-02-13T03:22:30.183 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_account_id_name - v1" 2024-02-13T03:22:30.183 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table api_key to api_key_v1 - v1" 2024-02-13T03:22:30.443 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[20824]: cluster 2024-02-13T03:22:28.564272+0000 mgr.y (mgr.24473) 50 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:30.444 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[20824]: audit 2024-02-13T03:22:29.839336+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:30.444 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table v2" 2024-02-13T03:22:30.444 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_api_key_org_id - v2" 2024-02-13T03:22:30.444 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_key - v2" 2024-02-13T03:22:30.444 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_org_id_name - v2" 2024-02-13T03:22:30.444 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="copy api_key v1 to v2" 2024-02-13T03:22:30.444 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table api_key_v1" 2024-02-13T03:22:30.444 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update api_key table charset" 2024-02-13T03:22:30.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:30 smithi161 bash[19769]: cluster 2024-02-13T03:22:28.564272+0000 mgr.y (mgr.24473) 50 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:30.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:30 smithi161 bash[19769]: audit 2024-02-13T03:22:29.839336+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:30 smithi161 bash[27879]: cluster 2024-02-13T03:22:28.564272+0000 mgr.y (mgr.24473) 50 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:30 smithi161 bash[27879]: audit 2024-02-13T03:22:29.839336+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:30.721 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add expires to api_key table" 2024-02-13T03:22:30.722 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add service account foreign key" 2024-02-13T03:22:30.722 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v4" 2024-02-13T03:22:30.722 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_snapshot_v4 #1" 2024-02-13T03:22:30.722 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v5 #2" 2024-02-13T03:22:30.722 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_key - v5" 2024-02-13T03:22:30.722 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-13T03:22:30.722 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-13T03:22:30.986 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_snapshot to mediumtext v2" 2024-02-13T03:22:30.987 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_snapshot table charset" 2024-02-13T03:22:30.987 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external_delete_url to dashboard_snapshots table" 2024-02-13T03:22:30.987 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add encrypted dashboard json column" 2024-02-13T03:22:30.987 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-13T03:22:30.987 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="create quota table v1" 2024-02-13T03:22:30.987 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-13T03:22:30.987 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:30 smithi183 bash[46554]: t=2024-02-13T03:22:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update quota table charset" 2024-02-13T03:22:31.269 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="create plugin_setting table" 2024-02-13T03:22:31.270 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-13T03:22:31.270 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_version to plugin_settings" 2024-02-13T03:22:31.270 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Update plugin_setting table charset" 2024-02-13T03:22:31.270 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="create session table" 2024-02-13T03:22:31.270 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist table" 2024-02-13T03:22:31.270 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist_item table" 2024-02-13T03:22:31.270 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist table v2" 2024-02-13T03:22:31.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:22:31] "GET /metrics HTTP/1.1" 200 172125 "" "Prometheus/2.33.4" 2024-02-13T03:22:31.522 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist item table v2" 2024-02-13T03:22:31.522 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist table charset" 2024-02-13T03:22:31.522 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist_item table charset" 2024-02-13T03:22:31.522 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v2" 2024-02-13T03:22:31.522 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v3" 2024-02-13T03:22:31.522 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="create preferences table v3" 2024-02-13T03:22:31.522 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Update preferences table charset" 2024-02-13T03:22:31.793 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add column team_id in preferences" 2024-02-13T03:22:31.793 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Update team_id column values in preferences" 2024-02-13T03:22:31.794 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add column week_start in preferences" 2024-02-13T03:22:31.794 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="create alert table v1" 2024-02-13T03:22:31.794 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert org_id & id " 2024-02-13T03:22:31.794 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert state" 2024-02-13T03:22:31.794 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert dashboard_id" 2024-02-13T03:22:31.794 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v1" 2024-02-13T03:22:31.794 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-13T03:22:32.062 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[20824]: cluster 2024-02-13T03:22:30.565787+0000 mgr.y (mgr.24473) 51 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:32.062 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[20824]: audit 2024-02-13T03:22:30.967231+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:32.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[20824]: audit 2024-02-13T03:22:31.366573+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:32.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[20824]: audit 2024-02-13T03:22:31.369183+0000 mon.c (mon.2) 42 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:22:32.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[20824]: audit 2024-02-13T03:22:31.370787+0000 mon.c (mon.2) 43 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:22:32.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[20824]: audit 2024-02-13T03:22:31.378648+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:32.063 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-13T03:22:32.063 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-13T03:22:32.063 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v2" 2024-02-13T03:22:32.063 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-13T03:22:32.063 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="copy alert_rule_tag v1 to v2" 2024-02-13T03:22:32.063 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="drop table alert_rule_tag_v1" 2024-02-13T03:22:32.064 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:31 smithi183 bash[46554]: t=2024-02-13T03:22:31+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification table v1" 2024-02-13T03:22:32.064 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add column is_default" 2024-02-13T03:22:32.323 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add column frequency" 2024-02-13T03:22:32.323 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add column send_reminder" 2024-02-13T03:22:32.324 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add column disable_resolve_message" 2024-02-13T03:22:32.324 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification org_id & name" 2024-02-13T03:22:32.324 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert table charset" 2024-02-13T03:22:32.324 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert_notification table charset" 2024-02-13T03:22:32.324 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="create notification_journal table v1" 2024-02-13T03:22:32.324 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-13T03:22:32.324 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_notification_journal" 2024-02-13T03:22:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:31 smithi161 bash[27879]: cluster 2024-02-13T03:22:30.565787+0000 mgr.y (mgr.24473) 51 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:31 smithi161 bash[27879]: audit 2024-02-13T03:22:30.967231+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:31 smithi161 bash[27879]: audit 2024-02-13T03:22:31.366573+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:31 smithi161 bash[27879]: audit 2024-02-13T03:22:31.369183+0000 mon.c (mon.2) 42 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:22:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:31 smithi161 bash[27879]: audit 2024-02-13T03:22:31.370787+0000 mon.c (mon.2) 43 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:22:32.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:31 smithi161 bash[27879]: audit 2024-02-13T03:22:31.378648+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:32.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:31 smithi161 bash[19769]: cluster 2024-02-13T03:22:30.565787+0000 mgr.y (mgr.24473) 51 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:32.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:31 smithi161 bash[19769]: audit 2024-02-13T03:22:30.967231+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:32.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:31 smithi161 bash[19769]: audit 2024-02-13T03:22:31.366573+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:32.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:31 smithi161 bash[19769]: audit 2024-02-13T03:22:31.369183+0000 mon.c (mon.2) 42 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:22:32.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:31 smithi161 bash[19769]: audit 2024-02-13T03:22:31.370787+0000 mon.c (mon.2) 43 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:22:32.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:31 smithi161 bash[19769]: audit 2024-02-13T03:22:31.378648+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:32.609 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification_state table v1" 2024-02-13T03:22:32.609 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-13T03:22:32.609 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add for to alert table" 2024-02-13T03:22:32.610 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in alert_notification" 2024-02-13T03:22:32.610 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in alert_notification" 2024-02-13T03:22:32.610 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_notification_org_id_uid" 2024-02-13T03:22:32.610 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_name" 2024-02-13T03:22:32.610 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add column secure_settings in alert_notification" 2024-02-13T03:22:32.610 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert.settings to mediumtext" 2024-02-13T03:22:32.879 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_notification_state_alert_id" 2024-02-13T03:22:32.879 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_rule_tag_alert_id" 2024-02-13T03:22:32.879 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old annotation table v4" 2024-02-13T03:22:32.879 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="create annotation table v5" 2024-02-13T03:22:32.879 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 0 v3" 2024-02-13T03:22:32.879 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 1 v3" 2024-02-13T03:22:32.879 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 2 v3" 2024-02-13T03:22:32.879 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 3 v3" 2024-02-13T03:22:33.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[20824]: cephadm 2024-02-13T03:22:31.388832+0000 mgr.y (mgr.24473) 52 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-13T03:22:33.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[20824]: cephadm 2024-02-13T03:22:31.393314+0000 mgr.y (mgr.24473) 53 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi161 2024-02-13T03:22:33.148 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 4 v3" 2024-02-13T03:22:33.148 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Update annotation table charset" 2024-02-13T03:22:33.148 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add column region_id to annotation table" 2024-02-13T03:22:33.148 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:32 smithi183 bash[46554]: t=2024-02-13T03:22:32+0000 lvl=info msg="Executing migration" logger=migrator id="Drop category_id index" 2024-02-13T03:22:33.149 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add column tags to annotation table" 2024-02-13T03:22:33.149 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v2" 2024-02-13T03:22:33.149 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-13T03:22:33.149 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-13T03:22:33.149 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-13T03:22:33.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:32 smithi161 bash[19769]: cephadm 2024-02-13T03:22:31.388832+0000 mgr.y (mgr.24473) 52 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-13T03:22:33.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:32 smithi161 bash[19769]: cephadm 2024-02-13T03:22:31.393314+0000 mgr.y (mgr.24473) 53 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi161 2024-02-13T03:22:33.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:32 smithi161 bash[27879]: cephadm 2024-02-13T03:22:31.388832+0000 mgr.y (mgr.24473) 52 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-13T03:22:33.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:32 smithi161 bash[27879]: cephadm 2024-02-13T03:22:31.393314+0000 mgr.y (mgr.24473) 53 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi161 2024-02-13T03:22:33.409 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v3" 2024-02-13T03:22:33.409 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-13T03:22:33.409 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="copy annotation_tag v2 to v3" 2024-02-13T03:22:33.409 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="drop table annotation_tag_v2" 2024-02-13T03:22:33.409 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert annotations and set TEXT to empty" 2024-02-13T03:22:33.410 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add created time to annotation table" 2024-02-13T03:22:33.410 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add updated time to annotation table" 2024-02-13T03:22:33.410 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for created in annotation table" 2024-02-13T03:22:33.410 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for updated in annotation table" 2024-02-13T03:22:33.662 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Convert existing annotations from seconds to milliseconds" 2024-02-13T03:22:33.662 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add epoch_end column" 2024-02-13T03:22:33.662 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for epoch_end" 2024-02-13T03:22:33.662 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Make epoch_end the same as epoch" 2024-02-13T03:22:33.662 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Move region to single row" 2024-02-13T03:22:33.662 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch from annotation table" 2024-02-13T03:22:33.662 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-13T03:22:33.662 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-13T03:22:33.662 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-13T03:22:33.923 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-13T03:22:33.923 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for alert_id on annotation table" 2024-02-13T03:22:33.923 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="create test_data table" 2024-02-13T03:22:33.923 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_version table v1" 2024-02-13T03:22:33.923 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_version.dashboard_id" 2024-02-13T03:22:33.923 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-13T03:22:33.923 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="Set dashboard version to 1 where 0" 2024-02-13T03:22:33.923 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="save existing dashboard data in dashboard_version table v1" 2024-02-13T03:22:33.924 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_version.data to mediumtext v1" 2024-02-13T03:22:33.924 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="create team table" 2024-02-13T03:22:34.044 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:22:34.175 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[20824]: cluster 2024-02-13T03:22:32.566594+0000 mgr.y (mgr.24473) 54 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:34.175 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="add index team.org_id" 2024-02-13T03:22:34.175 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:33 smithi183 bash[46554]: t=2024-02-13T03:22:33+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_org_id_name" 2024-02-13T03:22:34.175 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="create team member table" 2024-02-13T03:22:34.175 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.org_id" 2024-02-13T03:22:34.175 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_member_org_id_team_id_user_id" 2024-02-13T03:22:34.175 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.team_id" 2024-02-13T03:22:34.175 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column email to team table" 2024-02-13T03:22:34.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:33 smithi161 bash[19769]: cluster 2024-02-13T03:22:32.566594+0000 mgr.y (mgr.24473) 54 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:34.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:33 smithi161 bash[27879]: cluster 2024-02-13T03:22:32.566594+0000 mgr.y (mgr.24473) 54 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:34.427 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external to team_member table" 2024-02-13T03:22:34.428 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column permission to team_member table" 2024-02-13T03:22:34.428 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard acl table" 2024-02-13T03:22:34.428 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_dashboard_id" 2024-02-13T03:22:34.428 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-13T03:22:34.428 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-13T03:22:34.428 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_user_id" 2024-02-13T03:22:34.428 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_team_id" 2024-02-13T03:22:34.428 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_org_id_role" 2024-02-13T03:22:34.714 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_permission" 2024-02-13T03:22:34.714 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="save default acl rules in dashboard_acl table" 2024-02-13T03:22:34.714 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="delete acl rules for deleted dashboards and folders" 2024-02-13T03:22:34.714 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="create tag table" 2024-02-13T03:22:34.714 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index tag.key_value" 2024-02-13T03:22:34.714 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="create login attempt table" 2024-02-13T03:22:34.714 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="add index login_attempt.username" 2024-02-13T03:22:34.714 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_login_attempt_username - v1" 2024-02-13T03:22:34.714 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-13T03:22:34.714 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="create login_attempt v2" 2024-02-13T03:22:34.965 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_login_attempt_username - v2" 2024-02-13T03:22:34.966 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="copy login_attempt v1 to v2" 2024-02-13T03:22:34.966 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop login_attempt_tmp_qwerty" 2024-02-13T03:22:34.966 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth table" 2024-02-13T03:22:34.966 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-13T03:22:34.966 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="alter user_auth.auth_id to length 190" 2024-02-13T03:22:34.966 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth access token to user_auth" 2024-02-13T03:22:35.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:34 smithi161 bash[19769]: audit 2024-02-13T03:22:34.551189+0000 mon.c (mon.2) 44 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-02-13T03:22:35.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:34 smithi161 bash[19769]: audit 2024-02-13T03:22:34.551764+0000 mon.c (mon.2) 45 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-02-13T03:22:35.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:34 smithi161 bash[19769]: audit 2024-02-13T03:22:34.552242+0000 mon.c (mon.2) 46 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-02-13T03:22:35.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:34 smithi161 bash[19769]: audit 2024-02-13T03:22:34.552249+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-02-13T03:22:35.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:34 smithi161 bash[19769]: audit 2024-02-13T03:22:34.553010+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-02-13T03:22:35.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:34 smithi161 bash[19769]: audit 2024-02-13T03:22:34.553409+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-02-13T03:22:35.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:34 smithi161 bash[27879]: audit 2024-02-13T03:22:34.551189+0000 mon.c (mon.2) 44 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-02-13T03:22:35.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:34 smithi161 bash[27879]: audit 2024-02-13T03:22:34.551764+0000 mon.c (mon.2) 45 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-02-13T03:22:35.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:34 smithi161 bash[27879]: audit 2024-02-13T03:22:34.552242+0000 mon.c (mon.2) 46 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-02-13T03:22:35.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:34 smithi161 bash[27879]: audit 2024-02-13T03:22:34.552249+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-02-13T03:22:35.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:34 smithi161 bash[27879]: audit 2024-02-13T03:22:34.553010+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-02-13T03:22:35.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:34 smithi161 bash[27879]: audit 2024-02-13T03:22:34.553409+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-02-13T03:22:35.218 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[20824]: audit 2024-02-13T03:22:34.551189+0000 mon.c (mon.2) 44 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-02-13T03:22:35.218 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[20824]: audit 2024-02-13T03:22:34.551764+0000 mon.c (mon.2) 45 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-02-13T03:22:35.218 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[20824]: audit 2024-02-13T03:22:34.552242+0000 mon.c (mon.2) 46 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-02-13T03:22:35.218 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[20824]: audit 2024-02-13T03:22:34.552249+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-02-13T03:22:35.218 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[20824]: audit 2024-02-13T03:22:34.553010+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-02-13T03:22:35.218 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[20824]: audit 2024-02-13T03:22:34.553409+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-02-13T03:22:35.219 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:34 smithi183 bash[46554]: t=2024-02-13T03:22:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth refresh token to user_auth" 2024-02-13T03:22:35.219 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth token type to user_auth" 2024-02-13T03:22:35.219 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth expiry to user_auth" 2024-02-13T03:22:35.219 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add index to user_id column in user_auth" 2024-02-13T03:22:35.219 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="create server_lock table" 2024-02-13T03:22:35.219 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="add index server_lock.operation_uid" 2024-02-13T03:22:35.219 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth token table" 2024-02-13T03:22:35.219 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.auth_token" 2024-02-13T03:22:35.487 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.prev_auth_token" 2024-02-13T03:22:35.487 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_auth_token.user_id" 2024-02-13T03:22:35.487 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add revoked_at to the user auth token" 2024-02-13T03:22:35.487 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="create cache_data table" 2024-02-13T03:22:35.487 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index cache_data.cache_key" 2024-02-13T03:22:35.487 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="create short_url table v1" 2024-02-13T03:22:35.487 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="add index short_url.org_id-uid" 2024-02-13T03:22:35.487 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition table" 2024-02-13T03:22:35.487 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition table" 2024-02-13T03:22:35.674 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:35 smithi161 systemd[1]: Stopping Ceph alertmanager.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:22:35.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:22:35.774 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and title columns" 2024-02-13T03:22:35.774 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and uid columns" 2024-02-13T03:22:35.774 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition table data column to mediumtext in mysql" 2024-02-13T03:22:35.774 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and title columns" 2024-02-13T03:22:35.774 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and uid columns" 2024-02-13T03:22:35.774 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and title columns" 2024-02-13T03:22:35.774 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and uid columns" 2024-02-13T03:22:35.774 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add column paused in alert_definition" 2024-02-13T03:22:35.927 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[53829]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-alertmanager.a 2024-02-13T03:22:36.043 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[20824]: cluster 2024-02-13T03:22:34.567275+0000 mgr.y (mgr.24473) 55 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:36.043 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[20824]: audit 2024-02-13T03:22:34.987412+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-02-13T03:22:36.043 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[20824]: audit 2024-02-13T03:22:34.987598+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-02-13T03:22:36.043 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[20824]: audit 2024-02-13T03:22:34.987795+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-02-13T03:22:36.043 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[20824]: cluster 2024-02-13T03:22:34.987948+0000 mon.a (mon.0) 659 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-13T03:22:36.043 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[20824]: audit 2024-02-13T03:22:34.988292+0000 mon.c (mon.2) 47 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-13T03:22:36.043 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[20824]: audit 2024-02-13T03:22:34.990472+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-13T03:22:36.043 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[20824]: audit 2024-02-13T03:22:34.995345+0000 mon.c (mon.2) 48 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-13T03:22:36.044 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[20824]: audit 2024-02-13T03:22:34.995834+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-13T03:22:36.044 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition table" 2024-02-13T03:22:36.044 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition_version table" 2024-02-13T03:22:36.044 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition_version table" 2024-02-13T03:22:36.044 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-13T03:22:36.044 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-13T03:22:36.044 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-13T03:22:36.044 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition_version table" 2024-02-13T03:22:36.044 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:35 smithi183 bash[46554]: t=2024-02-13T03:22:35+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_instance table" 2024-02-13T03:22:36.044 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-13T03:22:36.295 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-13T03:22:36.296 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add column current_state_end to alert_instance" 2024-02-13T03:22:36.296 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-13T03:22:36.296 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, current_state on alert_instance" 2024-02-13T03:22:36.296 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_org_id to rule_org_id in alert_instance" 2024-02-13T03:22:36.296 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_uid to rule_uid in alert_instance" 2024-02-13T03:22:36.296 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-13T03:22:36.296 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, current_state on alert_instance" 2024-02-13T03:22:36.556 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[52200]: level=info ts=2024-02-13T03:22:35.970Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-02-13T03:22:36.556 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule table" 2024-02-13T03:22:36.557 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and title columns" 2024-02-13T03:22:36.557 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and uid columns" 2024-02-13T03:22:36.557 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-13T03:22:36.557 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule table data column to mediumtext in mysql" 2024-02-13T03:22:36.557 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule" 2024-02-13T03:22:36.557 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule" 2024-02-13T03:22:36.557 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule" 2024-02-13T03:22:36.557 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[27879]: cluster 2024-02-13T03:22:34.567275+0000 mgr.y (mgr.24473) 55 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:36.557 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[27879]: audit 2024-02-13T03:22:34.987412+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-02-13T03:22:36.557 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[27879]: audit 2024-02-13T03:22:34.987598+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-02-13T03:22:36.557 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[27879]: audit 2024-02-13T03:22:34.987795+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-02-13T03:22:36.557 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[27879]: cluster 2024-02-13T03:22:34.987948+0000 mon.a (mon.0) 659 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-13T03:22:36.558 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[27879]: audit 2024-02-13T03:22:34.988292+0000 mon.c (mon.2) 47 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-13T03:22:36.558 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[27879]: audit 2024-02-13T03:22:34.990472+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-13T03:22:36.558 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[27879]: audit 2024-02-13T03:22:34.995345+0000 mon.c (mon.2) 48 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-13T03:22:36.558 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[27879]: audit 2024-02-13T03:22:34.995834+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-13T03:22:36.558 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[19769]: cluster 2024-02-13T03:22:34.567275+0000 mgr.y (mgr.24473) 55 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:36.558 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[19769]: audit 2024-02-13T03:22:34.987412+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-02-13T03:22:36.558 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[19769]: audit 2024-02-13T03:22:34.987598+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-02-13T03:22:36.558 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[19769]: audit 2024-02-13T03:22:34.987795+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-02-13T03:22:36.558 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[19769]: cluster 2024-02-13T03:22:34.987948+0000 mon.a (mon.0) 659 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-13T03:22:36.558 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[19769]: audit 2024-02-13T03:22:34.988292+0000 mon.c (mon.2) 47 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-13T03:22:36.558 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[19769]: audit 2024-02-13T03:22:34.990472+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-13T03:22:36.559 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[19769]: audit 2024-02-13T03:22:34.995345+0000 mon.c (mon.2) 48 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-13T03:22:36.559 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:35 smithi161 bash[19769]: audit 2024-02-13T03:22:34.995834+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-13T03:22:36.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="remove unique index from alert_rule on org_id, title columns" 2024-02-13T03:22:36.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-13T03:22:36.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add dashboard_uid column to alert_rule" 2024-02-13T03:22:36.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add panel_id column to alert_rule" 2024-02-13T03:22:36.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-13T03:22:36.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule_version table" 2024-02-13T03:22:36.817 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-13T03:22:37.062 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:36 smithi161 bash[53829]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-alertmanager-a 2024-02-13T03:22:37.062 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:36 smithi161 bash[27879]: audit 2024-02-13T03:22:35.587379+0000 mgr.y (mgr.24473) 56 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:37.063 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:36 smithi161 bash[27879]: audit 2024-02-13T03:22:35.989340+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-13T03:22:37.063 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:36 smithi161 bash[27879]: cluster 2024-02-13T03:22:35.989412+0000 mon.a (mon.0) 663 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-13T03:22:37.063 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:36 smithi161 bash[19769]: audit 2024-02-13T03:22:35.587379+0000 mgr.y (mgr.24473) 56 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:37.063 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:36 smithi161 bash[19769]: audit 2024-02-13T03:22:35.989340+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-13T03:22:37.063 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:36 smithi161 bash[19769]: cluster 2024-02-13T03:22:35.989412+0000 mon.a (mon.0) 663 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-13T03:22:37.069 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-13T03:22:37.069 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-13T03:22:37.069 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule_version" 2024-02-13T03:22:37.070 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule_version" 2024-02-13T03:22:37.070 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule_version" 2024-02-13T03:22:37.070 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id=create_alert_configuration_table 2024-02-13T03:22:37.070 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[46554]: t=2024-02-13T03:22:36+0000 lvl=info msg="Executing migration" logger=migrator id="Add column default in alert_configuration" 2024-02-13T03:22:37.070 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-13T03:22:37.070 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[20824]: audit 2024-02-13T03:22:35.587379+0000 mgr.y (mgr.24473) 56 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:37.070 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[20824]: audit 2024-02-13T03:22:35.989340+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-13T03:22:37.070 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:36 smithi183 bash[20824]: cluster 2024-02-13T03:22:35.989412+0000 mon.a (mon.0) 663 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-13T03:22:37.355 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="add column org_id in alert_configuration" 2024-02-13T03:22:37.355 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_configuration table on org_id column" 2024-02-13T03:22:37.356 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id=create_ngalert_configuration_table 2024-02-13T03:22:37.356 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="add index in ngalert_configuration on org_id column" 2024-02-13T03:22:37.356 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="clear migration entry \"remove unified alerting data\"" 2024-02-13T03:22:37.356 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="move dashboard alerts to unified alerting" 2024-02-13T03:22:37.356 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element table v1" 2024-02-13T03:22:37.356 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element org_id-folder_id-name-kind" 2024-02-13T03:22:37.356 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element_connection table v1" 2024-02-13T03:22:37.356 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element_connection element_id-kind-connection_id" 2024-02-13T03:22:37.356 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index library_element org_id_uid" 2024-02-13T03:22:37.427 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:37 smithi161 bash[53932]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-alertmanager-a 2024-02-13T03:22:37.427 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:37 smithi161 systemd[1]: Stopped Ceph alertmanager.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:22:37.427 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:37 smithi161 systemd[1]: Started Ceph alertmanager.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:22:37.736 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="clone move dashboard alerts to unified alerting" 2024-02-13T03:22:38.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.048+0000 7f622083f700 1 -- 172.21.15.161:0/3406766408 >> v1:172.21.15.161:6790/0 conn(0x7f621c0fc150 legacy=0x7f621c0fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:38.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.048+0000 7f622083f700 1 -- 172.21.15.161:0/3406766408 shutdown_connections 2024-02-13T03:22:38.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.048+0000 7f622083f700 1 -- 172.21.15.161:0/3406766408 >> 172.21.15.161:0/3406766408 conn(0x7f621c06ff90 msgr2=0x7f621c0f4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:38.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.048+0000 7f622083f700 1 -- 172.21.15.161:0/3406766408 shutdown_connections 2024-02-13T03:22:38.055 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.048+0000 7f622083f700 1 -- 172.21.15.161:0/3406766408 wait complete. 2024-02-13T03:22:38.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f622083f700 1 Processor -- start 2024-02-13T03:22:38.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f622083f700 1 -- start start 2024-02-13T03:22:38.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f622083f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f621c1194f0 con 0x7f621c102ab0 2024-02-13T03:22:38.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f622083f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f621c11a6d0 con 0x7f621c0fefa0 2024-02-13T03:22:38.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f622083f700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f621c11b8d0 con 0x7f621c0fc150 2024-02-13T03:22:38.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f621ad9d700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f621c102ab0 0x7f621c117c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:36722/0 (socket says 172.21.15.161:36722) 2024-02-13T03:22:38.056 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f621ad9d700 1 -- 172.21.15.161:0/725065083 learned_addr learned my addr 172.21.15.161:0/725065083 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:22:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794189449 0 0) 0x7f621c11b8d0 con 0x7f621c0fc150 2024-02-13T03:22:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6204003030 con 0x7f621c0fc150 2024-02-13T03:22:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152134975 0 0) 0x7f621c1194f0 con 0x7f621c102ab0 2024-02-13T03:22:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f621c11b8d0 con 0x7f621c102ab0 2024-02-13T03:22:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1624610838 0 0) 0x7f621c11a6d0 con 0x7f621c0fefa0 2024-02-13T03:22:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f621c1194f0 con 0x7f621c0fefa0 2024-02-13T03:22:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1365284702 0 0) 0x7f6204003030 con 0x7f621c0fc150 2024-02-13T03:22:38.058 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f621c11a6d0 con 0x7f621c0fc150 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2146096910 0 0) 0x7f621c11b8d0 con 0x7f621c102ab0 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6204003030 con 0x7f621c102ab0 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f620c002df0 con 0x7f621c0fc150 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f6214001d50 con 0x7f621c102ab0 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 838781822 0 0) 0x7f621c1194f0 con 0x7f621c0fefa0 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f621c11b8d0 con 0x7f621c0fefa0 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f62100019f0 con 0x7f621c0fefa0 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 754906615 0 0) 0x7f621c11a6d0 con 0x7f621c0fc150 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 >> v1:172.21.15.183:6789/0 conn(0x7f621c0fefa0 legacy=0x7f621c114500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 >> v1:172.21.15.161:6789/0 conn(0x7f621c102ab0 legacy=0x7f621c117c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c11cad0 con 0x7f621c0fc150 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f622083f700 1 -- 172.21.15.161:0/725065083 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621c1196e0 con 0x7f621c0fc150 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f622083f700 1 -- 172.21.15.161:0/725065083 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c119b50 con 0x7f621c0fc150 2024-02-13T03:22:38.059 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f620c001a00 con 0x7f621c0fc150 2024-02-13T03:22:38.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.052+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f620c003770 con 0x7f621c0fc150 2024-02-13T03:22:38.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.056+0000 7f622083f700 1 -- 172.21.15.161:0/725065083 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61fc004f20 con 0x7f621c0fc150 2024-02-13T03:22:38.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.056+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f620c004a00 con 0x7f621c0fc150 2024-02-13T03:22:38.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.056+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f620c075f30 con 0x7f621c0fc150 2024-02-13T03:22:38.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.060+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f620c049aa0 con 0x7f621c0fc150 2024-02-13T03:22:38.077 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="create data_keys table" 2024-02-13T03:22:38.077 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:37 smithi183 bash[46554]: t=2024-02-13T03:22:37+0000 lvl=info msg="Executing migration" logger=migrator id="create kv_store table v1" 2024-02-13T03:22:38.077 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[46554]: t=2024-02-13T03:22:38+0000 lvl=info msg="Executing migration" logger=migrator id="add index kv_store.org_id-namespace-key" 2024-02-13T03:22:38.077 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[20824]: cluster 2024-02-13T03:22:36.568302+0000 mgr.y (mgr.24473) 57 : cluster [DBG] pgmap v36: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:22:38.077 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[20824]: cluster 2024-02-13T03:22:36.992183+0000 mon.a (mon.0) 664 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-13T03:22:38.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[20824]: audit 2024-02-13T03:22:37.132276+0000 mon.a (mon.0) 665 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:38.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[20824]: cephadm 2024-02-13T03:22:37.134700+0000 mgr.y (mgr.24473) 58 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-02-13T03:22:38.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[20824]: cephadm 2024-02-13T03:22:37.139242+0000 mgr.y (mgr.24473) 59 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi183 2024-02-13T03:22:38.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.244+0000 7f622083f700 1 -- 172.21.15.161:0/725065083 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"} v 0) v1 -- 0x7f61fc005750 con 0x7f621c0fc150 2024-02-13T03:22:38.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.244+0000 7f620b7fe700 1 -- 172.21.15.161:0/725065083 <== mon.2 v1:172.21.15.161:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (unknown 2126108022 0 3519293104) 0x7f620c047f10 con 0x7f621c0fc150 2024-02-13T03:22:38.248 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-02-13T03:22:38.250 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:37 smithi161 bash[27879]: cluster 2024-02-13T03:22:36.568302+0000 mgr.y (mgr.24473) 57 : cluster [DBG] pgmap v36: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:22:38.250 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:37 smithi161 bash[27879]: cluster 2024-02-13T03:22:36.992183+0000 mon.a (mon.0) 664 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-13T03:22:38.250 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:37 smithi161 bash[27879]: audit 2024-02-13T03:22:37.132276+0000 mon.a (mon.0) 665 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:38.250 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:37 smithi161 bash[27879]: cephadm 2024-02-13T03:22:37.134700+0000 mgr.y (mgr.24473) 58 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-02-13T03:22:38.251 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:37 smithi161 bash[27879]: cephadm 2024-02-13T03:22:37.139242+0000 mgr.y (mgr.24473) 59 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi183 2024-02-13T03:22:38.251 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:37 smithi161 bash[19769]: cluster 2024-02-13T03:22:36.568302+0000 mgr.y (mgr.24473) 57 : cluster [DBG] pgmap v36: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:22:38.251 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:37 smithi161 bash[19769]: cluster 2024-02-13T03:22:36.992183+0000 mon.a (mon.0) 664 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-13T03:22:38.251 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:37 smithi161 bash[19769]: audit 2024-02-13T03:22:37.132276+0000 mon.a (mon.0) 665 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:38.251 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:37 smithi161 bash[19769]: cephadm 2024-02-13T03:22:37.134700+0000 mgr.y (mgr.24473) 58 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-02-13T03:22:38.251 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:37 smithi161 bash[19769]: cephadm 2024-02-13T03:22:37.139242+0000 mgr.y (mgr.24473) 59 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi183 2024-02-13T03:22:38.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.244+0000 7f622083f700 1 -- 172.21.15.161:0/725065083 >> v1:172.21.15.161:6800/4114337139 conn(0x7f62040622f0 legacy=0x7f62040647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:38.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.244+0000 7f622083f700 1 -- 172.21.15.161:0/725065083 >> v1:172.21.15.161:6790/0 conn(0x7f621c0fc150 legacy=0x7f621c0fb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:38.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.248+0000 7f622083f700 1 -- 172.21.15.161:0/725065083 shutdown_connections 2024-02-13T03:22:38.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.248+0000 7f622083f700 1 -- 172.21.15.161:0/725065083 >> 172.21.15.161:0/725065083 conn(0x7f621c06ff90 msgr2=0x7f621c106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:38.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.248+0000 7f622083f700 1 -- 172.21.15.161:0/725065083 shutdown_connections 2024-02-13T03:22:38.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:38.248+0000 7f622083f700 1 -- 172.21.15.161:0/725065083 wait complete. 2024-02-13T03:22:38.331 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[46554]: t=2024-02-13T03:22:38+0000 lvl=info msg="Executing migration" logger=migrator id="update dashboard_uid and panel_id from existing annotations" 2024-02-13T03:22:38.503 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:38 smithi161 bash[53972]: level=info ts=2024-02-13T03:22:38.439Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-02-13T03:22:38.503 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:38 smithi161 bash[53972]: level=info ts=2024-02-13T03:22:38.439Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-02-13T03:22:38.503 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:38 smithi161 bash[53972]: level=info ts=2024-02-13T03:22:38.441Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.161 port=9094 2024-02-13T03:22:38.503 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:38 smithi161 bash[53972]: level=info ts=2024-02-13T03:22:38.443Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-13T03:22:38.612 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[46554]: t=2024-02-13T03:22:38+0000 lvl=info msg="Executing migration" logger=migrator id="create permission table" 2024-02-13T03:22:38.612 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[46554]: t=2024-02-13T03:22:38+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index permission.role_id" 2024-02-13T03:22:38.613 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[46554]: t=2024-02-13T03:22:38+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_id_action_scope" 2024-02-13T03:22:38.613 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[46554]: t=2024-02-13T03:22:38+0000 lvl=info msg="Executing migration" logger=migrator id="create role table" 2024-02-13T03:22:38.898 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[46554]: t=2024-02-13T03:22:38+0000 lvl=info msg="Executing migration" logger=migrator id="add column display_name" 2024-02-13T03:22:38.898 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[46554]: t=2024-02-13T03:22:38+0000 lvl=info msg="Executing migration" logger=migrator id="add column group_name" 2024-02-13T03:22:38.898 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[46554]: t=2024-02-13T03:22:38+0000 lvl=info msg="Executing migration" logger=migrator id="add index role.org_id" 2024-02-13T03:22:38.898 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[46554]: t=2024-02-13T03:22:38+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_org_id_name" 2024-02-13T03:22:38.927 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:38 smithi161 bash[53972]: level=info ts=2024-02-13T03:22:38.500Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-13T03:22:38.927 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:38 smithi161 bash[53972]: level=info ts=2024-02-13T03:22:38.501Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-13T03:22:38.927 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:38 smithi161 bash[53972]: level=info ts=2024-02-13T03:22:38.507Z caller=main.go:518 msg=Listening address=:9093 2024-02-13T03:22:38.927 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:38 smithi161 bash[53972]: level=info ts=2024-02-13T03:22:38.507Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-02-13T03:22:38.975 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd pool get default.rgw.control pg_num 2024-02-13T03:22:39.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[20824]: audit 2024-02-13T03:22:38.246348+0000 mon.c (mon.2) 49 : audit [DBG] from='client.? v1:172.21.15.161:0/725065083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-13T03:22:39.151 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:38 smithi183 bash[46554]: t=2024-02-13T03:22:38+0000 lvl=info msg="Executing migration" logger=migrator id="add index role_org_id_uid" 2024-02-13T03:22:39.151 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[46554]: t=2024-02-13T03:22:39+0000 lvl=info msg="Executing migration" logger=migrator id="create team role table" 2024-02-13T03:22:39.151 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[46554]: t=2024-02-13T03:22:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.org_id" 2024-02-13T03:22:39.151 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[46554]: t=2024-02-13T03:22:39+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_role_org_id_team_id_role_id" 2024-02-13T03:22:39.151 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[46554]: t=2024-02-13T03:22:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.team_id" 2024-02-13T03:22:39.151 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[46554]: t=2024-02-13T03:22:39+0000 lvl=info msg="Executing migration" logger=migrator id="create user role table" 2024-02-13T03:22:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:39 smithi161 bash[19769]: audit 2024-02-13T03:22:38.246348+0000 mon.c (mon.2) 49 : audit [DBG] from='client.? v1:172.21.15.161:0/725065083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-13T03:22:39.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:39 smithi161 bash[27879]: audit 2024-02-13T03:22:38.246348+0000 mon.c (mon.2) 49 : audit [DBG] from='client.? v1:172.21.15.161:0/725065083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-13T03:22:39.445 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:22:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:22:39.445 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[46554]: t=2024-02-13T03:22:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.org_id" 2024-02-13T03:22:39.446 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[46554]: t=2024-02-13T03:22:39+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_role_org_id_user_id_role_id" 2024-02-13T03:22:39.446 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[46554]: t=2024-02-13T03:22:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.user_id" 2024-02-13T03:22:39.736 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[46554]: t=2024-02-13T03:22:39+0000 lvl=info msg="Executing migration" logger=migrator id="create builtin role table" 2024-02-13T03:22:39.737 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[46554]: t=2024-02-13T03:22:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.role_id" 2024-02-13T03:22:40.065 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:40 smithi183 bash[20824]: cluster 2024-02-13T03:22:38.568963+0000 mgr.y (mgr.24473) 60 : cluster [DBG] pgmap v37: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:22:40.065 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[46554]: t=2024-02-13T03:22:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.name" 2024-02-13T03:22:40.065 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[46554]: t=2024-02-13T03:22:39+0000 lvl=info msg="Executing migration" logger=migrator id="Add column org_id to builtin_role table" 2024-02-13T03:22:40.065 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:39 smithi183 bash[46554]: t=2024-02-13T03:22:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.org_id" 2024-02-13T03:22:40.065 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:40 smithi183 bash[46554]: t=2024-02-13T03:22:40+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_org_id_role_id_role" 2024-02-13T03:22:40.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:40 smithi161 bash[27879]: cluster 2024-02-13T03:22:38.568963+0000 mgr.y (mgr.24473) 60 : cluster [DBG] pgmap v37: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:22:40.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:40 smithi161 bash[19769]: cluster 2024-02-13T03:22:38.568963+0000 mgr.y (mgr.24473) 60 : cluster [DBG] pgmap v37: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:22:40.433 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:40 smithi183 bash[46554]: t=2024-02-13T03:22:40+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index role_org_id_uid" 2024-02-13T03:22:40.434 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:40 smithi183 bash[46554]: t=2024-02-13T03:22:40+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role.uid" 2024-02-13T03:22:40.736 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:40 smithi183 bash[46554]: t=2024-02-13T03:22:40+0000 lvl=info msg="Executing migration" logger=migrator id="create seed assignment table" 2024-02-13T03:22:40.737 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:40 smithi183 bash[46554]: t=2024-02-13T03:22:40+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_role_name" 2024-02-13T03:22:40.737 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:40 smithi183 bash[46554]: t=2024-02-13T03:22:40+0000 lvl=info msg="migrations completed" logger=migrator performed=381 skipped=0 duration=14.330319196s 2024-02-13T03:22:40.737 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:40 smithi183 bash[46554]: t=2024-02-13T03:22:40+0000 lvl=info msg="Created default organization" logger=sqlstore 2024-02-13T03:22:40.927 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:40 smithi161 bash[53972]: level=info ts=2024-02-13T03:22:40.444Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.001099193s 2024-02-13T03:22:41.069 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:40 smithi183 bash[46554]: t=2024-02-13T03:22:40+0000 lvl=info msg="Initialising plugins" logger=plugin.manager 2024-02-13T03:22:41.069 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:40 smithi183 bash[46554]: t=2024-02-13T03:22:40+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=input 2024-02-13T03:22:41.069 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:40 smithi183 bash[46554]: t=2024-02-13T03:22:40+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=grafana-piechart-panel 2024-02-13T03:22:41.069 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:40 smithi183 bash[46554]: t=2024-02-13T03:22:40+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=vonage-status-panel 2024-02-13T03:22:41.069 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:40 smithi183 bash[46554]: t=2024-02-13T03:22:40+0000 lvl=info msg="Live Push Gateway initialization" logger=live.push_http 2024-02-13T03:22:41.069 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:41 smithi183 bash[46554]: t=2024-02-13T03:22:41+0000 lvl=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" logger=provisioning.datasources filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-02-13T03:22:41.069 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:41 smithi183 bash[46554]: t=2024-02-13T03:22:41+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-13T03:22:41.333 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:41 smithi183 bash[20824]: cluster 2024-02-13T03:22:41.011326+0000 mon.a (mon.0) 666 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-13T03:22:41.333 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:41 smithi183 bash[20824]: cluster 2024-02-13T03:22:41.011392+0000 mon.a (mon.0) 667 : cluster [INF] Cluster is now healthy 2024-02-13T03:22:41.333 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:41 smithi183 bash[46554]: t=2024-02-13T03:22:41+0000 lvl=info msg="warming cache for startup" logger=ngalert 2024-02-13T03:22:41.333 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:41 smithi183 bash[46554]: t=2024-02-13T03:22:41+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=https subUrl= socket= 2024-02-13T03:22:41.333 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:22:41 smithi183 bash[46554]: t=2024-02-13T03:22:41+0000 lvl=info msg="starting MultiOrg Alertmanager" logger=ngalert.multiorg.alertmanager 2024-02-13T03:22:41.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:41 smithi161 bash[19769]: cluster 2024-02-13T03:22:41.011326+0000 mon.a (mon.0) 666 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-13T03:22:41.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:41 smithi161 bash[19769]: cluster 2024-02-13T03:22:41.011392+0000 mon.a (mon.0) 667 : cluster [INF] Cluster is now healthy 2024-02-13T03:22:41.428 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:22:41] "GET /metrics HTTP/1.1" 200 172133 "" "Prometheus/2.33.4" 2024-02-13T03:22:41.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:41 smithi161 bash[27879]: cluster 2024-02-13T03:22:41.011326+0000 mon.a (mon.0) 666 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-13T03:22:41.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:41 smithi161 bash[27879]: cluster 2024-02-13T03:22:41.011392+0000 mon.a (mon.0) 667 : cluster [INF] Cluster is now healthy 2024-02-13T03:22:42.328 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[20824]: cluster 2024-02-13T03:22:40.570409+0000 mgr.y (mgr.24473) 61 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-02-13T03:22:42.329 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 systemd[1]: Stopping Ceph prometheus.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:22:42.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:42 smithi161 bash[27879]: cluster 2024-02-13T03:22:40.570409+0000 mgr.y (mgr.24473) 61 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-02-13T03:22:42.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:42 smithi161 bash[19769]: cluster 2024-02-13T03:22:40.570409+0000 mgr.y (mgr.24473) 61 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-02-13T03:22:42.737 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[47292]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-prometheus.a 2024-02-13T03:22:42.737 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[46136]: ts=2024-02-13T03:22:42.451Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-13T03:22:42.737 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[46136]: ts=2024-02-13T03:22:42.451Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-02-13T03:22:42.737 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[46136]: ts=2024-02-13T03:22:42.451Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-02-13T03:22:42.737 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[46136]: ts=2024-02-13T03:22:42.451Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-02-13T03:22:42.737 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[46136]: ts=2024-02-13T03:22:42.451Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-02-13T03:22:42.737 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[46136]: ts=2024-02-13T03:22:42.451Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-02-13T03:22:42.737 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[46136]: ts=2024-02-13T03:22:42.451Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-13T03:22:42.737 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[46136]: ts=2024-02-13T03:22:42.452Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-02-13T03:22:42.737 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[46136]: ts=2024-02-13T03:22:42.452Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-02-13T03:22:42.738 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[46136]: ts=2024-02-13T03:22:42.452Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-02-13T03:22:42.738 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[46136]: ts=2024-02-13T03:22:42.452Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-02-13T03:22:42.738 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:42 smithi183 bash[46136]: ts=2024-02-13T03:22:42.452Z caller=main.go:1066 level=info msg="See you next time!" 2024-02-13T03:22:43.486 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:43 smithi183 bash[47292]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-prometheus-a 2024-02-13T03:22:43.487 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:43 smithi183 systemd[1]: Stopped Ceph prometheus.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:22:43.487 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:43 smithi183 systemd[1]: Started Ceph prometheus.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:22:43.562 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Bus STOPPING 2024-02-13T03:22:43.562 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T03:22:43.562 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Bus STOPPED 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Bus STARTING 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Serving on http://:::9283 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Bus STARTED 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Bus STOPPING 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Bus STOPPED 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Bus STARTING 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Serving on http://:::9283 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Bus STARTED 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Bus STOPPING 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Bus STOPPED 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Bus STARTING 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Serving on http://:::9283 2024-02-13T03:22:43.563 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:43 smithi161 bash[20075]: [13/Feb/2024:03:22:43] ENGINE Bus STARTED 2024-02-13T03:22:43.811 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:22:44.487 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.331Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-02-13T03:22:44.487 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.332Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-02-13T03:22:44.487 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.332Z caller=main.go:518 level=info host_details="(Linux 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 smithi183 (none))" 2024-02-13T03:22:44.487 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.332Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-13T03:22:44.487 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.332Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-13T03:22:44.488 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.334Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-02-13T03:22:44.488 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.335Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-02-13T03:22:44.488 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.336Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-02-13T03:22:44.488 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.340Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-13T03:22:44.488 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.340Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.145µs 2024-02-13T03:22:44.488 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.340Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-13T03:22:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: cluster 2024-02-13T03:22:42.571181+0000 mgr.y (mgr.24473) 62 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-02-13T03:22:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.185307+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.190865+0000 mon.c (mon.2) 50 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-13T03:22:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.191441+0000 mgr.y (mgr.24473) 63 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-13T03:22:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.192789+0000 mon.c (mon.2) 51 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi161.front.sepia.ceph.com:9093"}]: dispatch 2024-02-13T03:22:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.193251+0000 mgr.y (mgr.24473) 64 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi161.front.sepia.ceph.com:9093"}]: dispatch 2024-02-13T03:22:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.203054+0000 mon.a (mon.0) 669 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.209900+0000 mon.c (mon.2) 52 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-13T03:22:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.210677+0000 mgr.y (mgr.24473) 65 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-13T03:22:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.218143+0000 mon.c (mon.2) 53 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi183.front.sepia.ceph.com:9095"}]: dispatch 2024-02-13T03:22:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.218799+0000 mgr.y (mgr.24473) 66 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi183.front.sepia.ceph.com:9095"}]: dispatch 2024-02-13T03:22:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.226829+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.236229+0000 mon.c (mon.2) 54 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-13T03:22:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.244699+0000 mon.c (mon.2) 55 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi183.front.sepia.ceph.com:3000"}]: dispatch 2024-02-13T03:22:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.254110+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[20824]: audit 2024-02-13T03:22:43.259145+0000 mon.c (mon.2) 56 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:22:44.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: cluster 2024-02-13T03:22:42.571181+0000 mgr.y (mgr.24473) 62 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-02-13T03:22:44.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.185307+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:44.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.190865+0000 mon.c (mon.2) 50 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-13T03:22:44.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.191441+0000 mgr.y (mgr.24473) 63 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-13T03:22:44.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.192789+0000 mon.c (mon.2) 51 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi161.front.sepia.ceph.com:9093"}]: dispatch 2024-02-13T03:22:44.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.193251+0000 mgr.y (mgr.24473) 64 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi161.front.sepia.ceph.com:9093"}]: dispatch 2024-02-13T03:22:44.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.203054+0000 mon.a (mon.0) 669 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:44.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.209900+0000 mon.c (mon.2) 52 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-13T03:22:44.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.210677+0000 mgr.y (mgr.24473) 65 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-13T03:22:44.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.218143+0000 mon.c (mon.2) 53 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi183.front.sepia.ceph.com:9095"}]: dispatch 2024-02-13T03:22:44.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.218799+0000 mgr.y (mgr.24473) 66 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi183.front.sepia.ceph.com:9095"}]: dispatch 2024-02-13T03:22:44.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.226829+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:44.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.236229+0000 mon.c (mon.2) 54 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-13T03:22:44.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.244699+0000 mon.c (mon.2) 55 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi183.front.sepia.ceph.com:3000"}]: dispatch 2024-02-13T03:22:44.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.254110+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:44.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[19769]: audit 2024-02-13T03:22:43.259145+0000 mon.c (mon.2) 56 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:22:44.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: cluster 2024-02-13T03:22:42.571181+0000 mgr.y (mgr.24473) 62 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-02-13T03:22:44.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.185307+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:44.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.190865+0000 mon.c (mon.2) 50 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-13T03:22:44.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.191441+0000 mgr.y (mgr.24473) 63 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-13T03:22:44.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.192789+0000 mon.c (mon.2) 51 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi161.front.sepia.ceph.com:9093"}]: dispatch 2024-02-13T03:22:44.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.193251+0000 mgr.y (mgr.24473) 64 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi161.front.sepia.ceph.com:9093"}]: dispatch 2024-02-13T03:22:44.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.203054+0000 mon.a (mon.0) 669 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:44.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.209900+0000 mon.c (mon.2) 52 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-13T03:22:44.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.210677+0000 mgr.y (mgr.24473) 65 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-13T03:22:44.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.218143+0000 mon.c (mon.2) 53 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi183.front.sepia.ceph.com:9095"}]: dispatch 2024-02-13T03:22:44.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.218799+0000 mgr.y (mgr.24473) 66 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi183.front.sepia.ceph.com:9095"}]: dispatch 2024-02-13T03:22:44.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.226829+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:44.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.236229+0000 mon.c (mon.2) 54 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-13T03:22:44.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.244699+0000 mon.c (mon.2) 55 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi183.front.sepia.ceph.com:3000"}]: dispatch 2024-02-13T03:22:44.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.254110+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:44.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:44 smithi161 bash[27879]: audit 2024-02-13T03:22:43.259145+0000 mon.c (mon.2) 56 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:22:44.987 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.615Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-02-13T03:22:44.987 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.616Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-02-13T03:22:44.987 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.616Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=34.169µs wal_replay_duration=276.325647ms total_replay_duration=276.371973ms 2024-02-13T03:22:44.987 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.621Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-02-13T03:22:44.987 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.621Z caller=main.go:947 level=info msg="TSDB started" 2024-02-13T03:22:44.987 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.621Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-13T03:22:44.987 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.641Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.239645ms db_storage=1.206µs remote_storage=1.811µs web_handler=625ns query_engine=1.272µs scrape=367.343µs scrape_sd=36.503µs notify=25.565µs notify_sd=7.951µs rules=19.492874ms 2024-02-13T03:22:44.987 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:22:44 smithi183 bash[47417]: ts=2024-02-13T03:22:44.641Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-02-13T03:22:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:45 smithi183 bash[20824]: audit 2024-02-13T03:22:43.236831+0000 mgr.y (mgr.24473) 67 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-13T03:22:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:45 smithi183 bash[20824]: audit 2024-02-13T03:22:43.245278+0000 mgr.y (mgr.24473) 68 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi183.front.sepia.ceph.com:3000"}]: dispatch 2024-02-13T03:22:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:45 smithi161 bash[27879]: audit 2024-02-13T03:22:43.236831+0000 mgr.y (mgr.24473) 67 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-13T03:22:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:45 smithi161 bash[27879]: audit 2024-02-13T03:22:43.245278+0000 mgr.y (mgr.24473) 68 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi183.front.sepia.ceph.com:3000"}]: dispatch 2024-02-13T03:22:45.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:45 smithi161 bash[19769]: audit 2024-02-13T03:22:43.236831+0000 mgr.y (mgr.24473) 67 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-13T03:22:45.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:45 smithi161 bash[19769]: audit 2024-02-13T03:22:43.245278+0000 mgr.y (mgr.24473) 68 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi183.front.sepia.ceph.com:3000"}]: dispatch 2024-02-13T03:22:45.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:22:45 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:22:46.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:46 smithi183 bash[20824]: cluster 2024-02-13T03:22:44.571888+0000 mgr.y (mgr.24473) 69 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 34 B/s, 1 objects/s recovering 2024-02-13T03:22:46.566 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:46 smithi161 bash[27879]: cluster 2024-02-13T03:22:44.571888+0000 mgr.y (mgr.24473) 69 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 34 B/s, 1 objects/s recovering 2024-02-13T03:22:46.567 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:46 smithi161 bash[19769]: cluster 2024-02-13T03:22:44.571888+0000 mgr.y (mgr.24473) 69 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 34 B/s, 1 objects/s recovering 2024-02-13T03:22:46.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.796+0000 7fad8a78b700 1 -- 172.21.15.161:0/2172528634 >> v1:172.21.15.161:6790/0 conn(0x7fad841067a0 legacy=0x7fad84108b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:46.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.796+0000 7fad8a78b700 1 -- 172.21.15.161:0/2172528634 shutdown_connections 2024-02-13T03:22:46.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.796+0000 7fad8a78b700 1 -- 172.21.15.161:0/2172528634 >> 172.21.15.161:0/2172528634 conn(0x7fad840f2610 msgr2=0x7fad840f4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:46.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.796+0000 7fad8a78b700 1 -- 172.21.15.161:0/2172528634 shutdown_connections 2024-02-13T03:22:46.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.796+0000 7fad8a78b700 1 -- 172.21.15.161:0/2172528634 wait complete. 2024-02-13T03:22:46.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad8a78b700 1 Processor -- start 2024-02-13T03:22:46.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad8a78b700 1 -- start start 2024-02-13T03:22:46.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad8a78b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fad84119780 con 0x7fad840f68d0 2024-02-13T03:22:46.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad8a78b700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fad8411a960 con 0x7fad841067a0 2024-02-13T03:22:46.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad8a78b700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fad8411bb60 con 0x7fad840f73c0 2024-02-13T03:22:46.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad83fff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fad840f68d0 0x7fad84101c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:36746/0 (socket says 172.21.15.161:36746) 2024-02-13T03:22:46.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad83fff700 1 -- 172.21.15.161:0/2442602860 learned_addr learned my addr 172.21.15.161:0/2442602860 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:22:46.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad7b7fe700 1 --1- 172.21.15.161:0/2442602860 >> v1:172.21.15.161:6790/0 conn(0x7fad840f73c0 0x7fad84102380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:43782/0 (socket says 172.21.15.161:43782) 2024-02-13T03:22:46.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103159635 0 0) 0x7fad84119780 con 0x7fad840f68d0 2024-02-13T03:22:46.806 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad70003630 con 0x7fad840f68d0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536946431 0 0) 0x7fad8411bb60 con 0x7fad840f73c0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad84119780 con 0x7fad840f73c0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692619153 0 0) 0x7fad8411a960 con 0x7fad841067a0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad8411bb60 con 0x7fad841067a0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1158789212 0 0) 0x7fad84119780 con 0x7fad840f73c0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fad8411a960 con 0x7fad840f73c0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fad6c0018c0 con 0x7fad840f73c0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 8570891 0 0) 0x7fad70003630 con 0x7fad840f68d0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fad84119780 con 0x7fad840f68d0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fad74001980 con 0x7fad840f68d0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 31616478 0 0) 0x7fad8411bb60 con 0x7fad841067a0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fad70003630 con 0x7fad841067a0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fad7c002240 con 0x7fad841067a0 2024-02-13T03:22:46.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4221379075 0 0) 0x7fad84119780 con 0x7fad840f68d0 2024-02-13T03:22:46.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 >> v1:172.21.15.161:6790/0 conn(0x7fad840f73c0 legacy=0x7fad84102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:46.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 >> v1:172.21.15.183:6789/0 conn(0x7fad841067a0 legacy=0x7fad84117e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:46.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8411cd60 con 0x7fad840f68d0 2024-02-13T03:22:46.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad8a78b700 1 -- 172.21.15.161:0/2442602860 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8411bd50 con 0x7fad840f68d0 2024-02-13T03:22:46.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fad74002ce0 con 0x7fad840f68d0 2024-02-13T03:22:46.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fad74005b50 con 0x7fad840f68d0 2024-02-13T03:22:46.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.800+0000 7fad8a78b700 1 -- 172.21.15.161:0/2442602860 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8411c240 con 0x7fad840f68d0 2024-02-13T03:22:46.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.804+0000 7fad8a78b700 1 -- 172.21.15.161:0/2442602860 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad840604a0 con 0x7fad840f68d0 2024-02-13T03:22:46.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.804+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fad74018aa0 con 0x7fad840f68d0 2024-02-13T03:22:46.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.804+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7fad74078670 con 0x7fad840f68d0 2024-02-13T03:22:46.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.808+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fad7404c330 con 0x7fad840f68d0 2024-02-13T03:22:46.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.992+0000 7fad8a78b700 1 -- 172.21.15.161:0/2442602860 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"} v 0) v1 -- 0x7fad8410c330 con 0x7fad840f68d0 2024-02-13T03:22:46.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:46.996+0000 7fad81ffb700 1 -- 172.21.15.161:0/2442602860 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]=0 v63) v1 ==== 108+0+11 (unknown 720975082 0 3519293104) 0x7fad7404a740 con 0x7fad840f68d0 2024-02-13T03:22:47.000 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-02-13T03:22:47.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:47.000+0000 7fad8a78b700 1 -- 172.21.15.161:0/2442602860 >> v1:172.21.15.161:6800/4114337139 conn(0x7fad700622b0 legacy=0x7fad70064770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:47.003 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:47.000+0000 7fad8a78b700 1 -- 172.21.15.161:0/2442602860 >> v1:172.21.15.161:6789/0 conn(0x7fad840f68d0 legacy=0x7fad84101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:47.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:47.000+0000 7fad8a78b700 1 -- 172.21.15.161:0/2442602860 shutdown_connections 2024-02-13T03:22:47.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:47.000+0000 7fad8a78b700 1 -- 172.21.15.161:0/2442602860 >> 172.21.15.161:0/2442602860 conn(0x7fad840f2610 msgr2=0x7fad84105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:47.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:47.000+0000 7fad8a78b700 1 -- 172.21.15.161:0/2442602860 shutdown_connections 2024-02-13T03:22:47.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:47.000+0000 7fad8a78b700 1 -- 172.21.15.161:0/2442602860 wait complete. 2024-02-13T03:22:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:47 smithi161 bash[27879]: audit 2024-02-13T03:22:45.595592+0000 mgr.y (mgr.24473) 70 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:47 smithi161 bash[27879]: audit 2024-02-13T03:22:46.997772+0000 mon.a (mon.0) 672 : audit [DBG] from='client.? v1:172.21.15.161:0/2442602860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-13T03:22:47.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:47 smithi161 bash[19769]: audit 2024-02-13T03:22:45.595592+0000 mgr.y (mgr.24473) 70 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:47.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:47 smithi161 bash[19769]: audit 2024-02-13T03:22:46.997772+0000 mon.a (mon.0) 672 : audit [DBG] from='client.? v1:172.21.15.161:0/2442602860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-13T03:22:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:47 smithi183 bash[20824]: audit 2024-02-13T03:22:45.595592+0000 mgr.y (mgr.24473) 70 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:47 smithi183 bash[20824]: audit 2024-02-13T03:22:46.997772+0000 mon.a (mon.0) 672 : audit [DBG] from='client.? v1:172.21.15.161:0/2442602860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-13T03:22:47.660 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd pool get default.rgw.meta pg_num 2024-02-13T03:22:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:48 smithi183 bash[20824]: cluster 2024-02-13T03:22:46.572773+0000 mgr.y (mgr.24473) 71 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 96 B/s wr, 3 op/s; 31 B/s, 1 objects/s recovering 2024-02-13T03:22:48.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:48 smithi161 bash[19769]: cluster 2024-02-13T03:22:46.572773+0000 mgr.y (mgr.24473) 71 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 96 B/s wr, 3 op/s; 31 B/s, 1 objects/s recovering 2024-02-13T03:22:48.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:48 smithi161 bash[27879]: cluster 2024-02-13T03:22:46.572773+0000 mgr.y (mgr.24473) 71 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 96 B/s wr, 3 op/s; 31 B/s, 1 objects/s recovering 2024-02-13T03:22:48.677 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:22:48 smithi161 bash[53972]: level=info ts=2024-02-13T03:22:48.447Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.004265916s 2024-02-13T03:22:50.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[19769]: cluster 2024-02-13T03:22:48.573503+0000 mgr.y (mgr.24473) 72 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s; 27 B/s, 0 objects/s recovering 2024-02-13T03:22:50.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[19769]: audit 2024-02-13T03:22:49.126216+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:50.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[19769]: audit 2024-02-13T03:22:49.598374+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:50.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[19769]: audit 2024-02-13T03:22:49.896316+0000 mon.c (mon.2) 57 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:22:50.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[19769]: audit 2024-02-13T03:22:49.898219+0000 mon.c (mon.2) 58 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:22:50.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[19769]: audit 2024-02-13T03:22:49.908693+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:50.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[19769]: audit 2024-02-13T03:22:49.916422+0000 mon.c (mon.2) 59 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:22:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[27879]: cluster 2024-02-13T03:22:48.573503+0000 mgr.y (mgr.24473) 72 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s; 27 B/s, 0 objects/s recovering 2024-02-13T03:22:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[27879]: audit 2024-02-13T03:22:49.126216+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[27879]: audit 2024-02-13T03:22:49.598374+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[27879]: audit 2024-02-13T03:22:49.896316+0000 mon.c (mon.2) 57 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:22:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[27879]: audit 2024-02-13T03:22:49.898219+0000 mon.c (mon.2) 58 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:22:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[27879]: audit 2024-02-13T03:22:49.908693+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:50.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:50 smithi161 bash[27879]: audit 2024-02-13T03:22:49.916422+0000 mon.c (mon.2) 59 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:22:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:50 smithi183 bash[20824]: cluster 2024-02-13T03:22:48.573503+0000 mgr.y (mgr.24473) 72 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s; 27 B/s, 0 objects/s recovering 2024-02-13T03:22:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:50 smithi183 bash[20824]: audit 2024-02-13T03:22:49.126216+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:50 smithi183 bash[20824]: audit 2024-02-13T03:22:49.598374+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:50 smithi183 bash[20824]: audit 2024-02-13T03:22:49.896316+0000 mon.c (mon.2) 57 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:22:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:50 smithi183 bash[20824]: audit 2024-02-13T03:22:49.898219+0000 mon.c (mon.2) 58 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:22:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:50 smithi183 bash[20824]: audit 2024-02-13T03:22:49.908693+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:50 smithi183 bash[20824]: audit 2024-02-13T03:22:49.916422+0000 mon.c (mon.2) 59 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:22:51.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:22:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:22:51] "GET /metrics HTTP/1.1" 200 172146 "" "Prometheus/2.33.4" 2024-02-13T03:22:52.247 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:22:52.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:52 smithi161 bash[27879]: cluster 2024-02-13T03:22:50.574985+0000 mgr.y (mgr.24473) 73 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 3 op/s; 27 B/s, 0 objects/s recovering 2024-02-13T03:22:52.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:52 smithi161 bash[19769]: cluster 2024-02-13T03:22:50.574985+0000 mgr.y (mgr.24473) 73 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 3 op/s; 27 B/s, 0 objects/s recovering 2024-02-13T03:22:52.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:52 smithi183 bash[20824]: cluster 2024-02-13T03:22:50.574985+0000 mgr.y (mgr.24473) 73 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 3 op/s; 27 B/s, 0 objects/s recovering 2024-02-13T03:22:53.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:53 smithi161 bash[27879]: cluster 2024-02-13T03:22:52.575682+0000 mgr.y (mgr.24473) 74 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:53.478 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:53 smithi161 bash[19769]: cluster 2024-02-13T03:22:52.575682+0000 mgr.y (mgr.24473) 74 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:53 smithi183 bash[20824]: cluster 2024-02-13T03:22:52.575682+0000 mgr.y (mgr.24473) 74 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:54.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.976+0000 7fe3a09eb700 1 -- 172.21.15.161:0/3245004038 >> v1:172.21.15.161:6789/0 conn(0x7fe39c0ff810 legacy=0x7fe39c101c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:54.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.976+0000 7fe3a09eb700 1 -- 172.21.15.161:0/3245004038 shutdown_connections 2024-02-13T03:22:54.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.976+0000 7fe3a09eb700 1 -- 172.21.15.161:0/3245004038 >> 172.21.15.161:0/3245004038 conn(0x7fe39c0f2610 msgr2=0x7fe39c0f4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:54.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.976+0000 7fe3a09eb700 1 -- 172.21.15.161:0/3245004038 shutdown_connections 2024-02-13T03:22:54.982 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.976+0000 7fe3a09eb700 1 -- 172.21.15.161:0/3245004038 wait complete. 2024-02-13T03:22:54.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.976+0000 7fe3a09eb700 1 Processor -- start 2024-02-13T03:22:54.984 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.976+0000 7fe3a09eb700 1 -- start start 2024-02-13T03:22:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.976+0000 7fe3a09eb700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe39c1194e0 con 0x7fe39c0ff810 2024-02-13T03:22:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.976+0000 7fe3a09eb700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe39c11a680 con 0x7fe39c103320 2024-02-13T03:22:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.976+0000 7fe3a09eb700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe39c11b820 con 0x7fe39c0fca30 2024-02-13T03:22:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe39a59c700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7fe39c0fca30 0x7fe39c102370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:44784/0 (socket says 172.21.15.161:44784) 2024-02-13T03:22:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe39a59c700 1 -- 172.21.15.161:0/2460937241 learned_addr learned my addr 172.21.15.161:0/2460937241 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:22:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964044985 0 0) 0x7fe39c1194e0 con 0x7fe39c0ff810 2024-02-13T03:22:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe388003110 con 0x7fe39c0ff810 2024-02-13T03:22:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559105980 0 0) 0x7fe39c11b820 con 0x7fe39c0fca30 2024-02-13T03:22:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39c1194e0 con 0x7fe39c0fca30 2024-02-13T03:22:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832370053 0 0) 0x7fe39c11a680 con 0x7fe39c103320 2024-02-13T03:22:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39c11b820 con 0x7fe39c103320 2024-02-13T03:22:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1756115568 0 0) 0x7fe388003110 con 0x7fe39c0ff810 2024-02-13T03:22:54.985 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe39c11a680 con 0x7fe39c0ff810 2024-02-13T03:22:54.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fe384000fe0 con 0x7fe39c0ff810 2024-02-13T03:22:54.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4100909652 0 0) 0x7fe39c1194e0 con 0x7fe39c0fca30 2024-02-13T03:22:54.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe388003110 con 0x7fe39c0fca30 2024-02-13T03:22:54.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fe38c001a70 con 0x7fe39c0fca30 2024-02-13T03:22:54.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 551443928 0 0) 0x7fe39c11a680 con 0x7fe39c0ff810 2024-02-13T03:22:54.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 >> v1:172.21.15.161:6790/0 conn(0x7fe39c0fca30 legacy=0x7fe39c102370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:54.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 >> v1:172.21.15.183:6789/0 conn(0x7fe39c103320 legacy=0x7fe39c117dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:54.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c11c9c0 con 0x7fe39c0ff810 2024-02-13T03:22:54.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3a09eb700 1 -- 172.21.15.161:0/2460937241 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39c11ba10 con 0x7fe39c0ff810 2024-02-13T03:22:54.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3a09eb700 1 -- 172.21.15.161:0/2460937241 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39c11bf00 con 0x7fe39c0ff810 2024-02-13T03:22:54.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fe384002e70 con 0x7fe39c0ff810 2024-02-13T03:22:54.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fe3840035b0 con 0x7fe39c0ff810 2024-02-13T03:22:54.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3a09eb700 1 -- 172.21.15.161:0/2460937241 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37c004f20 con 0x7fe39c0ff810 2024-02-13T03:22:54.987 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.980+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fe384016560 con 0x7fe39c0ff810 2024-02-13T03:22:54.988 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.984+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7fe384076020 con 0x7fe39c0ff810 2024-02-13T03:22:54.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:54.988+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fe384049b90 con 0x7fe39c0ff810 2024-02-13T03:22:55.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:55.180+0000 7fe3a09eb700 1 -- 172.21.15.161:0/2460937241 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"} v 0) v1 -- 0x7fe37c005750 con 0x7fe39c0ff810 2024-02-13T03:22:55.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:55.180+0000 7fe3937fe700 1 -- 172.21.15.161:0/2460937241 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]=0 v63) v1 ==== 105+0+11 (unknown 1671938237 0 3519293104) 0x7fe384048000 con 0x7fe39c0ff810 2024-02-13T03:22:55.185 INFO:teuthology.orchestra.run.smithi161.stdout:pg_num: 32 2024-02-13T03:22:55.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:55.184+0000 7fe3a09eb700 1 -- 172.21.15.161:0/2460937241 >> v1:172.21.15.161:6800/4114337139 conn(0x7fe388062090 legacy=0x7fe388064550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:55.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:55.184+0000 7fe3a09eb700 1 -- 172.21.15.161:0/2460937241 >> v1:172.21.15.161:6789/0 conn(0x7fe39c0ff810 legacy=0x7fe39c102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:22:55.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:55.184+0000 7fe3a09eb700 1 -- 172.21.15.161:0/2460937241 shutdown_connections 2024-02-13T03:22:55.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:55.184+0000 7fe3a09eb700 1 -- 172.21.15.161:0/2460937241 >> 172.21.15.161:0/2460937241 conn(0x7fe39c0f2610 msgr2=0x7fe39c0f5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:22:55.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:55.184+0000 7fe3a09eb700 1 -- 172.21.15.161:0/2460937241 shutdown_connections 2024-02-13T03:22:55.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:22:55.184+0000 7fe3a09eb700 1 -- 172.21.15.161:0/2460937241 wait complete. 2024-02-13T03:22:55.857 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-13T03:22:55.857 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-13T03:22:55.857 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph mgr dump --format=json 2024-02-13T03:22:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:55 smithi161 bash[27879]: cluster 2024-02-13T03:22:54.576200+0000 mgr.y (mgr.24473) 75 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:55 smithi161 bash[27879]: audit 2024-02-13T03:22:55.183126+0000 mon.a (mon.0) 676 : audit [DBG] from='client.? v1:172.21.15.161:0/2460937241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-13T03:22:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:55 smithi161 bash[27879]: audit 2024-02-13T03:22:55.441859+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:55 smithi161 bash[19769]: cluster 2024-02-13T03:22:54.576200+0000 mgr.y (mgr.24473) 75 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:55.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:55 smithi161 bash[19769]: audit 2024-02-13T03:22:55.183126+0000 mon.a (mon.0) 676 : audit [DBG] from='client.? v1:172.21.15.161:0/2460937241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-13T03:22:55.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:55 smithi161 bash[19769]: audit 2024-02-13T03:22:55.441859+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:55.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:22:55 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:22:55.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:55 smithi183 bash[20824]: cluster 2024-02-13T03:22:54.576200+0000 mgr.y (mgr.24473) 75 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:55.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:55 smithi183 bash[20824]: audit 2024-02-13T03:22:55.183126+0000 mon.a (mon.0) 676 : audit [DBG] from='client.? v1:172.21.15.161:0/2460937241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-13T03:22:55.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:55 smithi183 bash[20824]: audit 2024-02-13T03:22:55.441859+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:57 smithi183 bash[20824]: audit 2024-02-13T03:22:55.603527+0000 mgr.y (mgr.24473) 76 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:57 smithi183 bash[20824]: audit 2024-02-13T03:22:56.221739+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:57 smithi183 bash[20824]: audit 2024-02-13T03:22:56.223616+0000 mon.c (mon.2) 60 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:22:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:57 smithi183 bash[20824]: audit 2024-02-13T03:22:56.225171+0000 mon.c (mon.2) 61 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:22:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:57 smithi183 bash[20824]: audit 2024-02-13T03:22:56.234195+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:57 smithi161 bash[19769]: audit 2024-02-13T03:22:55.603527+0000 mgr.y (mgr.24473) 76 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:57 smithi161 bash[19769]: audit 2024-02-13T03:22:56.221739+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:57 smithi161 bash[19769]: audit 2024-02-13T03:22:56.223616+0000 mon.c (mon.2) 60 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:22:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:57 smithi161 bash[19769]: audit 2024-02-13T03:22:56.225171+0000 mon.c (mon.2) 61 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:22:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:57 smithi161 bash[19769]: audit 2024-02-13T03:22:56.234195+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:57 smithi161 bash[27879]: audit 2024-02-13T03:22:55.603527+0000 mgr.y (mgr.24473) 76 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:22:57.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:57 smithi161 bash[27879]: audit 2024-02-13T03:22:56.221739+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:57.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:57 smithi161 bash[27879]: audit 2024-02-13T03:22:56.223616+0000 mon.c (mon.2) 60 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:22:57.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:57 smithi161 bash[27879]: audit 2024-02-13T03:22:56.225171+0000 mon.c (mon.2) 61 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:22:57.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:57 smithi161 bash[27879]: audit 2024-02-13T03:22:56.234195+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:22:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:58 smithi183 bash[20824]: cluster 2024-02-13T03:22:56.577661+0000 mgr.y (mgr.24473) 77 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:58 smithi161 bash[27879]: cluster 2024-02-13T03:22:56.577661+0000 mgr.y (mgr.24473) 77 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:58 smithi161 bash[19769]: cluster 2024-02-13T03:22:56.577661+0000 mgr.y (mgr.24473) 77 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:22:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:22:59 smithi161 bash[19769]: cluster 2024-02-13T03:22:58.578348+0000 mgr.y (mgr.24473) 78 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:22:59 smithi161 bash[27879]: cluster 2024-02-13T03:22:58.578348+0000 mgr.y (mgr.24473) 78 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:22:59 smithi183 bash[20824]: cluster 2024-02-13T03:22:58.578348+0000 mgr.y (mgr.24473) 78 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:22:59.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:22:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:22:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:23:00.696 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:23:01.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:23:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:23:01] "GET /metrics HTTP/1.1" 200 172146 "" "Prometheus/2.33.4" 2024-02-13T03:23:01.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:01 smithi161 bash[27879]: cluster 2024-02-13T03:23:00.579829+0000 mgr.y (mgr.24473) 79 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:01.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:01 smithi161 bash[19769]: cluster 2024-02-13T03:23:00.579829+0000 mgr.y (mgr.24473) 79 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:01.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:01 smithi183 bash[20824]: cluster 2024-02-13T03:23:00.579829+0000 mgr.y (mgr.24473) 79 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:03.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.512+0000 7f1254484700 1 -- 172.21.15.161:0/615378629 >> v1:172.21.15.161:6789/0 conn(0x7f124c06b210 legacy=0x7f124c06b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:03.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.512+0000 7f1254484700 1 -- 172.21.15.161:0/615378629 shutdown_connections 2024-02-13T03:23:03.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.512+0000 7f1254484700 1 -- 172.21.15.161:0/615378629 >> 172.21.15.161:0/615378629 conn(0x7f124c0f36d0 msgr2=0x7f124c0f5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:03.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.512+0000 7f1254484700 1 -- 172.21.15.161:0/615378629 shutdown_connections 2024-02-13T03:23:03.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.512+0000 7f1254484700 1 -- 172.21.15.161:0/615378629 wait complete. 2024-02-13T03:23:03.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f1254484700 1 Processor -- start 2024-02-13T03:23:03.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f1254484700 1 -- start start 2024-02-13T03:23:03.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f1254484700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f124c119620 con 0x7f124c06bd00 2024-02-13T03:23:03.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f1254484700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f124c11a800 con 0x7f124c06b210 2024-02-13T03:23:03.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f1254484700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f124c11ba00 con 0x7f124c1066e0 2024-02-13T03:23:03.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f1252a21700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f124c1066e0 0x7f124c117ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:59798/0 (socket says 172.21.15.161:59798) 2024-02-13T03:23:03.520 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f1252a21700 1 -- 172.21.15.161:0/890376002 learned_addr learned my addr 172.21.15.161:0/890376002 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:23:03.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683754180 0 0) 0x7f124c11ba00 con 0x7f124c1066e0 2024-02-13T03:23:03.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1238003630 con 0x7f124c1066e0 2024-02-13T03:23:03.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033837222 0 0) 0x7f124c11a800 con 0x7f124c06b210 2024-02-13T03:23:03.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f124c11ba00 con 0x7f124c06b210 2024-02-13T03:23:03.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750263972 0 0) 0x7f124c119620 con 0x7f124c06bd00 2024-02-13T03:23:03.522 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f124c11a800 con 0x7f124c06bd00 2024-02-13T03:23:03.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1171208713 0 0) 0x7f124c11a800 con 0x7f124c06bd00 2024-02-13T03:23:03.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f124c119620 con 0x7f124c06bd00 2024-02-13T03:23:03.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f12480017c0 con 0x7f124c06bd00 2024-02-13T03:23:03.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 735200098 0 0) 0x7f1238003630 con 0x7f124c1066e0 2024-02-13T03:23:03.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f124c11a800 con 0x7f124c1066e0 2024-02-13T03:23:03.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f1244001c40 con 0x7f124c1066e0 2024-02-13T03:23:03.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1903952537 0 0) 0x7f124c11ba00 con 0x7f124c06b210 2024-02-13T03:23:03.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1238003630 con 0x7f124c06b210 2024-02-13T03:23:03.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f1240002cd0 con 0x7f124c06b210 2024-02-13T03:23:03.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2567471953 0 0) 0x7f124c119620 con 0x7f124c06bd00 2024-02-13T03:23:03.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 >> v1:172.21.15.161:6790/0 conn(0x7f124c1066e0 legacy=0x7f124c117ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:03.523 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 >> v1:172.21.15.183:6789/0 conn(0x7f124c06b210 legacy=0x7f124c101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:03.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c11cc00 con 0x7f124c06bd00 2024-02-13T03:23:03.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f1254484700 1 -- 172.21.15.161:0/890376002 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c11a9f0 con 0x7f124c06bd00 2024-02-13T03:23:03.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f1254484700 1 -- 172.21.15.161:0/890376002 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c11ae60 con 0x7f124c06bd00 2024-02-13T03:23:03.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f1248000fe0 con 0x7f124c06bd00 2024-02-13T03:23:03.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.516+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f1248003150 con 0x7f124c06bd00 2024-02-13T03:23:03.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.520+0000 7f1254484700 1 -- 172.21.15.161:0/890376002 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1230004f20 con 0x7f124c06bd00 2024-02-13T03:23:03.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.520+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f1248016060 con 0x7f124c06bd00 2024-02-13T03:23:03.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.520+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f1248075bd0 con 0x7f124c06bd00 2024-02-13T03:23:03.532 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.528+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f1248049740 con 0x7f124c06bd00 2024-02-13T03:23:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:03 smithi161 bash[27879]: cluster 2024-02-13T03:23:02.580574+0000 mgr.y (mgr.24473) 80 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:03 smithi161 bash[19769]: cluster 2024-02-13T03:23:02.580574+0000 mgr.y (mgr.24473) 80 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:03 smithi183 bash[20824]: cluster 2024-02-13T03:23:02.580574+0000 mgr.y (mgr.24473) 80 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:03.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.760+0000 7f1254484700 1 -- 172.21.15.161:0/890376002 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f12300051d0 con 0x7f124c06bd00 2024-02-13T03:23:03.773 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.768+0000 7f123f7fe700 1 -- 172.21.15.161:0/890376002 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+151943 (unknown 3383617513 0 1845932902) 0x7f1248047b50 con 0x7f124c06bd00 2024-02-13T03:23:03.776 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-02-13T03:23:03.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.772+0000 7f1254484700 1 -- 172.21.15.161:0/890376002 >> v1:172.21.15.161:6800/4114337139 conn(0x7f1238062310 legacy=0x7f12380647d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:03.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.772+0000 7f1254484700 1 -- 172.21.15.161:0/890376002 >> v1:172.21.15.161:6789/0 conn(0x7f124c06bd00 legacy=0x7f124c101fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:03.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.772+0000 7f1254484700 1 -- 172.21.15.161:0/890376002 shutdown_connections 2024-02-13T03:23:03.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.772+0000 7f1254484700 1 -- 172.21.15.161:0/890376002 >> 172.21.15.161:0/890376002 conn(0x7f124c0f36d0 msgr2=0x7f124c109ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:03.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.772+0000 7f1254484700 1 -- 172.21.15.161:0/890376002 shutdown_connections 2024-02-13T03:23:03.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:03.772+0000 7f1254484700 1 -- 172.21.15.161:0/890376002 wait complete. 2024-02-13T03:23:04.421 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":19,"active_gid":24473,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6800","nonce":4114337139}]},"active_addr":"172.21.15.161:6800/4114337139","active_change":"2024-02-13T03:21:34.516636+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":24466,"name":"x","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.161:8443/","prometheus":"http://172.21.15.161:9283/"},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":61,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":2041173255}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":3033497004}]}]} 2024-02-13T03:23:04.425 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-13T03:23:04.425 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-13T03:23:04.425 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-13T03:23:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:04 smithi161 bash[27879]: audit 2024-02-13T03:23:03.763945+0000 mon.a (mon.0) 680 : audit [DBG] from='client.? v1:172.21.15.161:0/890376002' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-13T03:23:04.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:04 smithi161 bash[19769]: audit 2024-02-13T03:23:03.763945+0000 mon.a (mon.0) 680 : audit [DBG] from='client.? v1:172.21.15.161:0/890376002' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-13T03:23:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:04 smithi183 bash[20824]: audit 2024-02-13T03:23:03.763945+0000 mon.a (mon.0) 680 : audit [DBG] from='client.? v1:172.21.15.161:0/890376002' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-13T03:23:05.610 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:05 smithi183 bash[20824]: cluster 2024-02-13T03:23:04.581250+0000 mgr.y (mgr.24473) 81 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:05 smithi161 bash[27879]: cluster 2024-02-13T03:23:04.581250+0000 mgr.y (mgr.24473) 81 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:05 smithi161 bash[19769]: cluster 2024-02-13T03:23:04.581250+0000 mgr.y (mgr.24473) 81 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:05.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:23:05 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:23:06.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:06 smithi161 bash[27879]: audit 2024-02-13T03:23:05.607466+0000 mgr.y (mgr.24473) 82 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:06.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:06 smithi161 bash[19769]: audit 2024-02-13T03:23:05.607466+0000 mgr.y (mgr.24473) 82 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:06.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:06 smithi183 bash[20824]: audit 2024-02-13T03:23:05.607466+0000 mgr.y (mgr.24473) 82 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:07 smithi161 bash[19769]: cluster 2024-02-13T03:23:06.582715+0000 mgr.y (mgr.24473) 83 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:07 smithi161 bash[27879]: cluster 2024-02-13T03:23:06.582715+0000 mgr.y (mgr.24473) 83 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:07.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:07 smithi183 bash[20824]: cluster 2024-02-13T03:23:06.582715+0000 mgr.y (mgr.24473) 83 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:09.255 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:23:09.647 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:23:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:23:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:23:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:09 smithi161 bash[27879]: cluster 2024-02-13T03:23:08.583408+0000 mgr.y (mgr.24473) 84 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:09.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:09 smithi161 bash[19769]: cluster 2024-02-13T03:23:08.583408+0000 mgr.y (mgr.24473) 84 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:09.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:09 smithi183 bash[20824]: cluster 2024-02-13T03:23:08.583408+0000 mgr.y (mgr.24473) 84 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:11.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:23:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:23:11] "GET /metrics HTTP/1.1" 200 172143 "" "Prometheus/2.33.4" 2024-02-13T03:23:11.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:11 smithi161 bash[27879]: cluster 2024-02-13T03:23:10.584447+0000 mgr.y (mgr.24473) 85 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:11.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:11 smithi161 bash[19769]: cluster 2024-02-13T03:23:10.584447+0000 mgr.y (mgr.24473) 85 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:11.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:11 smithi183 bash[20824]: cluster 2024-02-13T03:23:10.584447+0000 mgr.y (mgr.24473) 85 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:12.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f6416ad9700 1 -- 172.21.15.161:0/3876230867 >> v1:172.21.15.161:6789/0 conn(0x7f6410104450 legacy=0x7f64101068f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:12.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f6416ad9700 1 -- 172.21.15.161:0/3876230867 shutdown_connections 2024-02-13T03:23:12.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f6416ad9700 1 -- 172.21.15.161:0/3876230867 >> 172.21.15.161:0/3876230867 conn(0x7f64100f2610 msgr2=0x7f64100f4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:12.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f6416ad9700 1 -- 172.21.15.161:0/3876230867 shutdown_connections 2024-02-13T03:23:12.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f6416ad9700 1 -- 172.21.15.161:0/3876230867 wait complete. 2024-02-13T03:23:12.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f6416ad9700 1 Processor -- start 2024-02-13T03:23:12.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f6416ad9700 1 -- start start 2024-02-13T03:23:12.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f6416ad9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6410119610 con 0x7f64100f96d0 2024-02-13T03:23:12.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f6416ad9700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f641011a7f0 con 0x7f6410104450 2024-02-13T03:23:12.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f6416ad9700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f641011b9f0 con 0x7f64100f8b20 2024-02-13T03:23:12.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f6414875700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f64100f8b20 0x7f6410101970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:58494/0 (socket says 172.21.15.161:58494) 2024-02-13T03:23:12.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f6414875700 1 -- 172.21.15.161:0/251310163 learned_addr learned my addr 172.21.15.161:0/251310163 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:23:12.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051074959 0 0) 0x7f641011b9f0 con 0x7f64100f8b20 2024-02-13T03:23:12.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63f80033f0 con 0x7f64100f8b20 2024-02-13T03:23:12.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281034789 0 0) 0x7f6410119610 con 0x7f64100f96d0 2024-02-13T03:23:12.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f641011b9f0 con 0x7f64100f96d0 2024-02-13T03:23:12.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814432053 0 0) 0x7f641011a7f0 con 0x7f6410104450 2024-02-13T03:23:12.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.139+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6410119610 con 0x7f6410104450 2024-02-13T03:23:12.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 484440850 0 0) 0x7f63f80033f0 con 0x7f64100f8b20 2024-02-13T03:23:12.147 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f641011a7f0 con 0x7f64100f8b20 2024-02-13T03:23:12.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f6404002a10 con 0x7f64100f8b20 2024-02-13T03:23:12.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 558756427 0 0) 0x7f641011b9f0 con 0x7f64100f96d0 2024-02-13T03:23:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63f80033f0 con 0x7f64100f96d0 2024-02-13T03:23:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f6400001760 con 0x7f64100f96d0 2024-02-13T03:23:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 509974344 0 0) 0x7f641011a7f0 con 0x7f64100f8b20 2024-02-13T03:23:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 >> v1:172.21.15.183:6789/0 conn(0x7f6410104450 legacy=0x7f6410117cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 >> v1:172.21.15.161:6789/0 conn(0x7f64100f96d0 legacy=0x7f6410102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641011cbf0 con 0x7f64100f8b20 2024-02-13T03:23:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f6416ad9700 1 -- 172.21.15.161:0/251310163 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6410119800 con 0x7f64100f8b20 2024-02-13T03:23:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f6416ad9700 1 -- 172.21.15.161:0/251310163 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6410119c70 con 0x7f64100f8b20 2024-02-13T03:23:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f6404002cd0 con 0x7f64100f8b20 2024-02-13T03:23:12.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f6404005b50 con 0x7f64100f8b20 2024-02-13T03:23:12.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f6404001720 con 0x7f64100f8b20 2024-02-13T03:23:12.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f6416ad9700 1 -- 172.21.15.161:0/251310163 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63fc004f20 con 0x7f64100f8b20 2024-02-13T03:23:12.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.143+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f64040786e0 con 0x7f64100f8b20 2024-02-13T03:23:12.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.151+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f640404c250 con 0x7f64100f8b20 2024-02-13T03:23:12.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.335+0000 7f6416ad9700 1 -- 172.21.15.161:0/251310163 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63fc0051d0 con 0x7f64100f8b20 2024-02-13T03:23:12.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.339+0000 7f640df76700 1 -- 172.21.15.161:0/251310163 <== mon.2 v1:172.21.15.161:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+19197 (unknown 2546300363 0 1770458764) 0x7f640404a660 con 0x7f64100f8b20 2024-02-13T03:23:12.344 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-02-13T03:23:12.344 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":63,"fsid":"b9b3ef98-ca1d-11ee-95b9-87774f69a715","created":"2024-02-13T03:13:05.822934+0000","modified":"2024-02-13T03:22:35.980127+0000","last_up_change":"2024-02-13T03:20:54.783700+0000","last_in_change":"2024-02-13T03:20:29.453920+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-13T03:17:31.445899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-13T03:21:07.091767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-13T03:21:08.103932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-13T03:21:10.382753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-13T03:21:12.105682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-13T03:21:14.239505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"cf603e39-02f0-4508-b426-6fdd8402f1b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6801","nonce":2012328581}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6802","nonce":2012328581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6804","nonce":2012328581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6803","nonce":2012328581}]},"public_addr":"172.21.15.161:6801/2012328581","cluster_addr":"172.21.15.161:6802/2012328581","heartbeat_back_addr":"172.21.15.161:6804/2012328581","heartbeat_front_addr":"172.21.15.161:6803/2012328581","state":["exists","up"]},{"osd":1,"uuid":"3bda8b4b-0bd5-47f1-a699-93a180d0a278","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6805","nonce":72291598}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6806","nonce":72291598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6808","nonce":72291598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6807","nonce":72291598}]},"public_addr":"172.21.15.161:6805/72291598","cluster_addr":"172.21.15.161:6806/72291598","heartbeat_back_addr":"172.21.15.161:6808/72291598","heartbeat_front_addr":"172.21.15.161:6807/72291598","state":["exists","up"]},{"osd":2,"uuid":"c521de51-8f7c-4ec2-94b7-96ca43b52d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6809","nonce":1657207182}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6810","nonce":1657207182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6812","nonce":1657207182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6811","nonce":1657207182}]},"public_addr":"172.21.15.161:6809/1657207182","cluster_addr":"172.21.15.161:6810/1657207182","heartbeat_back_addr":"172.21.15.161:6812/1657207182","heartbeat_front_addr":"172.21.15.161:6811/1657207182","state":["exists","up"]},{"osd":3,"uuid":"871101d1-6af5-47da-9d56-085de6869077","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6813","nonce":2788336304}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6814","nonce":2788336304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6816","nonce":2788336304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6815","nonce":2788336304}]},"public_addr":"172.21.15.161:6813/2788336304","cluster_addr":"172.21.15.161:6814/2788336304","heartbeat_back_addr":"172.21.15.161:6816/2788336304","heartbeat_front_addr":"172.21.15.161:6815/2788336304","state":["exists","up"]},{"osd":4,"uuid":"02e5c987-89d4-490f-b6cb-be43e286cb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6800","nonce":384629816}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6801","nonce":384629816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6803","nonce":384629816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6802","nonce":384629816}]},"public_addr":"172.21.15.183:6800/384629816","cluster_addr":"172.21.15.183:6801/384629816","heartbeat_back_addr":"172.21.15.183:6803/384629816","heartbeat_front_addr":"172.21.15.183:6802/384629816","state":["exists","up"]},{"osd":5,"uuid":"68fc8d37-f86b-4f2e-8cfe-5e47161d27c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6804","nonce":2958722347}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6805","nonce":2958722347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6807","nonce":2958722347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6806","nonce":2958722347}]},"public_addr":"172.21.15.183:6804/2958722347","cluster_addr":"172.21.15.183:6805/2958722347","heartbeat_back_addr":"172.21.15.183:6807/2958722347","heartbeat_front_addr":"172.21.15.183:6806/2958722347","state":["exists","up"]},{"osd":6,"uuid":"aad4c7d8-2a9d-46f2-bf74-fb1bfa83a64c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6808","nonce":3716679598}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6809","nonce":3716679598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6811","nonce":3716679598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6810","nonce":3716679598}]},"public_addr":"172.21.15.183:6808/3716679598","cluster_addr":"172.21.15.183:6809/3716679598","heartbeat_back_addr":"172.21.15.183:6811/3716679598","heartbeat_front_addr":"172.21.15.183:6810/3716679598","state":["exists","up"]},{"osd":7,"uuid":"4dc175e5-44c6-4454-81e3-92a465398a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6812","nonce":1821141317}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6813","nonce":1821141317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6815","nonce":1821141317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6814","nonce":1821141317}]},"public_addr":"172.21.15.183:6812/1821141317","cluster_addr":"172.21.15.183:6813/1821141317","heartbeat_back_addr":"172.21.15.183:6815/1821141317","heartbeat_front_addr":"172.21.15.183:6814/1821141317","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:16:07.873972+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:16:47.940673+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:17:28.981004+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:18:10.307597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:18:49.391774+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:19:29.925899+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:20:11.948394+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:20:52.997527+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/2234015705":"2024-02-14T03:21:34.516409+0000","172.21.15.161:6800/1187535987":"2024-02-14T03:21:34.516409+0000","172.21.15.161:0/3385104551":"2024-02-14T03:14:02.911245+0000","172.21.15.161:0/2943775157":"2024-02-14T03:14:02.911245+0000","172.21.15.161:6800/2381271555":"2024-02-14T03:14:02.911245+0000","172.21.15.161:6800/1159674441":"2024-02-14T03:13:33.423519+0000","172.21.15.161:0/2907902373":"2024-02-14T03:21:34.516409+0000","172.21.15.161:0/3042112516":"2024-02-14T03:21:34.516409+0000","172.21.15.161:0/2926342541":"2024-02-14T03:13:33.423519+0000","172.21.15.161:0/869165781":"2024-02-14T03:13:33.423519+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-13T03:23:12.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.339+0000 7f6416ad9700 1 -- 172.21.15.161:0/251310163 >> v1:172.21.15.161:6800/4114337139 conn(0x7f63f8062300 legacy=0x7f63f80647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:12.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.339+0000 7f6416ad9700 1 -- 172.21.15.161:0/251310163 >> v1:172.21.15.161:6790/0 conn(0x7f64100f8b20 legacy=0x7f6410101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:12.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.339+0000 7f6416ad9700 1 -- 172.21.15.161:0/251310163 shutdown_connections 2024-02-13T03:23:12.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.339+0000 7f6416ad9700 1 -- 172.21.15.161:0/251310163 >> 172.21.15.161:0/251310163 conn(0x7f64100f2610 msgr2=0x7f64101078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:12.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.343+0000 7f6416ad9700 1 -- 172.21.15.161:0/251310163 shutdown_connections 2024-02-13T03:23:12.347 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:12.343+0000 7f6416ad9700 1 -- 172.21.15.161:0/251310163 wait complete. 2024-02-13T03:23:13.040 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-13T03:23:13.040 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-13T03:23:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:13 smithi161 bash[27879]: audit 2024-02-13T03:23:12.340106+0000 mon.c (mon.2) 62 : audit [DBG] from='client.? v1:172.21.15.161:0/251310163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T03:23:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:13 smithi161 bash[27879]: cluster 2024-02-13T03:23:12.585135+0000 mgr.y (mgr.24473) 86 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:13.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:13 smithi161 bash[19769]: audit 2024-02-13T03:23:12.340106+0000 mon.c (mon.2) 62 : audit [DBG] from='client.? v1:172.21.15.161:0/251310163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T03:23:13.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:13 smithi161 bash[19769]: cluster 2024-02-13T03:23:12.585135+0000 mgr.y (mgr.24473) 86 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:13 smithi183 bash[20824]: audit 2024-02-13T03:23:12.340106+0000 mon.c (mon.2) 62 : audit [DBG] from='client.? v1:172.21.15.161:0/251310163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T03:23:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:13 smithi183 bash[20824]: cluster 2024-02-13T03:23:12.585135+0000 mgr.y (mgr.24473) 86 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:15.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:15 smithi161 bash[27879]: cluster 2024-02-13T03:23:14.585870+0000 mgr.y (mgr.24473) 87 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:15.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:15 smithi161 bash[19769]: cluster 2024-02-13T03:23:14.585870+0000 mgr.y (mgr.24473) 87 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:15.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:15 smithi183 bash[20824]: cluster 2024-02-13T03:23:14.585870+0000 mgr.y (mgr.24473) 87 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:15.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:23:15 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:23:16.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:16 smithi161 bash[27879]: audit 2024-02-13T03:23:15.618701+0000 mgr.y (mgr.24473) 88 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:16 smithi161 bash[19769]: audit 2024-02-13T03:23:15.618701+0000 mgr.y (mgr.24473) 88 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:16.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:16 smithi183 bash[20824]: audit 2024-02-13T03:23:15.618701+0000 mgr.y (mgr.24473) 88 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:17.889 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:23:17.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:17 smithi161 bash[19769]: cluster 2024-02-13T03:23:16.587362+0000 mgr.y (mgr.24473) 89 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:17.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:17 smithi161 bash[27879]: cluster 2024-02-13T03:23:16.587362+0000 mgr.y (mgr.24473) 89 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:17.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:17 smithi183 bash[20824]: cluster 2024-02-13T03:23:16.587362+0000 mgr.y (mgr.24473) 89 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:19.662 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:23:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:23:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:23:19.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:19 smithi183 bash[20824]: cluster 2024-02-13T03:23:18.588144+0000 mgr.y (mgr.24473) 90 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:20.031 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:19 smithi161 bash[27879]: cluster 2024-02-13T03:23:18.588144+0000 mgr.y (mgr.24473) 90 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:20.031 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:19 smithi161 bash[19769]: cluster 2024-02-13T03:23:18.588144+0000 mgr.y (mgr.24473) 90 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:20.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.651+0000 7fbed0fff700 1 -- 172.21.15.161:0/852602668 >> v1:172.21.15.161:6790/0 conn(0x7fbecc104490 legacy=0x7fbecc106930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:20.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.651+0000 7fbed0fff700 1 -- 172.21.15.161:0/852602668 shutdown_connections 2024-02-13T03:23:20.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.651+0000 7fbed0fff700 1 -- 172.21.15.161:0/852602668 >> 172.21.15.161:0/852602668 conn(0x7fbecc0f2630 msgr2=0x7fbecc0f4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:20.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.651+0000 7fbed0fff700 1 -- 172.21.15.161:0/852602668 shutdown_connections 2024-02-13T03:23:20.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.651+0000 7fbed0fff700 1 -- 172.21.15.161:0/852602668 wait complete. 2024-02-13T03:23:20.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbed0fff700 1 Processor -- start 2024-02-13T03:23:20.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbed0fff700 1 -- start start 2024-02-13T03:23:20.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbed0fff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbecc10fa00 con 0x7fbecc104490 2024-02-13T03:23:20.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbed0fff700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbecc11ac10 con 0x7fbecc0f8b00 2024-02-13T03:23:20.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbed0fff700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbecc11be30 con 0x7fbecc0f96b0 2024-02-13T03:23:20.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbecad9d700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fbecc104490 0x7fbecc1184a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:50352/0 (socket says 172.21.15.161:50352) 2024-02-13T03:23:20.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbecad9d700 1 -- 172.21.15.161:0/744804028 learned_addr learned my addr 172.21.15.161:0/744804028 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:23:20.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec9d9b700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7fbecc0f96b0 0x7fbecc114d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:54724/0 (socket says 172.21.15.161:54724) 2024-02-13T03:23:20.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133673297 0 0) 0x7fbecc11be30 con 0x7fbecc0f96b0 2024-02-13T03:23:20.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeb80033f0 con 0x7fbecc0f96b0 2024-02-13T03:23:20.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926053178 0 0) 0x7fbecc11ac10 con 0x7fbecc0f8b00 2024-02-13T03:23:20.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbecc11be30 con 0x7fbecc0f8b00 2024-02-13T03:23:20.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891064826 0 0) 0x7fbecc10fa00 con 0x7fbecc104490 2024-02-13T03:23:20.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbecc11ac10 con 0x7fbecc104490 2024-02-13T03:23:20.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1057710061 0 0) 0x7fbeb80033f0 con 0x7fbecc0f96b0 2024-02-13T03:23:20.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbecc10fa00 con 0x7fbecc0f96b0 2024-02-13T03:23:20.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fbebc0018c0 con 0x7fbecc0f96b0 2024-02-13T03:23:20.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3328900428 0 0) 0x7fbecc11ac10 con 0x7fbecc104490 2024-02-13T03:23:20.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbeb80033f0 con 0x7fbecc104490 2024-02-13T03:23:20.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fbec40013f0 con 0x7fbecc104490 2024-02-13T03:23:20.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1464882996 0 0) 0x7fbeb80033f0 con 0x7fbecc104490 2024-02-13T03:23:20.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 >> v1:172.21.15.161:6790/0 conn(0x7fbecc0f96b0 legacy=0x7fbecc114d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:20.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 >> v1:172.21.15.183:6789/0 conn(0x7fbecc0f8b00 legacy=0x7fbecc10ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:20.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc11d030 con 0x7fbecc104490 2024-02-13T03:23:20.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbed0fff700 1 -- 172.21.15.161:0/744804028 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc11c020 con 0x7fbecc104490 2024-02-13T03:23:20.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fbec4003310 con 0x7fbecc104490 2024-02-13T03:23:20.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbed0fff700 1 -- 172.21.15.161:0/744804028 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc11c490 con 0x7fbecc104490 2024-02-13T03:23:20.663 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.655+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fbec4003c80 con 0x7fbecc104490 2024-02-13T03:23:20.665 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.659+0000 7fbed0fff700 1 -- 172.21.15.161:0/744804028 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeac004f20 con 0x7fbecc104490 2024-02-13T03:23:20.666 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.659+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fbec4001c00 con 0x7fbecc104490 2024-02-13T03:23:20.667 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.663+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7fbec4078a90 con 0x7fbecc104490 2024-02-13T03:23:20.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.667+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fbec404c6c0 con 0x7fbecc104490 2024-02-13T03:23:20.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.851+0000 7fbed0fff700 1 -- 172.21.15.161:0/744804028 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbeac0051d0 con 0x7fbecc104490 2024-02-13T03:23:20.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.855+0000 7fbec37fe700 1 -- 172.21.15.161:0/744804028 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+19197 (unknown 2546300363 0 1770458764) 0x7fbec404ab30 con 0x7fbecc104490 2024-02-13T03:23:20.860 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-02-13T03:23:20.861 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":63,"fsid":"b9b3ef98-ca1d-11ee-95b9-87774f69a715","created":"2024-02-13T03:13:05.822934+0000","modified":"2024-02-13T03:22:35.980127+0000","last_up_change":"2024-02-13T03:20:54.783700+0000","last_in_change":"2024-02-13T03:20:29.453920+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-13T03:17:31.445899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-13T03:21:07.091767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-13T03:21:08.103932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-13T03:21:10.382753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-13T03:21:12.105682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-13T03:21:14.239505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"cf603e39-02f0-4508-b426-6fdd8402f1b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6801","nonce":2012328581}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6802","nonce":2012328581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6804","nonce":2012328581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6803","nonce":2012328581}]},"public_addr":"172.21.15.161:6801/2012328581","cluster_addr":"172.21.15.161:6802/2012328581","heartbeat_back_addr":"172.21.15.161:6804/2012328581","heartbeat_front_addr":"172.21.15.161:6803/2012328581","state":["exists","up"]},{"osd":1,"uuid":"3bda8b4b-0bd5-47f1-a699-93a180d0a278","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6805","nonce":72291598}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6806","nonce":72291598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6808","nonce":72291598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6807","nonce":72291598}]},"public_addr":"172.21.15.161:6805/72291598","cluster_addr":"172.21.15.161:6806/72291598","heartbeat_back_addr":"172.21.15.161:6808/72291598","heartbeat_front_addr":"172.21.15.161:6807/72291598","state":["exists","up"]},{"osd":2,"uuid":"c521de51-8f7c-4ec2-94b7-96ca43b52d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6809","nonce":1657207182}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6810","nonce":1657207182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6812","nonce":1657207182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6811","nonce":1657207182}]},"public_addr":"172.21.15.161:6809/1657207182","cluster_addr":"172.21.15.161:6810/1657207182","heartbeat_back_addr":"172.21.15.161:6812/1657207182","heartbeat_front_addr":"172.21.15.161:6811/1657207182","state":["exists","up"]},{"osd":3,"uuid":"871101d1-6af5-47da-9d56-085de6869077","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6813","nonce":2788336304}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6814","nonce":2788336304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6816","nonce":2788336304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.161:6815","nonce":2788336304}]},"public_addr":"172.21.15.161:6813/2788336304","cluster_addr":"172.21.15.161:6814/2788336304","heartbeat_back_addr":"172.21.15.161:6816/2788336304","heartbeat_front_addr":"172.21.15.161:6815/2788336304","state":["exists","up"]},{"osd":4,"uuid":"02e5c987-89d4-490f-b6cb-be43e286cb30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6800","nonce":384629816}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6801","nonce":384629816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6803","nonce":384629816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6802","nonce":384629816}]},"public_addr":"172.21.15.183:6800/384629816","cluster_addr":"172.21.15.183:6801/384629816","heartbeat_back_addr":"172.21.15.183:6803/384629816","heartbeat_front_addr":"172.21.15.183:6802/384629816","state":["exists","up"]},{"osd":5,"uuid":"68fc8d37-f86b-4f2e-8cfe-5e47161d27c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6804","nonce":2958722347}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6805","nonce":2958722347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6807","nonce":2958722347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6806","nonce":2958722347}]},"public_addr":"172.21.15.183:6804/2958722347","cluster_addr":"172.21.15.183:6805/2958722347","heartbeat_back_addr":"172.21.15.183:6807/2958722347","heartbeat_front_addr":"172.21.15.183:6806/2958722347","state":["exists","up"]},{"osd":6,"uuid":"aad4c7d8-2a9d-46f2-bf74-fb1bfa83a64c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6808","nonce":3716679598}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6809","nonce":3716679598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6811","nonce":3716679598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6810","nonce":3716679598}]},"public_addr":"172.21.15.183:6808/3716679598","cluster_addr":"172.21.15.183:6809/3716679598","heartbeat_back_addr":"172.21.15.183:6811/3716679598","heartbeat_front_addr":"172.21.15.183:6810/3716679598","state":["exists","up"]},{"osd":7,"uuid":"4dc175e5-44c6-4454-81e3-92a465398a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6812","nonce":1821141317}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6813","nonce":1821141317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6815","nonce":1821141317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.183:6814","nonce":1821141317}]},"public_addr":"172.21.15.183:6812/1821141317","cluster_addr":"172.21.15.183:6813/1821141317","heartbeat_back_addr":"172.21.15.183:6815/1821141317","heartbeat_front_addr":"172.21.15.183:6814/1821141317","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:16:07.873972+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:16:47.940673+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:17:28.981004+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:18:10.307597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:18:49.391774+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:19:29.925899+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:20:11.948394+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T03:20:52.997527+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:0/2234015705":"2024-02-14T03:21:34.516409+0000","172.21.15.161:6800/1187535987":"2024-02-14T03:21:34.516409+0000","172.21.15.161:0/3385104551":"2024-02-14T03:14:02.911245+0000","172.21.15.161:0/2943775157":"2024-02-14T03:14:02.911245+0000","172.21.15.161:6800/2381271555":"2024-02-14T03:14:02.911245+0000","172.21.15.161:6800/1159674441":"2024-02-14T03:13:33.423519+0000","172.21.15.161:0/2907902373":"2024-02-14T03:21:34.516409+0000","172.21.15.161:0/3042112516":"2024-02-14T03:21:34.516409+0000","172.21.15.161:0/2926342541":"2024-02-14T03:13:33.423519+0000","172.21.15.161:0/869165781":"2024-02-14T03:13:33.423519+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-13T03:23:20.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.859+0000 7fbed0fff700 1 -- 172.21.15.161:0/744804028 >> v1:172.21.15.161:6800/4114337139 conn(0x7fbeb80622b0 legacy=0x7fbeb8064770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:20.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.859+0000 7fbed0fff700 1 -- 172.21.15.161:0/744804028 >> v1:172.21.15.161:6789/0 conn(0x7fbecc104490 legacy=0x7fbecc1184a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:20.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.859+0000 7fbed0fff700 1 -- 172.21.15.161:0/744804028 shutdown_connections 2024-02-13T03:23:20.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.859+0000 7fbed0fff700 1 -- 172.21.15.161:0/744804028 >> 172.21.15.161:0/744804028 conn(0x7fbecc0f2630 msgr2=0x7fbecc0f2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:20.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.859+0000 7fbed0fff700 1 -- 172.21.15.161:0/744804028 shutdown_connections 2024-02-13T03:23:20.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:20.859+0000 7fbed0fff700 1 -- 172.21.15.161:0/744804028 wait complete. 2024-02-13T03:23:21.177 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:23:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:23:21] "GET /metrics HTTP/1.1" 200 172159 "" "Prometheus/2.33.4" 2024-02-13T03:23:21.489 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-13T03:23:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:21 smithi161 bash[19769]: cluster 2024-02-13T03:23:20.589162+0000 mgr.y (mgr.24473) 91 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:21 smithi161 bash[19769]: audit 2024-02-13T03:23:20.856229+0000 mon.a (mon.0) 681 : audit [DBG] from='client.? v1:172.21.15.161:0/744804028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T03:23:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:21 smithi161 bash[27879]: cluster 2024-02-13T03:23:20.589162+0000 mgr.y (mgr.24473) 91 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:21.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:21 smithi161 bash[27879]: audit 2024-02-13T03:23:20.856229+0000 mon.a (mon.0) 681 : audit [DBG] from='client.? v1:172.21.15.161:0/744804028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T03:23:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:21 smithi183 bash[20824]: cluster 2024-02-13T03:23:20.589162+0000 mgr.y (mgr.24473) 91 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:21 smithi183 bash[20824]: audit 2024-02-13T03:23:20.856229+0000 mon.a (mon.0) 681 : audit [DBG] from='client.? v1:172.21.15.161:0/744804028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T03:23:23.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:23 smithi161 bash[19769]: cluster 2024-02-13T03:23:22.589954+0000 mgr.y (mgr.24473) 92 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:23 smithi161 bash[27879]: cluster 2024-02-13T03:23:22.589954+0000 mgr.y (mgr.24473) 92 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:23 smithi183 bash[20824]: cluster 2024-02-13T03:23:22.589954+0000 mgr.y (mgr.24473) 92 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:25 smithi161 bash[19769]: cluster 2024-02-13T03:23:24.590824+0000 mgr.y (mgr.24473) 93 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:25 smithi161 bash[27879]: cluster 2024-02-13T03:23:24.590824+0000 mgr.y (mgr.24473) 93 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:25 smithi183 bash[20824]: cluster 2024-02-13T03:23:24.590824+0000 mgr.y (mgr.24473) 93 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:25.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:23:25 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:23:26.332 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:23:26.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:26 smithi161 bash[19769]: audit 2024-02-13T03:23:25.624137+0000 mgr.y (mgr.24473) 94 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:26 smithi161 bash[27879]: audit 2024-02-13T03:23:25.624137+0000 mgr.y (mgr.24473) 94 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:26 smithi183 bash[20824]: audit 2024-02-13T03:23:25.624137+0000 mgr.y (mgr.24473) 94 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:27.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:27 smithi161 bash[19769]: cluster 2024-02-13T03:23:26.592320+0000 mgr.y (mgr.24473) 95 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:27.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:27 smithi161 bash[27879]: cluster 2024-02-13T03:23:26.592320+0000 mgr.y (mgr.24473) 95 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:27 smithi183 bash[20824]: cluster 2024-02-13T03:23:26.592320+0000 mgr.y (mgr.24473) 95 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:29.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed922c8700 1 -- 172.21.15.161:0/4037928986 >> v1:172.21.15.161:6790/0 conn(0x7fed8c103320 legacy=0x7fed8c1057c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:29.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed922c8700 1 -- 172.21.15.161:0/4037928986 shutdown_connections 2024-02-13T03:23:29.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed922c8700 1 -- 172.21.15.161:0/4037928986 >> 172.21.15.161:0/4037928986 conn(0x7fed8c0f2610 msgr2=0x7fed8c0f4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:29.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed922c8700 1 -- 172.21.15.161:0/4037928986 shutdown_connections 2024-02-13T03:23:29.065 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed922c8700 1 -- 172.21.15.161:0/4037928986 wait complete. 2024-02-13T03:23:29.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed922c8700 1 Processor -- start 2024-02-13T03:23:29.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed922c8700 1 -- start start 2024-02-13T03:23:29.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed922c8700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fed8c102e60 con 0x7fed8c103320 2024-02-13T03:23:29.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed922c8700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fed8c103030 con 0x7fed8c0ff810 2024-02-13T03:23:29.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed922c8700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fed8c11bc10 con 0x7fed8c0fca30 2024-02-13T03:23:29.066 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed8bfff700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7fed8c0fca30 0x7fed8c1026c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:35696/0 (socket says 172.21.15.161:35696) 2024-02-13T03:23:29.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed8bfff700 1 -- 172.21.15.161:0/955141737 learned_addr learned my addr 172.21.15.161:0/955141737 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:23:29.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657104880 0 0) 0x7fed8c11bc10 con 0x7fed8c0fca30 2024-02-13T03:23:29.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed740033f0 con 0x7fed8c0fca30 2024-02-13T03:23:29.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507661159 0 0) 0x7fed8c102e60 con 0x7fed8c103320 2024-02-13T03:23:29.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed8c11bc10 con 0x7fed8c103320 2024-02-13T03:23:29.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506006950 0 0) 0x7fed8c103030 con 0x7fed8c0ff810 2024-02-13T03:23:29.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed8c102e60 con 0x7fed8c0ff810 2024-02-13T03:23:29.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1310999539 0 0) 0x7fed740033f0 con 0x7fed8c0fca30 2024-02-13T03:23:29.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed8c103030 con 0x7fed8c0fca30 2024-02-13T03:23:29.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fed7c002d90 con 0x7fed8c0fca30 2024-02-13T03:23:29.067 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4224532421 0 0) 0x7fed8c11bc10 con 0x7fed8c103320 2024-02-13T03:23:29.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed740033f0 con 0x7fed8c103320 2024-02-13T03:23:29.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fed84001070 con 0x7fed8c103320 2024-02-13T03:23:29.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3864671913 0 0) 0x7fed8c103030 con 0x7fed8c0fca30 2024-02-13T03:23:29.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 >> v1:172.21.15.183:6789/0 conn(0x7fed8c0ff810 legacy=0x7fed8c114da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:29.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 >> v1:172.21.15.161:6789/0 conn(0x7fed8c103320 legacy=0x7fed8c1184d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:29.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c11cdb0 con 0x7fed8c0fca30 2024-02-13T03:23:29.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed922c8700 1 -- 172.21.15.161:0/955141737 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8c11be00 con 0x7fed8c0fca30 2024-02-13T03:23:29.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed922c8700 1 -- 172.21.15.161:0/955141737 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8c11c270 con 0x7fed8c0fca30 2024-02-13T03:23:29.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fed7c003120 con 0x7fed8c0fca30 2024-02-13T03:23:29.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.059+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fed7c005ed0 con 0x7fed8c0fca30 2024-02-13T03:23:29.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.063+0000 7fed922c8700 1 -- 172.21.15.161:0/955141737 --> v1:172.21.15.161:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fed78000f10 con 0x7fed8c0fca30 2024-02-13T03:23:29.069 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.063+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fed7c018d60 con 0x7fed8c0fca30 2024-02-13T03:23:29.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.063+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7fed7c078ae0 con 0x7fed8c0fca30 2024-02-13T03:23:29.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.063+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 --> v1:172.21.15.161:6801/2012328581 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed740704a0 con 0x7fed7406c790 2024-02-13T03:23:29.070 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.063+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (unknown 2499814743 0 0) 0x7fed7c078e60 con 0x7fed8c0fca30 2024-02-13T03:23:29.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.067+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== osd.0 v1:172.21.15.161:6801/2012328581 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7fed740704a0 con 0x7fed7406c790 2024-02-13T03:23:29.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.087+0000 7fed922c8700 1 -- 172.21.15.161:0/955141737 --> v1:172.21.15.161:6801/2012328581 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fed780029f0 con 0x7fed7406c790 2024-02-13T03:23:29.093 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.087+0000 7fed89775700 1 -- 172.21.15.161:0/955141737 <== osd.0 v1:172.21.15.161:6801/2012328581 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 723570492) 0x7fed780029f0 con 0x7fed7406c790 2024-02-13T03:23:29.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.087+0000 7fed922c8700 1 -- 172.21.15.161:0/955141737 >> v1:172.21.15.161:6801/2012328581 conn(0x7fed7406c790 legacy=0x7fed7406ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:29.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.087+0000 7fed922c8700 1 -- 172.21.15.161:0/955141737 >> v1:172.21.15.161:6800/4114337139 conn(0x7fed74061fd0 legacy=0x7fed74064490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:29.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.087+0000 7fed922c8700 1 -- 172.21.15.161:0/955141737 >> v1:172.21.15.161:6790/0 conn(0x7fed8c0fca30 legacy=0x7fed8c1026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:29.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.087+0000 7fed90865700 1 -- 172.21.15.161:0/955141737 reap_dead start 2024-02-13T03:23:29.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.087+0000 7fed922c8700 1 -- 172.21.15.161:0/955141737 shutdown_connections 2024-02-13T03:23:29.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.087+0000 7fed922c8700 1 -- 172.21.15.161:0/955141737 >> 172.21.15.161:0/955141737 conn(0x7fed8c0f2610 msgr2=0x7fed8c0f5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:29.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.087+0000 7fed922c8700 1 -- 172.21.15.161:0/955141737 shutdown_connections 2024-02-13T03:23:29.095 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:29.087+0000 7fed922c8700 1 -- 172.21.15.161:0/955141737 wait complete. 2024-02-13T03:23:29.671 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:23:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:23:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:23:29.726 INFO:teuthology.orchestra.run.smithi161.stdout:34359738457 2024-02-13T03:23:29.726 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-13T03:23:29.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:29 smithi161 bash[27879]: cluster 2024-02-13T03:23:28.593163+0000 mgr.y (mgr.24473) 96 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:29.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:29 smithi161 bash[19769]: cluster 2024-02-13T03:23:28.593163+0000 mgr.y (mgr.24473) 96 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:29.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:29 smithi183 bash[20824]: cluster 2024-02-13T03:23:28.593163+0000 mgr.y (mgr.24473) 96 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:31.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:23:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:23:31] "GET /metrics HTTP/1.1" 200 172159 "" "Prometheus/2.33.4" 2024-02-13T03:23:31.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:31 smithi161 bash[19769]: cluster 2024-02-13T03:23:30.594619+0000 mgr.y (mgr.24473) 97 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:31 smithi161 bash[27879]: cluster 2024-02-13T03:23:30.594619+0000 mgr.y (mgr.24473) 97 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:31 smithi183 bash[20824]: cluster 2024-02-13T03:23:30.594619+0000 mgr.y (mgr.24473) 97 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:33 smithi161 bash[27879]: cluster 2024-02-13T03:23:32.595369+0000 mgr.y (mgr.24473) 98 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:33.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:33 smithi161 bash[19769]: cluster 2024-02-13T03:23:32.595369+0000 mgr.y (mgr.24473) 98 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:33.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:33 smithi183 bash[20824]: cluster 2024-02-13T03:23:32.595369+0000 mgr.y (mgr.24473) 98 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:34.568 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:23:35.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:35 smithi161 bash[19769]: cluster 2024-02-13T03:23:34.596125+0000 mgr.y (mgr.24473) 99 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:35 smithi161 bash[27879]: cluster 2024-02-13T03:23:34.596125+0000 mgr.y (mgr.24473) 99 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:35 smithi183 bash[20824]: cluster 2024-02-13T03:23:34.596125+0000 mgr.y (mgr.24473) 99 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:35.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:23:35 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:23:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:36 smithi161 bash[19769]: audit 2024-02-13T03:23:35.633789+0000 mgr.y (mgr.24473) 100 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:36 smithi161 bash[27879]: audit 2024-02-13T03:23:35.633789+0000 mgr.y (mgr.24473) 100 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:36 smithi183 bash[20824]: audit 2024-02-13T03:23:35.633789+0000 mgr.y (mgr.24473) 100 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:37.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5c207e700 1 -- 172.21.15.161:0/1266609923 >> v1:172.21.15.161:6790/0 conn(0x7fa5bc06b190 legacy=0x7fa5bc06b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:37.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5c207e700 1 -- 172.21.15.161:0/1266609923 shutdown_connections 2024-02-13T03:23:37.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5c207e700 1 -- 172.21.15.161:0/1266609923 >> 172.21.15.161:0/1266609923 conn(0x7fa5bc0f36d0 msgr2=0x7fa5bc0f5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:37.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5c207e700 1 -- 172.21.15.161:0/1266609923 shutdown_connections 2024-02-13T03:23:37.213 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5c207e700 1 -- 172.21.15.161:0/1266609923 wait complete. 2024-02-13T03:23:37.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5c207e700 1 Processor -- start 2024-02-13T03:23:37.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5c207e700 1 -- start start 2024-02-13T03:23:37.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5c207e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa5bc119560 con 0x7fa5bc1066c0 2024-02-13T03:23:37.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5c207e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa5bc11a700 con 0x7fa5bc06bc80 2024-02-13T03:23:37.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5c207e700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa5bc11b900 con 0x7fa5bc06b190 2024-02-13T03:23:37.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5bb7fe700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7fa5bc06b190 0x7fa5bc102830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:60152/0 (socket says 172.21.15.161:60152) 2024-02-13T03:23:37.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5bb7fe700 1 -- 172.21.15.161:0/972016776 learned_addr learned my addr 172.21.15.161:0/972016776 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:23:37.215 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859583673 0 0) 0x7fa5bc11b900 con 0x7fa5bc06b190 2024-02-13T03:23:37.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5a4003030 con 0x7fa5bc06b190 2024-02-13T03:23:37.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294841581 0 0) 0x7fa5bc119560 con 0x7fa5bc1066c0 2024-02-13T03:23:37.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5bc11b900 con 0x7fa5bc1066c0 2024-02-13T03:23:37.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759585613 0 0) 0x7fa5bc11a700 con 0x7fa5bc06bc80 2024-02-13T03:23:37.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5bc119560 con 0x7fa5bc06bc80 2024-02-13T03:23:37.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.207+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2073064641 0 0) 0x7fa5a4003030 con 0x7fa5bc06b190 2024-02-13T03:23:37.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5bc11a700 con 0x7fa5bc06b190 2024-02-13T03:23:37.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fa5b0002f20 con 0x7fa5bc06b190 2024-02-13T03:23:37.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1669856959 0 0) 0x7fa5bc119560 con 0x7fa5bc06bc80 2024-02-13T03:23:37.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5a4003030 con 0x7fa5bc06bc80 2024-02-13T03:23:37.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2669842859 0 0) 0x7fa5bc11a700 con 0x7fa5bc06b190 2024-02-13T03:23:37.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 >> v1:172.21.15.183:6789/0 conn(0x7fa5bc06bc80 legacy=0x7fa5bc102f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:37.216 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 >> v1:172.21.15.161:6789/0 conn(0x7fa5bc1066c0 legacy=0x7fa5bc117ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:37.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc11cb00 con 0x7fa5bc06b190 2024-02-13T03:23:37.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5c207e700 1 -- 172.21.15.161:0/972016776 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5bc11a8f0 con 0x7fa5bc06b190 2024-02-13T03:23:37.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5c207e700 1 -- 172.21.15.161:0/972016776 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5bc11ad60 con 0x7fa5bc06b190 2024-02-13T03:23:37.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fa5b00024f0 con 0x7fa5bc06b190 2024-02-13T03:23:37.217 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fa5b0003980 con 0x7fa5bc06b190 2024-02-13T03:23:37.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5c207e700 1 -- 172.21.15.161:0/972016776 --> v1:172.21.15.161:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa5a8000f40 con 0x7fa5bc06b190 2024-02-13T03:23:37.218 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fa5b00031c0 con 0x7fa5bc06b190 2024-02-13T03:23:37.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7fa5b0078cb0 con 0x7fa5bc06b190 2024-02-13T03:23:37.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.211+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 --> v1:172.21.15.161:6805/72291598 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5a4070470 con 0x7fa5a406c800 2024-02-13T03:23:37.220 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.215+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (unknown 2499814743 0 0) 0x7fa5b0079030 con 0x7fa5bc06b190 2024-02-13T03:23:37.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.215+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== osd.1 v1:172.21.15.161:6805/72291598 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7fa5a4070470 con 0x7fa5a406c800 2024-02-13T03:23:37.240 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.235+0000 7fa5c207e700 1 -- 172.21.15.161:0/972016776 --> v1:172.21.15.161:6805/72291598 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa5a8002a20 con 0x7fa5a406c800 2024-02-13T03:23:37.241 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.235+0000 7fa5b8ff9700 1 -- 172.21.15.161:0/972016776 <== osd.1 v1:172.21.15.161:6805/72291598 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 74841285) 0x7fa5a8002a20 con 0x7fa5a406c800 2024-02-13T03:23:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.235+0000 7fa5c207e700 1 -- 172.21.15.161:0/972016776 >> v1:172.21.15.161:6805/72291598 conn(0x7fa5a406c800 legacy=0x7fa5a406eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.235+0000 7fa5c207e700 1 -- 172.21.15.161:0/972016776 >> v1:172.21.15.161:6800/4114337139 conn(0x7fa5a4061fb0 legacy=0x7fa5a4064470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.235+0000 7fa5c207e700 1 -- 172.21.15.161:0/972016776 >> v1:172.21.15.161:6790/0 conn(0x7fa5bc06b190 legacy=0x7fa5bc102830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.235+0000 7fa5bbfff700 1 -- 172.21.15.161:0/972016776 reap_dead start 2024-02-13T03:23:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.235+0000 7fa5c207e700 1 -- 172.21.15.161:0/972016776 shutdown_connections 2024-02-13T03:23:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.235+0000 7fa5c207e700 1 -- 172.21.15.161:0/972016776 >> 172.21.15.161:0/972016776 conn(0x7fa5bc0f36d0 msgr2=0x7fa5bc0f5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.235+0000 7fa5c207e700 1 -- 172.21.15.161:0/972016776 shutdown_connections 2024-02-13T03:23:37.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:37.235+0000 7fa5c207e700 1 -- 172.21.15.161:0/972016776 wait complete. 2024-02-13T03:23:37.852 INFO:teuthology.orchestra.run.smithi161.stdout:55834574930 2024-02-13T03:23:37.853 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-13T03:23:37.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:37 smithi161 bash[27879]: cluster 2024-02-13T03:23:36.597703+0000 mgr.y (mgr.24473) 101 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:37.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:37 smithi161 bash[19769]: cluster 2024-02-13T03:23:36.597703+0000 mgr.y (mgr.24473) 101 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:37 smithi183 bash[20824]: cluster 2024-02-13T03:23:36.597703+0000 mgr.y (mgr.24473) 101 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:39.675 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:23:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:23:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:23:39.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:39 smithi161 bash[19769]: cluster 2024-02-13T03:23:38.598515+0000 mgr.y (mgr.24473) 102 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:39.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:39 smithi161 bash[27879]: cluster 2024-02-13T03:23:38.598515+0000 mgr.y (mgr.24473) 102 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:39.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:39 smithi183 bash[20824]: cluster 2024-02-13T03:23:38.598515+0000 mgr.y (mgr.24473) 102 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:41.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:23:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:23:41] "GET /metrics HTTP/1.1" 200 172147 "" "Prometheus/2.33.4" 2024-02-13T03:23:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:41 smithi183 bash[20824]: cluster 2024-02-13T03:23:40.599856+0000 mgr.y (mgr.24473) 103 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:42.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:41 smithi161 bash[19769]: cluster 2024-02-13T03:23:40.599856+0000 mgr.y (mgr.24473) 103 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:42.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:41 smithi161 bash[27879]: cluster 2024-02-13T03:23:40.599856+0000 mgr.y (mgr.24473) 103 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:42.682 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:23:43.566 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:43 smithi161 bash[19769]: cluster 2024-02-13T03:23:42.600580+0000 mgr.y (mgr.24473) 104 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:43.566 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:43 smithi161 bash[27879]: cluster 2024-02-13T03:23:42.600580+0000 mgr.y (mgr.24473) 104 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:43.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:43 smithi183 bash[20824]: cluster 2024-02-13T03:23:42.600580+0000 mgr.y (mgr.24473) 104 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:45.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.347+0000 7eff283f6700 1 -- 172.21.15.161:0/1851977285 >> v1:172.21.15.161:6789/0 conn(0x7eff20100930 legacy=0x7eff20102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:45.354 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.347+0000 7eff283f6700 1 -- 172.21.15.161:0/1851977285 shutdown_connections 2024-02-13T03:23:45.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.347+0000 7eff283f6700 1 -- 172.21.15.161:0/1851977285 >> 172.21.15.161:0/1851977285 conn(0x7eff200f36d0 msgr2=0x7eff200f5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:45.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.347+0000 7eff283f6700 1 -- 172.21.15.161:0/1851977285 shutdown_connections 2024-02-13T03:23:45.355 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.347+0000 7eff283f6700 1 -- 172.21.15.161:0/1851977285 wait complete. 2024-02-13T03:23:45.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff283f6700 1 Processor -- start 2024-02-13T03:23:45.356 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff283f6700 1 -- start start 2024-02-13T03:23:45.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff283f6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7eff201173d0 con 0x7eff20100930 2024-02-13T03:23:45.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff283f6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7eff201185d0 con 0x7eff200fdae0 2024-02-13T03:23:45.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff283f6700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7eff201197d0 con 0x7eff20104440 2024-02-13T03:23:45.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff26993700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7eff20104440 0x7eff20115ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:60174/0 (socket says 172.21.15.161:60174) 2024-02-13T03:23:45.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff26993700 1 -- 172.21.15.161:0/3476984840 learned_addr learned my addr 172.21.15.161:0/3476984840 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:23:45.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580577409 0 0) 0x7eff201173d0 con 0x7eff20100930 2024-02-13T03:23:45.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff080033f0 con 0x7eff20100930 2024-02-13T03:23:45.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532101082 0 0) 0x7eff201197d0 con 0x7eff20104440 2024-02-13T03:23:45.358 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff201173d0 con 0x7eff20104440 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833957818 0 0) 0x7eff201185d0 con 0x7eff200fdae0 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff201197d0 con 0x7eff200fdae0 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2373806960 0 0) 0x7eff080033f0 con 0x7eff20100930 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7eff201185d0 con 0x7eff20100930 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7eff18002ee0 con 0x7eff20100930 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1412907328 0 0) 0x7eff201173d0 con 0x7eff20104440 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7eff080033f0 con 0x7eff20104440 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 305852627 0 0) 0x7eff201197d0 con 0x7eff200fdae0 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7eff201173d0 con 0x7eff200fdae0 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7eff10001880 con 0x7eff20104440 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7eff140017c0 con 0x7eff200fdae0 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 695016237 0 0) 0x7eff201185d0 con 0x7eff20100930 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 >> v1:172.21.15.161:6790/0 conn(0x7eff20104440 legacy=0x7eff20115ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:45.359 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 >> v1:172.21.15.183:6789/0 conn(0x7eff200fdae0 legacy=0x7eff200f9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:45.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2011a9d0 con 0x7eff20100930 2024-02-13T03:23:45.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff283f6700 1 -- 172.21.15.161:0/3476984840 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff201175c0 con 0x7eff20100930 2024-02-13T03:23:45.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff283f6700 1 -- 172.21.15.161:0/3476984840 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7eff20117a30 con 0x7eff20100930 2024-02-13T03:23:45.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7eff18003100 con 0x7eff20100930 2024-02-13T03:23:45.360 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7eff18003a20 con 0x7eff20100930 2024-02-13T03:23:45.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.351+0000 7eff283f6700 1 -- 172.21.15.161:0/3476984840 --> v1:172.21.15.161:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff0c000f10 con 0x7eff20100930 2024-02-13T03:23:45.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.355+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7eff180024b0 con 0x7eff20100930 2024-02-13T03:23:45.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.355+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7eff180762b0 con 0x7eff20100930 2024-02-13T03:23:45.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.355+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 --> v1:172.21.15.161:6809/1657207182 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff08070a60 con 0x7eff0806cd50 2024-02-13T03:23:45.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.355+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (unknown 2499814743 0 0) 0x7eff180766e0 con 0x7eff20100930 2024-02-13T03:23:45.363 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.355+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== osd.2 v1:172.21.15.161:6809/1657207182 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7eff08070a60 con 0x7eff0806cd50 2024-02-13T03:23:45.382 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.375+0000 7eff283f6700 1 -- 172.21.15.161:0/3476984840 --> v1:172.21.15.161:6809/1657207182 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7eff0c0029f0 con 0x7eff0806cd50 2024-02-13T03:23:45.383 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.375+0000 7eff1f7fe700 1 -- 172.21.15.161:0/3476984840 <== osd.2 v1:172.21.15.161:6809/1657207182 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2659494926) 0x7eff0c0029f0 con 0x7eff0806cd50 2024-02-13T03:23:45.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.375+0000 7eff283f6700 1 -- 172.21.15.161:0/3476984840 >> v1:172.21.15.161:6809/1657207182 conn(0x7eff0806cd50 legacy=0x7eff0806f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:45.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.375+0000 7eff283f6700 1 -- 172.21.15.161:0/3476984840 >> v1:172.21.15.161:6800/4114337139 conn(0x7eff08062400 legacy=0x7eff080648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:45.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.375+0000 7eff283f6700 1 -- 172.21.15.161:0/3476984840 >> v1:172.21.15.161:6789/0 conn(0x7eff20100930 legacy=0x7eff200fa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:45.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.375+0000 7eff26993700 1 -- 172.21.15.161:0/3476984840 reap_dead start 2024-02-13T03:23:45.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.379+0000 7eff283f6700 1 -- 172.21.15.161:0/3476984840 shutdown_connections 2024-02-13T03:23:45.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.379+0000 7eff283f6700 1 -- 172.21.15.161:0/3476984840 >> 172.21.15.161:0/3476984840 conn(0x7eff200f36d0 msgr2=0x7eff201078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:45.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.379+0000 7eff283f6700 1 -- 172.21.15.161:0/3476984840 shutdown_connections 2024-02-13T03:23:45.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:45.379+0000 7eff283f6700 1 -- 172.21.15.161:0/3476984840 wait complete. 2024-02-13T03:23:45.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:45 smithi161 bash[27879]: cluster 2024-02-13T03:23:44.601285+0000 mgr.y (mgr.24473) 105 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:45.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:45 smithi161 bash[19769]: cluster 2024-02-13T03:23:44.601285+0000 mgr.y (mgr.24473) 105 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:45.958 INFO:teuthology.orchestra.run.smithi161.stdout:77309411405 2024-02-13T03:23:45.958 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-13T03:23:45.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:45 smithi183 bash[20824]: cluster 2024-02-13T03:23:44.601285+0000 mgr.y (mgr.24473) 105 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:45.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:23:45 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:23:46.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:46 smithi161 bash[27879]: audit 2024-02-13T03:23:45.639361+0000 mgr.y (mgr.24473) 106 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:46.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:46 smithi161 bash[19769]: audit 2024-02-13T03:23:45.639361+0000 mgr.y (mgr.24473) 106 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:46.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:46 smithi183 bash[20824]: audit 2024-02-13T03:23:45.639361+0000 mgr.y (mgr.24473) 106 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:47.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:47 smithi161 bash[27879]: cluster 2024-02-13T03:23:46.602791+0000 mgr.y (mgr.24473) 107 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:47.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:47 smithi161 bash[19769]: cluster 2024-02-13T03:23:46.602791+0000 mgr.y (mgr.24473) 107 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:47.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:47 smithi183 bash[20824]: cluster 2024-02-13T03:23:46.602791+0000 mgr.y (mgr.24473) 107 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:49.678 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:23:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:23:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:23:49.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:49 smithi183 bash[20824]: cluster 2024-02-13T03:23:48.603483+0000 mgr.y (mgr.24473) 108 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:50.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:49 smithi161 bash[27879]: cluster 2024-02-13T03:23:48.603483+0000 mgr.y (mgr.24473) 108 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:50.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:49 smithi161 bash[19769]: cluster 2024-02-13T03:23:48.603483+0000 mgr.y (mgr.24473) 108 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:50.795 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:23:51.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:23:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:23:51] "GET /metrics HTTP/1.1" 200 172162 "" "Prometheus/2.33.4" 2024-02-13T03:23:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:51 smithi183 bash[20824]: cluster 2024-02-13T03:23:50.604878+0000 mgr.y (mgr.24473) 109 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:52.000 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:51 smithi161 bash[27879]: cluster 2024-02-13T03:23:50.604878+0000 mgr.y (mgr.24473) 109 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:52.000 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:51 smithi161 bash[19769]: cluster 2024-02-13T03:23:50.604878+0000 mgr.y (mgr.24473) 109 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:53.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.547+0000 7ff1bfec1700 1 -- 172.21.15.161:0/3763500171 >> v1:172.21.15.161:6789/0 conn(0x7ff1b80f8b70 legacy=0x7ff1b80f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:53.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.547+0000 7ff1bfec1700 1 -- 172.21.15.161:0/3763500171 shutdown_connections 2024-02-13T03:23:53.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.547+0000 7ff1bfec1700 1 -- 172.21.15.161:0/3763500171 >> 172.21.15.161:0/3763500171 conn(0x7ff1b80f2620 msgr2=0x7ff1b80f4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:53.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.547+0000 7ff1bfec1700 1 -- 172.21.15.161:0/3763500171 shutdown_connections 2024-02-13T03:23:53.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.547+0000 7ff1bfec1700 1 -- 172.21.15.161:0/3763500171 wait complete. 2024-02-13T03:23:53.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.547+0000 7ff1bfec1700 1 Processor -- start 2024-02-13T03:23:53.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.547+0000 7ff1bfec1700 1 -- start start 2024-02-13T03:23:53.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.547+0000 7ff1bfec1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff1b81173a0 con 0x7ff1b8104470 2024-02-13T03:23:53.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.547+0000 7ff1bfec1700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff1b8118580 con 0x7ff1b80f9660 2024-02-13T03:23:53.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.547+0000 7ff1bdc5d700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7ff1b80f8b70 0x7ff1b80ff570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:35328/0 (socket says 172.21.15.161:35328) 2024-02-13T03:23:53.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.547+0000 7ff1bdc5d700 1 -- 172.21.15.161:0/1623252212 learned_addr learned my addr 172.21.15.161:0/1623252212 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:23:53.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.547+0000 7ff1bfec1700 1 -- 172.21.15.161:0/1623252212 --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff1b8119780 con 0x7ff1b80f8b70 2024-02-13T03:23:53.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56971823 0 0) 0x7ff1b8119780 con 0x7ff1b80f8b70 2024-02-13T03:23:53.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1a40033f0 con 0x7ff1b80f8b70 2024-02-13T03:23:53.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923901473 0 0) 0x7ff1b81173a0 con 0x7ff1b8104470 2024-02-13T03:23:53.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1b8119780 con 0x7ff1b8104470 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777623625 0 0) 0x7ff1b8118580 con 0x7ff1b80f9660 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1b81173a0 con 0x7ff1b80f9660 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2140489268 0 0) 0x7ff1a40033f0 con 0x7ff1b80f8b70 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1b8118580 con 0x7ff1b80f8b70 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7ff1b4001370 con 0x7ff1b80f8b70 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3267352098 0 0) 0x7ff1b81173a0 con 0x7ff1b80f9660 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1a40033f0 con 0x7ff1b80f9660 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7ff1a80019d0 con 0x7ff1b80f9660 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2998314689 0 0) 0x7ff1b8119780 con 0x7ff1b8104470 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1b81173a0 con 0x7ff1b8104470 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7ff1b0001bc0 con 0x7ff1b8104470 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2945526697 0 0) 0x7ff1b8118580 con 0x7ff1b80f8b70 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 >> v1:172.21.15.183:6789/0 conn(0x7ff1b80f9660 legacy=0x7ff1b80ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:53.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 >> v1:172.21.15.161:6789/0 conn(0x7ff1b8104470 legacy=0x7ff1b8115ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:53.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b811a980 con 0x7ff1b80f8b70 2024-02-13T03:23:53.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1bfec1700 1 -- 172.21.15.161:0/1623252212 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b8117590 con 0x7ff1b80f8b70 2024-02-13T03:23:53.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1bfec1700 1 -- 172.21.15.161:0/1623252212 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b8117a00 con 0x7ff1b80f8b70 2024-02-13T03:23:53.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7ff1b40014d0 con 0x7ff1b80f8b70 2024-02-13T03:23:53.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7ff1b4003870 con 0x7ff1b80f8b70 2024-02-13T03:23:53.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1bfec1700 1 -- 172.21.15.161:0/1623252212 --> v1:172.21.15.161:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff19c000f10 con 0x7ff1b80f8b70 2024-02-13T03:23:53.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.551+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7ff1b4003ab0 con 0x7ff1b80f8b70 2024-02-13T03:23:53.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.555+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7ff1b40788b0 con 0x7ff1b80f8b70 2024-02-13T03:23:53.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.555+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 --> v1:172.21.15.161:6813/2788336304 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1a4074b50 con 0x7ff1a4070e40 2024-02-13T03:23:53.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.555+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (unknown 2499814743 0 0) 0x7ff1b4078c70 con 0x7ff1b80f8b70 2024-02-13T03:23:53.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.555+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== osd.3 v1:172.21.15.161:6813/2788336304 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7ff1a4074b50 con 0x7ff1a4070e40 2024-02-13T03:23:53.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.575+0000 7ff1bfec1700 1 -- 172.21.15.161:0/1623252212 --> v1:172.21.15.161:6813/2788336304 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff19c0029f0 con 0x7ff1a4070e40 2024-02-13T03:23:53.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.575+0000 7ff1aeffd700 1 -- 172.21.15.161:0/1623252212 <== osd.3 v1:172.21.15.161:6813/2788336304 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1746907681) 0x7ff19c0029f0 con 0x7ff1a4070e40 2024-02-13T03:23:53.583 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.579+0000 7ff1bfec1700 1 -- 172.21.15.161:0/1623252212 >> v1:172.21.15.161:6813/2788336304 conn(0x7ff1a4070e40 legacy=0x7ff1a40732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:53.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.579+0000 7ff1bfec1700 1 -- 172.21.15.161:0/1623252212 >> v1:172.21.15.161:6800/4114337139 conn(0x7ff1a4066700 legacy=0x7ff1a4068bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:53.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.579+0000 7ff1bfec1700 1 -- 172.21.15.161:0/1623252212 >> v1:172.21.15.161:6790/0 conn(0x7ff1b80f8b70 legacy=0x7ff1b80ff570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:23:53.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.579+0000 7ff1be45e700 1 -- 172.21.15.161:0/1623252212 reap_dead start 2024-02-13T03:23:53.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.579+0000 7ff1bfec1700 1 -- 172.21.15.161:0/1623252212 shutdown_connections 2024-02-13T03:23:53.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.579+0000 7ff1bfec1700 1 -- 172.21.15.161:0/1623252212 >> 172.21.15.161:0/1623252212 conn(0x7ff1b80f2620 msgr2=0x7ff1b8106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:23:53.584 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.579+0000 7ff1bfec1700 1 -- 172.21.15.161:0/1623252212 shutdown_connections 2024-02-13T03:23:53.585 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:23:53.579+0000 7ff1bfec1700 1 -- 172.21.15.161:0/1623252212 wait complete. 2024-02-13T03:23:53.646 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:53 smithi161 bash[19769]: cluster 2024-02-13T03:23:52.605627+0000 mgr.y (mgr.24473) 110 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:53.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:53 smithi161 bash[27879]: cluster 2024-02-13T03:23:52.605627+0000 mgr.y (mgr.24473) 110 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:53 smithi183 bash[20824]: cluster 2024-02-13T03:23:52.605627+0000 mgr.y (mgr.24473) 110 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:54.222 INFO:teuthology.orchestra.run.smithi161.stdout:107374182471 2024-02-13T03:23:54.223 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-13T03:23:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:55 smithi161 bash[19769]: cluster 2024-02-13T03:23:54.606472+0000 mgr.y (mgr.24473) 111 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:55 smithi161 bash[27879]: cluster 2024-02-13T03:23:54.606472+0000 mgr.y (mgr.24473) 111 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:55.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:55 smithi183 bash[20824]: cluster 2024-02-13T03:23:54.606472+0000 mgr.y (mgr.24473) 111 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:23:55.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:23:55 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:23:56.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:56 smithi161 bash[19769]: audit 2024-02-13T03:23:55.650708+0000 mgr.y (mgr.24473) 112 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:56.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:56 smithi161 bash[19769]: audit 2024-02-13T03:23:56.245653+0000 mon.c (mon.2) 63 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:23:56.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:56 smithi161 bash[19769]: audit 2024-02-13T03:23:56.555316+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:23:56.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:56 smithi161 bash[27879]: audit 2024-02-13T03:23:55.650708+0000 mgr.y (mgr.24473) 112 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:56.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:56 smithi161 bash[27879]: audit 2024-02-13T03:23:56.245653+0000 mon.c (mon.2) 63 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:23:56.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:56 smithi161 bash[27879]: audit 2024-02-13T03:23:56.555316+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:23:56.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:56 smithi183 bash[20824]: audit 2024-02-13T03:23:55.650708+0000 mgr.y (mgr.24473) 112 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:23:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:56 smithi183 bash[20824]: audit 2024-02-13T03:23:56.245653+0000 mon.c (mon.2) 63 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:23:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:56 smithi183 bash[20824]: audit 2024-02-13T03:23:56.555316+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:23:57.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:57 smithi183 bash[20824]: cluster 2024-02-13T03:23:56.607960+0000 mgr.y (mgr.24473) 113 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:58.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:57 smithi161 bash[19769]: cluster 2024-02-13T03:23:56.607960+0000 mgr.y (mgr.24473) 113 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:58.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:57 smithi161 bash[27879]: cluster 2024-02-13T03:23:56.607960+0000 mgr.y (mgr.24473) 113 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:23:59.050 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:23:59.682 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:23:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:23:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:23:59.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:23:59 smithi183 bash[20824]: cluster 2024-02-13T03:23:58.608671+0000 mgr.y (mgr.24473) 114 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:00.027 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:23:59 smithi161 bash[19769]: cluster 2024-02-13T03:23:58.608671+0000 mgr.y (mgr.24473) 114 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:00.027 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:23:59 smithi161 bash[27879]: cluster 2024-02-13T03:23:58.608671+0000 mgr.y (mgr.24473) 114 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:01.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:24:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:24:01] "GET /metrics HTTP/1.1" 200 172162 "" "Prometheus/2.33.4" 2024-02-13T03:24:01.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7c666a700 1 -- 172.21.15.161:0/2685007888 >> v1:172.21.15.161:6790/0 conn(0x7ff7c0104470 legacy=0x7ff7c0106910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:01.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7c666a700 1 -- 172.21.15.161:0/2685007888 shutdown_connections 2024-02-13T03:24:01.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7c666a700 1 -- 172.21.15.161:0/2685007888 >> 172.21.15.161:0/2685007888 conn(0x7ff7c006c1e0 msgr2=0x7ff7c006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:01.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7c666a700 1 -- 172.21.15.161:0/2685007888 shutdown_connections 2024-02-13T03:24:01.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7c666a700 1 -- 172.21.15.161:0/2685007888 wait complete. 2024-02-13T03:24:01.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7c666a700 1 Processor -- start 2024-02-13T03:24:01.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7c666a700 1 -- start start 2024-02-13T03:24:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7c666a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff7c0117580 con 0x7ff7c0100960 2024-02-13T03:24:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7c666a700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff7c0118760 con 0x7ff7c0104470 2024-02-13T03:24:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7bffff700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7ff7c00fdb80 0x7ff7c00f9f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:38024/0 (socket says 172.21.15.161:38024) 2024-02-13T03:24:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7c666a700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff7c0119960 con 0x7ff7c00fdb80 2024-02-13T03:24:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7bffff700 1 -- 172.21.15.161:0/2188733456 learned_addr learned my addr 172.21.15.161:0/2188733456 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:24:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091314603 0 0) 0x7ff7c0119960 con 0x7ff7c00fdb80 2024-02-13T03:24:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7ac0033f0 con 0x7ff7c00fdb80 2024-02-13T03:24:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062227600 0 0) 0x7ff7c0117580 con 0x7ff7c0100960 2024-02-13T03:24:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7c0119960 con 0x7ff7c0100960 2024-02-13T03:24:01.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785903761 0 0) 0x7ff7c0118760 con 0x7ff7c0104470 2024-02-13T03:24:01.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.870+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7c0117580 con 0x7ff7c0104470 2024-02-13T03:24:01.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3169162575 0 0) 0x7ff7ac0033f0 con 0x7ff7c00fdb80 2024-02-13T03:24:01.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7c0118760 con 0x7ff7c00fdb80 2024-02-13T03:24:01.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7ff7b8002950 con 0x7ff7c00fdb80 2024-02-13T03:24:01.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 602355253 0 0) 0x7ff7c0119960 con 0x7ff7c0100960 2024-02-13T03:24:01.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7ac0033f0 con 0x7ff7c0100960 2024-02-13T03:24:01.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7ff7a8001800 con 0x7ff7c0100960 2024-02-13T03:24:01.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 370437035 0 0) 0x7ff7c0117580 con 0x7ff7c0104470 2024-02-13T03:24:01.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7c0119960 con 0x7ff7c0104470 2024-02-13T03:24:01.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7ff7b0002240 con 0x7ff7c0104470 2024-02-13T03:24:01.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 41165235 0 0) 0x7ff7c0118760 con 0x7ff7c00fdb80 2024-02-13T03:24:01.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 >> v1:172.21.15.183:6789/0 conn(0x7ff7c0104470 legacy=0x7ff7c0115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:01.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 >> v1:172.21.15.161:6789/0 conn(0x7ff7c0100960 legacy=0x7ff7c00fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:01.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c011ab60 con 0x7ff7c00fdb80 2024-02-13T03:24:01.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7c666a700 1 -- 172.21.15.161:0/2188733456 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c0118950 con 0x7ff7c00fdb80 2024-02-13T03:24:01.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7c666a700 1 -- 172.21.15.161:0/2188733456 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c0118dc0 con 0x7ff7c00fdb80 2024-02-13T03:24:01.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7ff7b8002d00 con 0x7ff7c00fdb80 2024-02-13T03:24:01.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7ff7b8005b70 con 0x7ff7c00fdb80 2024-02-13T03:24:01.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7c666a700 1 -- 172.21.15.161:0/2188733456 --> v1:172.21.15.161:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff7a0000f10 con 0x7ff7c00fdb80 2024-02-13T03:24:01.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.874+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7ff7b8001660 con 0x7ff7c00fdb80 2024-02-13T03:24:01.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.878+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7ff7b8078580 con 0x7ff7c00fdb80 2024-02-13T03:24:01.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.878+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 --> v1:172.21.15.183:6800/384629816 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7ac070840 con 0x7ff7ac06cb30 2024-02-13T03:24:01.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.878+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (unknown 2499814743 0 0) 0x7ff7b80789b0 con 0x7ff7c00fdb80 2024-02-13T03:24:01.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.878+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== osd.4 v1:172.21.15.183:6800/384629816 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7ff7ac070840 con 0x7ff7ac06cb30 2024-02-13T03:24:01.906 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.898+0000 7ff7c666a700 1 -- 172.21.15.161:0/2188733456 --> v1:172.21.15.183:6800/384629816 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff7a00029f0 con 0x7ff7ac06cb30 2024-02-13T03:24:01.907 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.898+0000 7ff7bd7fa700 1 -- 172.21.15.161:0/2188733456 <== osd.4 v1:172.21.15.183:6800/384629816 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3730495807) 0x7ff7a00029f0 con 0x7ff7ac06cb30 2024-02-13T03:24:01.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.902+0000 7ff7c666a700 1 -- 172.21.15.161:0/2188733456 >> v1:172.21.15.183:6800/384629816 conn(0x7ff7ac06cb30 legacy=0x7ff7ac06efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:01.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.902+0000 7ff7c666a700 1 -- 172.21.15.161:0/2188733456 >> v1:172.21.15.161:6800/4114337139 conn(0x7ff7ac062310 legacy=0x7ff7ac0647d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:01.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.902+0000 7ff7c666a700 1 -- 172.21.15.161:0/2188733456 >> v1:172.21.15.161:6790/0 conn(0x7ff7c00fdb80 legacy=0x7ff7c00f9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:01.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.902+0000 7ff7c4c07700 1 -- 172.21.15.161:0/2188733456 reap_dead start 2024-02-13T03:24:01.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.902+0000 7ff7c666a700 1 -- 172.21.15.161:0/2188733456 shutdown_connections 2024-02-13T03:24:01.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.902+0000 7ff7c666a700 1 -- 172.21.15.161:0/2188733456 >> 172.21.15.161:0/2188733456 conn(0x7ff7c006c1e0 msgr2=0x7ff7c01068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:01.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.902+0000 7ff7c666a700 1 -- 172.21.15.161:0/2188733456 shutdown_connections 2024-02-13T03:24:01.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:01.902+0000 7ff7c666a700 1 -- 172.21.15.161:0/2188733456 wait complete. 2024-02-13T03:24:01.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:01 smithi161 bash[27879]: cluster 2024-02-13T03:24:00.610326+0000 mgr.y (mgr.24473) 115 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:01.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:01 smithi161 bash[19769]: cluster 2024-02-13T03:24:00.610326+0000 mgr.y (mgr.24473) 115 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:01.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:01 smithi183 bash[20824]: cluster 2024-02-13T03:24:00.610326+0000 mgr.y (mgr.24473) 115 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:02.509 INFO:teuthology.orchestra.run.smithi161.stdout:128849018944 2024-02-13T03:24:02.509 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-13T03:24:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:03 smithi161 bash[27879]: audit 2024-02-13T03:24:02.588192+0000 mon.a (mon.0) 683 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:24:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:03 smithi161 bash[27879]: cluster 2024-02-13T03:24:02.611147+0000 mgr.y (mgr.24473) 116 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:03 smithi161 bash[27879]: audit 2024-02-13T03:24:02.876571+0000 mon.c (mon.2) 64 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:24:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:03 smithi161 bash[27879]: audit 2024-02-13T03:24:02.878398+0000 mon.c (mon.2) 65 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:24:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:03 smithi161 bash[27879]: audit 2024-02-13T03:24:02.889586+0000 mon.a (mon.0) 684 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:24:03.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:03 smithi161 bash[19769]: audit 2024-02-13T03:24:02.588192+0000 mon.a (mon.0) 683 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:24:03.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:03 smithi161 bash[19769]: cluster 2024-02-13T03:24:02.611147+0000 mgr.y (mgr.24473) 116 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:03.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:03 smithi161 bash[19769]: audit 2024-02-13T03:24:02.876571+0000 mon.c (mon.2) 64 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:24:03.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:03 smithi161 bash[19769]: audit 2024-02-13T03:24:02.878398+0000 mon.c (mon.2) 65 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:24:03.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:03 smithi161 bash[19769]: audit 2024-02-13T03:24:02.889586+0000 mon.a (mon.0) 684 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:24:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:03 smithi183 bash[20824]: audit 2024-02-13T03:24:02.588192+0000 mon.a (mon.0) 683 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:24:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:03 smithi183 bash[20824]: cluster 2024-02-13T03:24:02.611147+0000 mgr.y (mgr.24473) 116 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:03 smithi183 bash[20824]: audit 2024-02-13T03:24:02.876571+0000 mon.c (mon.2) 64 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:24:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:03 smithi183 bash[20824]: audit 2024-02-13T03:24:02.878398+0000 mon.c (mon.2) 65 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:24:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:03 smithi183 bash[20824]: audit 2024-02-13T03:24:02.889586+0000 mon.a (mon.0) 684 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:24:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:05 smithi161 bash[27879]: cluster 2024-02-13T03:24:04.611855+0000 mgr.y (mgr.24473) 117 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:05 smithi161 bash[19769]: cluster 2024-02-13T03:24:04.611855+0000 mgr.y (mgr.24473) 117 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:05.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:05 smithi183 bash[20824]: cluster 2024-02-13T03:24:04.611855+0000 mgr.y (mgr.24473) 117 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:05.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:24:05 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:24:06.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:06 smithi183 bash[20824]: audit 2024-02-13T03:24:05.652940+0000 mgr.y (mgr.24473) 118 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:06 smithi161 bash[19769]: audit 2024-02-13T03:24:05.652940+0000 mgr.y (mgr.24473) 118 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:06 smithi161 bash[27879]: audit 2024-02-13T03:24:05.652940+0000 mgr.y (mgr.24473) 118 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:07.347 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:24:07.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:07 smithi183 bash[20824]: cluster 2024-02-13T03:24:06.613321+0000 mgr.y (mgr.24473) 119 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:08.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:07 smithi161 bash[27879]: cluster 2024-02-13T03:24:06.613321+0000 mgr.y (mgr.24473) 119 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:08.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:07 smithi161 bash[19769]: cluster 2024-02-13T03:24:06.613321+0000 mgr.y (mgr.24473) 119 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:09.688 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:24:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:24:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:24:09.955 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:09 smithi161 bash[27879]: cluster 2024-02-13T03:24:08.614044+0000 mgr.y (mgr.24473) 120 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:09.955 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:09 smithi161 bash[19769]: cluster 2024-02-13T03:24:08.614044+0000 mgr.y (mgr.24473) 120 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:09.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:09 smithi183 bash[20824]: cluster 2024-02-13T03:24:08.614044+0000 mgr.y (mgr.24473) 120 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:10.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.186+0000 7fa3cf315700 1 -- 172.21.15.161:0/2343390686 >> v1:172.21.15.161:6790/0 conn(0x7fa3c80ff850 legacy=0x7fa3c8101c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:10.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.186+0000 7fa3cf315700 1 -- 172.21.15.161:0/2343390686 shutdown_connections 2024-02-13T03:24:10.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.186+0000 7fa3cf315700 1 -- 172.21.15.161:0/2343390686 >> 172.21.15.161:0/2343390686 conn(0x7fa3c80f2630 msgr2=0x7fa3c80f4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:10.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.186+0000 7fa3cf315700 1 -- 172.21.15.161:0/2343390686 shutdown_connections 2024-02-13T03:24:10.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.186+0000 7fa3cf315700 1 -- 172.21.15.161:0/2343390686 wait complete. 2024-02-13T03:24:10.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.186+0000 7fa3cf315700 1 Processor -- start 2024-02-13T03:24:10.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.186+0000 7fa3cf315700 1 -- start start 2024-02-13T03:24:10.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3cf315700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa3c80fc2a0 con 0x7fa3c80fca00 2024-02-13T03:24:10.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3cf315700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa3c80fc470 con 0x7fa3c80ff850 2024-02-13T03:24:10.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3cf315700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa3c811bc20 con 0x7fa3c8103360 2024-02-13T03:24:10.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3cd0b1700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa3c80fca00 0x7fa3c80fba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56042/0 (socket says 172.21.15.161:56042) 2024-02-13T03:24:10.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3cd0b1700 1 -- 172.21.15.161:0/26223787 learned_addr learned my addr 172.21.15.161:0/26223787 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:24:10.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230693959 0 0) 0x7fa3c80fc2a0 con 0x7fa3c80fca00 2024-02-13T03:24:10.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3b4003030 con 0x7fa3c80fca00 2024-02-13T03:24:10.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457010046 0 0) 0x7fa3c811bc20 con 0x7fa3c8103360 2024-02-13T03:24:10.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c80fc2a0 con 0x7fa3c8103360 2024-02-13T03:24:10.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975061918 0 0) 0x7fa3c80fc470 con 0x7fa3c80ff850 2024-02-13T03:24:10.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c811bc20 con 0x7fa3c80ff850 2024-02-13T03:24:10.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2763507360 0 0) 0x7fa3b4003030 con 0x7fa3c80fca00 2024-02-13T03:24:10.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa3c80fc470 con 0x7fa3c80fca00 2024-02-13T03:24:10.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fa3bc001980 con 0x7fa3c80fca00 2024-02-13T03:24:10.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 869720092 0 0) 0x7fa3c80fc2a0 con 0x7fa3c8103360 2024-02-13T03:24:10.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa3b4003030 con 0x7fa3c8103360 2024-02-13T03:24:10.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fa3c0001d70 con 0x7fa3c8103360 2024-02-13T03:24:10.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3631508641 0 0) 0x7fa3c80fc470 con 0x7fa3c80fca00 2024-02-13T03:24:10.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 >> v1:172.21.15.161:6790/0 conn(0x7fa3c8103360 legacy=0x7fa3c81184e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:10.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 >> v1:172.21.15.183:6789/0 conn(0x7fa3c80ff850 legacy=0x7fa3c8114db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:10.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c811ce00 con 0x7fa3c80fca00 2024-02-13T03:24:10.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3cf315700 1 -- 172.21.15.161:0/26223787 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c811be10 con 0x7fa3c80fca00 2024-02-13T03:24:10.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3cf315700 1 -- 172.21.15.161:0/26223787 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c811c300 con 0x7fa3c80fca00 2024-02-13T03:24:10.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fa3bc002100 con 0x7fa3c80fca00 2024-02-13T03:24:10.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fa3bc005a70 con 0x7fa3c80fca00 2024-02-13T03:24:10.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3cf315700 1 -- 172.21.15.161:0/26223787 --> v1:172.21.15.161:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa3ac000f10 con 0x7fa3c80fca00 2024-02-13T03:24:10.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.190+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fa3bc005bd0 con 0x7fa3c80fca00 2024-02-13T03:24:10.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.194+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7fa3bc0784e0 con 0x7fa3c80fca00 2024-02-13T03:24:10.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.194+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 --> v1:172.21.15.183:6804/2958722347 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3b40704c0 con 0x7fa3b406c850 2024-02-13T03:24:10.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.194+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (unknown 2499814743 0 0) 0x7fa3bc0788e0 con 0x7fa3c80fca00 2024-02-13T03:24:10.202 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.194+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== osd.5 v1:172.21.15.183:6804/2958722347 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7fa3b40704c0 con 0x7fa3b406c850 2024-02-13T03:24:10.221 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.214+0000 7fa3cf315700 1 -- 172.21.15.161:0/26223787 --> v1:172.21.15.183:6804/2958722347 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa3ac0029f0 con 0x7fa3b406c850 2024-02-13T03:24:10.222 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.214+0000 7fa3ba7fc700 1 -- 172.21.15.161:0/26223787 <== osd.5 v1:172.21.15.183:6804/2958722347 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3684147131) 0x7fa3ac0029f0 con 0x7fa3b406c850 2024-02-13T03:24:10.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.214+0000 7fa3cf315700 1 -- 172.21.15.161:0/26223787 >> v1:172.21.15.183:6804/2958722347 conn(0x7fa3b406c850 legacy=0x7fa3b406ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:10.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.214+0000 7fa3cf315700 1 -- 172.21.15.161:0/26223787 >> v1:172.21.15.161:6800/4114337139 conn(0x7fa3b4062000 legacy=0x7fa3b40644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:10.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.214+0000 7fa3cf315700 1 -- 172.21.15.161:0/26223787 >> v1:172.21.15.161:6789/0 conn(0x7fa3c80fca00 legacy=0x7fa3c80fba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:10.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.214+0000 7fa3cd8b2700 1 -- 172.21.15.161:0/26223787 reap_dead start 2024-02-13T03:24:10.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.214+0000 7fa3cf315700 1 -- 172.21.15.161:0/26223787 shutdown_connections 2024-02-13T03:24:10.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.214+0000 7fa3cf315700 1 -- 172.21.15.161:0/26223787 >> 172.21.15.161:0/26223787 conn(0x7fa3c80f2630 msgr2=0x7fa3c80f2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:10.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.214+0000 7fa3cf315700 1 -- 172.21.15.161:0/26223787 shutdown_connections 2024-02-13T03:24:10.224 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:10.214+0000 7fa3cf315700 1 -- 172.21.15.161:0/26223787 wait complete. 2024-02-13T03:24:10.825 INFO:teuthology.orchestra.run.smithi161.stdout:154618822713 2024-02-13T03:24:10.825 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-13T03:24:11.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:24:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:24:11] "GET /metrics HTTP/1.1" 200 172155 "" "Prometheus/2.33.4" 2024-02-13T03:24:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:11 smithi183 bash[20824]: cluster 2024-02-13T03:24:10.615511+0000 mgr.y (mgr.24473) 121 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:12.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:11 smithi161 bash[19769]: cluster 2024-02-13T03:24:10.615511+0000 mgr.y (mgr.24473) 121 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:11 smithi161 bash[27879]: cluster 2024-02-13T03:24:10.615511+0000 mgr.y (mgr.24473) 121 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:13 smithi161 bash[27879]: cluster 2024-02-13T03:24:12.616241+0000 mgr.y (mgr.24473) 122 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:13.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:13 smithi161 bash[19769]: cluster 2024-02-13T03:24:12.616241+0000 mgr.y (mgr.24473) 122 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:13 smithi183 bash[20824]: cluster 2024-02-13T03:24:12.616241+0000 mgr.y (mgr.24473) 122 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:15.654 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:24:15.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:15 smithi161 bash[19769]: cluster 2024-02-13T03:24:14.616927+0000 mgr.y (mgr.24473) 123 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:15.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:15 smithi161 bash[27879]: cluster 2024-02-13T03:24:14.616927+0000 mgr.y (mgr.24473) 123 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:15.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:15 smithi183 bash[20824]: cluster 2024-02-13T03:24:14.616927+0000 mgr.y (mgr.24473) 123 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:15.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:24:15 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:24:16.885 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:16 smithi161 bash[27879]: audit 2024-02-13T03:24:15.656160+0000 mgr.y (mgr.24473) 124 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:16.885 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:16 smithi161 bash[19769]: audit 2024-02-13T03:24:15.656160+0000 mgr.y (mgr.24473) 124 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:16.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:16 smithi183 bash[20824]: audit 2024-02-13T03:24:15.656160+0000 mgr.y (mgr.24473) 124 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:17.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:17 smithi183 bash[20824]: cluster 2024-02-13T03:24:16.617882+0000 mgr.y (mgr.24473) 125 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:18.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:17 smithi161 bash[27879]: cluster 2024-02-13T03:24:16.617882+0000 mgr.y (mgr.24473) 125 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:18.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:17 smithi161 bash[19769]: cluster 2024-02-13T03:24:16.617882+0000 mgr.y (mgr.24473) 125 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:18.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.498+0000 7fee17822700 1 -- 172.21.15.161:0/168470937 >> v1:172.21.15.161:6789/0 conn(0x7fee10106630 legacy=0x7fee10108ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:18.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.498+0000 7fee17822700 1 -- 172.21.15.161:0/168470937 shutdown_connections 2024-02-13T03:24:18.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.498+0000 7fee17822700 1 -- 172.21.15.161:0/168470937 >> 172.21.15.161:0/168470937 conn(0x7fee100f5900 msgr2=0x7fee100f7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:18.507 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.498+0000 7fee17822700 1 -- 172.21.15.161:0/168470937 shutdown_connections 2024-02-13T03:24:18.507 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.498+0000 7fee17822700 1 -- 172.21.15.161:0/168470937 wait complete. 2024-02-13T03:24:18.507 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee17822700 1 Processor -- start 2024-02-13T03:24:18.507 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee17822700 1 -- start start 2024-02-13T03:24:18.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee17822700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fee101197e0 con 0x7fee10102b20 2024-02-13T03:24:18.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee17822700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fee1011a9e0 con 0x7fee100ffcd0 2024-02-13T03:24:18.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee17822700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fee1011bbe0 con 0x7fee10106630 2024-02-13T03:24:18.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee15dbf700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7fee10106630 0x7fee10117ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:34024/0 (socket says 172.21.15.161:34024) 2024-02-13T03:24:18.508 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee15dbf700 1 -- 172.21.15.161:0/2263253710 learned_addr learned my addr 172.21.15.161:0/2263253710 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:24:18.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194605389 0 0) 0x7fee1011bbe0 con 0x7fee10106630 2024-02-13T03:24:18.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedf8003630 con 0x7fee10106630 2024-02-13T03:24:18.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486160436 0 0) 0x7fee101197e0 con 0x7fee10102b20 2024-02-13T03:24:18.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee1011bbe0 con 0x7fee10102b20 2024-02-13T03:24:18.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265206053 0 0) 0x7fee1011a9e0 con 0x7fee100ffcd0 2024-02-13T03:24:18.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee101197e0 con 0x7fee100ffcd0 2024-02-13T03:24:18.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2581734789 0 0) 0x7fedf8003630 con 0x7fee10106630 2024-02-13T03:24:18.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fee1011a9e0 con 0x7fee10106630 2024-02-13T03:24:18.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fee080023d0 con 0x7fee10106630 2024-02-13T03:24:18.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3891945214 0 0) 0x7fee1011a9e0 con 0x7fee10106630 2024-02-13T03:24:18.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 >> v1:172.21.15.183:6789/0 conn(0x7fee100ffcd0 legacy=0x7fee100fc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:18.510 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 >> v1:172.21.15.161:6789/0 conn(0x7fee10102b20 legacy=0x7fee100fc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:18.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1011cde0 con 0x7fee10106630 2024-02-13T03:24:18.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee17822700 1 -- 172.21.15.161:0/2263253710 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee101199d0 con 0x7fee10106630 2024-02-13T03:24:18.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee17822700 1 -- 172.21.15.161:0/2263253710 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fee10119e40 con 0x7fee10106630 2024-02-13T03:24:18.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fee08002630 con 0x7fee10106630 2024-02-13T03:24:18.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fee08003b10 con 0x7fee10106630 2024-02-13T03:24:18.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.502+0000 7fee17822700 1 -- 172.21.15.161:0/2263253710 --> v1:172.21.15.161:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fedfc000f40 con 0x7fee10106630 2024-02-13T03:24:18.511 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.506+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fee08001bc0 con 0x7fee10106630 2024-02-13T03:24:18.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.506+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7fee08078bc0 con 0x7fee10106630 2024-02-13T03:24:18.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.506+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 --> v1:172.21.15.183:6808/3716679598 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fedf8070570 con 0x7fedf806c860 2024-02-13T03:24:18.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.506+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (unknown 2499814743 0 0) 0x7fee08078fd0 con 0x7fee10106630 2024-02-13T03:24:18.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.506+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 <== osd.6 v1:172.21.15.183:6808/3716679598 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7fedf8070570 con 0x7fedf806c860 2024-02-13T03:24:18.534 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.526+0000 7fee17822700 1 -- 172.21.15.161:0/2263253710 --> v1:172.21.15.183:6808/3716679598 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fedfc002a20 con 0x7fedf806c860 2024-02-13T03:24:18.535 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.530+0000 7fee067fc700 1 -- 172.21.15.161:0/2263253710 <== osd.6 v1:172.21.15.183:6808/3716679598 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1134719554) 0x7fedfc002a20 con 0x7fedf806c860 2024-02-13T03:24:18.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.530+0000 7fee17822700 1 -- 172.21.15.161:0/2263253710 >> v1:172.21.15.183:6808/3716679598 conn(0x7fedf806c860 legacy=0x7fedf806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:18.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.530+0000 7fee17822700 1 -- 172.21.15.161:0/2263253710 >> v1:172.21.15.161:6800/4114337139 conn(0x7fedf80620c0 legacy=0x7fedf8064580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:18.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.530+0000 7fee17822700 1 -- 172.21.15.161:0/2263253710 >> v1:172.21.15.161:6790/0 conn(0x7fee10106630 legacy=0x7fee10117ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:18.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.530+0000 7fee15dbf700 1 -- 172.21.15.161:0/2263253710 reap_dead start 2024-02-13T03:24:18.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.530+0000 7fee17822700 1 -- 172.21.15.161:0/2263253710 shutdown_connections 2024-02-13T03:24:18.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.530+0000 7fee17822700 1 -- 172.21.15.161:0/2263253710 >> 172.21.15.161:0/2263253710 conn(0x7fee100f5900 msgr2=0x7fee10109ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:18.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.530+0000 7fee17822700 1 -- 172.21.15.161:0/2263253710 shutdown_connections 2024-02-13T03:24:18.537 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:18.530+0000 7fee17822700 1 -- 172.21.15.161:0/2263253710 wait complete. 2024-02-13T03:24:19.145 INFO:teuthology.orchestra.run.smithi161.stdout:180388626481 2024-02-13T03:24:19.145 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-13T03:24:19.693 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:24:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:24:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:24:19.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:19 smithi183 bash[20824]: cluster 2024-02-13T03:24:18.618597+0000 mgr.y (mgr.24473) 126 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:20.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:19 smithi161 bash[27879]: cluster 2024-02-13T03:24:18.618597+0000 mgr.y (mgr.24473) 126 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:20.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:19 smithi161 bash[19769]: cluster 2024-02-13T03:24:18.618597+0000 mgr.y (mgr.24473) 126 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:21.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:24:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:24:21] "GET /metrics HTTP/1.1" 200 172166 "" "Prometheus/2.33.4" 2024-02-13T03:24:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:21 smithi183 bash[20824]: cluster 2024-02-13T03:24:20.620015+0000 mgr.y (mgr.24473) 127 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:22.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:21 smithi161 bash[27879]: cluster 2024-02-13T03:24:20.620015+0000 mgr.y (mgr.24473) 127 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:22.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:21 smithi161 bash[19769]: cluster 2024-02-13T03:24:20.620015+0000 mgr.y (mgr.24473) 127 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:23.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:23 smithi161 bash[19769]: cluster 2024-02-13T03:24:22.620776+0000 mgr.y (mgr.24473) 128 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:23 smithi161 bash[27879]: cluster 2024-02-13T03:24:22.620776+0000 mgr.y (mgr.24473) 128 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:23 smithi183 bash[20824]: cluster 2024-02-13T03:24:22.620776+0000 mgr.y (mgr.24473) 128 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:23.985 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:24:25.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:25 smithi183 bash[20824]: cluster 2024-02-13T03:24:24.621492+0000 mgr.y (mgr.24473) 129 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:25.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:24:25 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:24:26.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:25 smithi161 bash[27879]: cluster 2024-02-13T03:24:24.621492+0000 mgr.y (mgr.24473) 129 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:26.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:25 smithi161 bash[19769]: cluster 2024-02-13T03:24:24.621492+0000 mgr.y (mgr.24473) 129 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f57acebc700 1 -- 172.21.15.161:0/1300956824 >> v1:172.21.15.161:6790/0 conn(0x7f57a8104450 legacy=0x7f57a81068f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:26.856 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f57acebc700 1 -- 172.21.15.161:0/1300956824 shutdown_connections 2024-02-13T03:24:26.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f57acebc700 1 -- 172.21.15.161:0/1300956824 >> 172.21.15.161:0/1300956824 conn(0x7f57a80f2610 msgr2=0x7f57a80f4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:26.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f57acebc700 1 -- 172.21.15.161:0/1300956824 shutdown_connections 2024-02-13T03:24:26.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f57acebc700 1 -- 172.21.15.161:0/1300956824 wait complete. 2024-02-13T03:24:26.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f57acebc700 1 Processor -- start 2024-02-13T03:24:26.857 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f57acebc700 1 -- start start 2024-02-13T03:24:26.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f57acebc700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f57a8117410 con 0x7f57a80f8ac0 2024-02-13T03:24:26.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f57acebc700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f57a81185f0 con 0x7f57a80f9670 2024-02-13T03:24:26.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f57acebc700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f57a81197f0 con 0x7f57a8104450 2024-02-13T03:24:26.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f57a659c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f57a80f8ac0 0x7f57a80ff550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:56996/0 (socket says 172.21.15.161:56996) 2024-02-13T03:24:26.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f57a6d9d700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f57a8104450 0x7f57a8115ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:34044/0 (socket says 172.21.15.161:34044) 2024-02-13T03:24:26.858 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f57a659c700 1 -- 172.21.15.161:0/3921879180 learned_addr learned my addr 172.21.15.161:0/3921879180 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:24:26.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 109373139 0 0) 0x7f57a81197f0 con 0x7f57a8104450 2024-02-13T03:24:26.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57940033f0 con 0x7f57a8104450 2024-02-13T03:24:26.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556638644 0 0) 0x7f57a8117410 con 0x7f57a80f8ac0 2024-02-13T03:24:26.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a81197f0 con 0x7f57a80f8ac0 2024-02-13T03:24:26.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004041281 0 0) 0x7f57a81185f0 con 0x7f57a80f9670 2024-02-13T03:24:26.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a8117410 con 0x7f57a80f9670 2024-02-13T03:24:26.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2603878808 0 0) 0x7f57940033f0 con 0x7f57a8104450 2024-02-13T03:24:26.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f57a81185f0 con 0x7f57a8104450 2024-02-13T03:24:26.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.850+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f57a0000fe0 con 0x7f57a8104450 2024-02-13T03:24:26.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2329205451 0 0) 0x7f57a8117410 con 0x7f57a80f9670 2024-02-13T03:24:26.860 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f57940033f0 con 0x7f57a80f9670 2024-02-13T03:24:26.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5790001c40 con 0x7f57a80f9670 2024-02-13T03:24:26.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1740324355 0 0) 0x7f57a81197f0 con 0x7f57a80f8ac0 2024-02-13T03:24:26.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f57a8117410 con 0x7f57a80f8ac0 2024-02-13T03:24:26.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f57980019d0 con 0x7f57a80f8ac0 2024-02-13T03:24:26.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2898457419 0 0) 0x7f57a81185f0 con 0x7f57a8104450 2024-02-13T03:24:26.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 >> v1:172.21.15.183:6789/0 conn(0x7f57a80f9670 legacy=0x7f57a80ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:26.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 >> v1:172.21.15.161:6789/0 conn(0x7f57a80f8ac0 legacy=0x7f57a80ff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:26.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a811a9f0 con 0x7f57a8104450 2024-02-13T03:24:26.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f57acebc700 1 -- 172.21.15.161:0/3921879180 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a8117600 con 0x7f57a8104450 2024-02-13T03:24:26.861 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f57acebc700 1 -- 172.21.15.161:0/3921879180 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a8117a70 con 0x7f57a8104450 2024-02-13T03:24:26.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f57a00022b0 con 0x7f57a8104450 2024-02-13T03:24:26.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f57acebc700 1 -- 172.21.15.161:0/3921879180 --> v1:172.21.15.161:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f57a8105b70 con 0x7f57a8104450 2024-02-13T03:24:26.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f57a00036e0 con 0x7f57a8104450 2024-02-13T03:24:26.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f57a0003980 con 0x7f57a8104450 2024-02-13T03:24:26.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f57a00788b0 con 0x7f57a8104450 2024-02-13T03:24:26.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 --> v1:172.21.15.183:6812/1821141317 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5794070840 con 0x7f579406cb30 2024-02-13T03:24:26.863 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.854+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (unknown 2499814743 0 0) 0x7f57a0078cb0 con 0x7f57a8104450 2024-02-13T03:24:26.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.858+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== osd.7 v1:172.21.15.183:6812/1821141317 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7f5794070840 con 0x7f579406cb30 2024-02-13T03:24:26.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.878+0000 7f57acebc700 1 -- 172.21.15.161:0/3921879180 --> v1:172.21.15.183:6812/1821141317 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f57a805f850 con 0x7f579406cb30 2024-02-13T03:24:26.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.878+0000 7f579f7fe700 1 -- 172.21.15.161:0/3921879180 <== osd.7 v1:172.21.15.183:6812/1821141317 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 4004338942) 0x7f57a805f850 con 0x7f579406cb30 2024-02-13T03:24:26.885 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.878+0000 7f57acebc700 1 -- 172.21.15.161:0/3921879180 >> v1:172.21.15.183:6812/1821141317 conn(0x7f579406cb30 legacy=0x7f579406efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:26.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.878+0000 7f57acebc700 1 -- 172.21.15.161:0/3921879180 >> v1:172.21.15.161:6800/4114337139 conn(0x7f57940622e0 legacy=0x7f57940647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:26.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.878+0000 7f57acebc700 1 -- 172.21.15.161:0/3921879180 >> v1:172.21.15.161:6790/0 conn(0x7f57a8104450 legacy=0x7f57a8115ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:26.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.878+0000 7f57a6d9d700 1 -- 172.21.15.161:0/3921879180 reap_dead start 2024-02-13T03:24:26.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.878+0000 7f57acebc700 1 -- 172.21.15.161:0/3921879180 shutdown_connections 2024-02-13T03:24:26.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.878+0000 7f57acebc700 1 -- 172.21.15.161:0/3921879180 >> 172.21.15.161:0/3921879180 conn(0x7f57a80f2610 msgr2=0x7f57a81078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:26.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.878+0000 7f57acebc700 1 -- 172.21.15.161:0/3921879180 shutdown_connections 2024-02-13T03:24:26.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:26.878+0000 7f57acebc700 1 -- 172.21.15.161:0/3921879180 wait complete. 2024-02-13T03:24:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:26 smithi161 bash[27879]: audit 2024-02-13T03:24:25.660242+0000 mgr.y (mgr.24473) 130 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:26 smithi161 bash[19769]: audit 2024-02-13T03:24:25.660242+0000 mgr.y (mgr.24473) 130 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:26.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:26 smithi183 bash[20824]: audit 2024-02-13T03:24:25.660242+0000 mgr.y (mgr.24473) 130 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:27.523 INFO:teuthology.orchestra.run.smithi161.stdout:206158430252 2024-02-13T03:24:27.524 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-13T03:24:27.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:27 smithi161 bash[19769]: cluster 2024-02-13T03:24:26.623134+0000 mgr.y (mgr.24473) 131 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:27 smithi161 bash[27879]: cluster 2024-02-13T03:24:26.623134+0000 mgr.y (mgr.24473) 131 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:27.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:27 smithi183 bash[20824]: cluster 2024-02-13T03:24:26.623134+0000 mgr.y (mgr.24473) 131 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:29.700 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:24:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:24:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:24:29.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:29 smithi183 bash[20824]: cluster 2024-02-13T03:24:28.624010+0000 mgr.y (mgr.24473) 132 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-02-13T03:24:30.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:29 smithi161 bash[19769]: cluster 2024-02-13T03:24:28.624010+0000 mgr.y (mgr.24473) 132 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-02-13T03:24:30.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:29 smithi161 bash[27879]: cluster 2024-02-13T03:24:28.624010+0000 mgr.y (mgr.24473) 132 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-02-13T03:24:31.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:24:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:24:31] "GET /metrics HTTP/1.1" 200 172166 "" "Prometheus/2.33.4" 2024-02-13T03:24:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:31 smithi183 bash[20824]: cluster 2024-02-13T03:24:30.625417+0000 mgr.y (mgr.24473) 133 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:32.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:31 smithi161 bash[19769]: cluster 2024-02-13T03:24:30.625417+0000 mgr.y (mgr.24473) 133 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:32.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:31 smithi161 bash[27879]: cluster 2024-02-13T03:24:30.625417+0000 mgr.y (mgr.24473) 133 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:32.360 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:24:33.503 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:33 smithi161 bash[19769]: cluster 2024-02-13T03:24:32.626121+0000 mgr.y (mgr.24473) 134 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:33.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:33 smithi161 bash[27879]: cluster 2024-02-13T03:24:32.626121+0000 mgr.y (mgr.24473) 134 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:33.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:33 smithi183 bash[20824]: cluster 2024-02-13T03:24:32.626121+0000 mgr.y (mgr.24473) 134 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:35.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.026+0000 7f2a93529700 1 -- 172.21.15.161:0/2459929615 >> v1:172.21.15.161:6789/0 conn(0x7f2a8c106650 legacy=0x7f2a8c108ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:35.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.026+0000 7f2a93529700 1 -- 172.21.15.161:0/2459929615 shutdown_connections 2024-02-13T03:24:35.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.026+0000 7f2a93529700 1 -- 172.21.15.161:0/2459929615 >> 172.21.15.161:0/2459929615 conn(0x7f2a8c0f5960 msgr2=0x7f2a8c0f7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:35.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.026+0000 7f2a93529700 1 -- 172.21.15.161:0/2459929615 shutdown_connections 2024-02-13T03:24:35.035 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.026+0000 7f2a93529700 1 -- 172.21.15.161:0/2459929615 wait complete. 2024-02-13T03:24:35.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a93529700 1 Processor -- start 2024-02-13T03:24:35.036 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a93529700 1 -- start start 2024-02-13T03:24:35.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a93529700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2a8c119620 con 0x7f2a8c106650 2024-02-13T03:24:35.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a93529700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2a8c11a800 con 0x7f2a8c102b40 2024-02-13T03:24:35.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a93529700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2a8c11ba00 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a912c5700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f2a8c0ffcf0 0x7f2a8c0fbd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:39748/0 (socket says 172.21.15.161:39748) 2024-02-13T03:24:35.037 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a912c5700 1 -- 172.21.15.161:0/2163633803 learned_addr learned my addr 172.21.15.161:0/2163633803 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:24:35.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267232328 0 0) 0x7f2a8c11ba00 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a78003630 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191381272 0 0) 0x7f2a8c11a800 con 0x7f2a8c102b40 2024-02-13T03:24:35.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a8c11ba00 con 0x7f2a8c102b40 2024-02-13T03:24:35.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896802151 0 0) 0x7f2a8c119620 con 0x7f2a8c106650 2024-02-13T03:24:35.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a8c11a800 con 0x7f2a8c106650 2024-02-13T03:24:35.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3500094813 0 0) 0x7f2a78003630 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a8c119620 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f2a88001900 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2191234009 0 0) 0x7f2a8c11ba00 con 0x7f2a8c102b40 2024-02-13T03:24:35.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a78003630 con 0x7f2a8c102b40 2024-02-13T03:24:35.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f2a7c001a10 con 0x7f2a8c102b40 2024-02-13T03:24:35.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 815760979 0 0) 0x7f2a8c11a800 con 0x7f2a8c106650 2024-02-13T03:24:35.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a8c11ba00 con 0x7f2a8c106650 2024-02-13T03:24:35.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4053935243 0 0) 0x7f2a8c119620 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 >> v1:172.21.15.183:6789/0 conn(0x7f2a8c102b40 legacy=0x7f2a8c0fc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:35.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 >> v1:172.21.15.161:6789/0 conn(0x7f2a8c106650 legacy=0x7f2a8c117d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:35.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c11cc00 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a93529700 1 -- 172.21.15.161:0/2163633803 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8c11a9f0 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.039 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a93529700 1 -- 172.21.15.161:0/2163633803 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8c11ae60 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f2a88001cb0 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.030+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f2a88005ae0 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.040 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.034+0000 7f2a93529700 1 -- 172.21.15.161:0/2163633803 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a70004f20 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.034+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f2a88002230 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.042 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.034+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f2a88078440 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.045 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.038+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f2a8804c010 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.214+0000 7f2a93529700 1 -- 172.21.15.161:0/2163633803 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2a700051d0 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.223 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.218+0000 7f2a827fc700 1 -- 172.21.15.161:0/2163633803 <== mon.2 v1:172.21.15.161:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 594220510) 0x7f2a8804a420 con 0x7f2a8c0ffcf0 2024-02-13T03:24:35.224 INFO:teuthology.orchestra.run.smithi161.stdout:34359738470 2024-02-13T03:24:35.225 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.218+0000 7f2a93529700 1 -- 172.21.15.161:0/2163633803 >> v1:172.21.15.161:6800/4114337139 conn(0x7f2a78062350 legacy=0x7f2a78064810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:35.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.218+0000 7f2a93529700 1 -- 172.21.15.161:0/2163633803 >> v1:172.21.15.161:6790/0 conn(0x7f2a8c0ffcf0 legacy=0x7f2a8c0fbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:35.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.218+0000 7f2a93529700 1 -- 172.21.15.161:0/2163633803 shutdown_connections 2024-02-13T03:24:35.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.218+0000 7f2a93529700 1 -- 172.21.15.161:0/2163633803 >> 172.21.15.161:0/2163633803 conn(0x7f2a8c0f5960 msgr2=0x7f2a8c109ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:35.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.218+0000 7f2a93529700 1 -- 172.21.15.161:0/2163633803 shutdown_connections 2024-02-13T03:24:35.226 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:35.218+0000 7f2a93529700 1 -- 172.21.15.161:0/2163633803 wait complete. 2024-02-13T03:24:35.857 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738470 for osd.0 2024-02-13T03:24:35.858 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-13T03:24:35.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:24:35 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:24:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:35 smithi183 bash[20824]: cluster 2024-02-13T03:24:34.626847+0000 mgr.y (mgr.24473) 135 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:35 smithi183 bash[20824]: audit 2024-02-13T03:24:35.221634+0000 mon.c (mon.2) 66 : audit [DBG] from='client.? v1:172.21.15.161:0/2163633803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-13T03:24:36.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:35 smithi161 bash[27879]: cluster 2024-02-13T03:24:34.626847+0000 mgr.y (mgr.24473) 135 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:36.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:35 smithi161 bash[27879]: audit 2024-02-13T03:24:35.221634+0000 mon.c (mon.2) 66 : audit [DBG] from='client.? v1:172.21.15.161:0/2163633803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-13T03:24:36.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:35 smithi161 bash[19769]: cluster 2024-02-13T03:24:34.626847+0000 mgr.y (mgr.24473) 135 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:36.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:35 smithi161 bash[19769]: audit 2024-02-13T03:24:35.221634+0000 mon.c (mon.2) 66 : audit [DBG] from='client.? v1:172.21.15.161:0/2163633803' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-13T03:24:36.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:36 smithi183 bash[20824]: audit 2024-02-13T03:24:35.671514+0000 mgr.y (mgr.24473) 136 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:37.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:36 smithi161 bash[19769]: audit 2024-02-13T03:24:35.671514+0000 mgr.y (mgr.24473) 136 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:37.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:36 smithi161 bash[27879]: audit 2024-02-13T03:24:35.671514+0000 mgr.y (mgr.24473) 136 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:37.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:37 smithi183 bash[20824]: cluster 2024-02-13T03:24:36.628318+0000 mgr.y (mgr.24473) 137 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:38.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:37 smithi161 bash[19769]: cluster 2024-02-13T03:24:36.628318+0000 mgr.y (mgr.24473) 137 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:38.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:37 smithi161 bash[27879]: cluster 2024-02-13T03:24:36.628318+0000 mgr.y (mgr.24473) 137 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:39.702 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:24:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:24:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:24:39.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:39 smithi183 bash[20824]: cluster 2024-02-13T03:24:38.629011+0000 mgr.y (mgr.24473) 138 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:39 smithi161 bash[27879]: cluster 2024-02-13T03:24:38.629011+0000 mgr.y (mgr.24473) 138 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:40.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:39 smithi161 bash[19769]: cluster 2024-02-13T03:24:38.629011+0000 mgr.y (mgr.24473) 138 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:40.701 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:24:41.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:24:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:24:41] "GET /metrics HTTP/1.1" 200 172162 "" "Prometheus/2.33.4" 2024-02-13T03:24:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:41 smithi183 bash[20824]: cluster 2024-02-13T03:24:40.630558+0000 mgr.y (mgr.24473) 139 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:42.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:41 smithi161 bash[19769]: cluster 2024-02-13T03:24:40.630558+0000 mgr.y (mgr.24473) 139 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:42.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:41 smithi161 bash[27879]: cluster 2024-02-13T03:24:40.630558+0000 mgr.y (mgr.24473) 139 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:43.445 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.438+0000 7f77f9b65700 1 -- 172.21.15.161:0/1900017791 >> v1:172.21.15.161:6789/0 conn(0x7f77f4103360 legacy=0x7f77f41057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:43.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.438+0000 7f77f9b65700 1 -- 172.21.15.161:0/1900017791 shutdown_connections 2024-02-13T03:24:43.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.438+0000 7f77f9b65700 1 -- 172.21.15.161:0/1900017791 >> 172.21.15.161:0/1900017791 conn(0x7f77f40f2630 msgr2=0x7f77f40f4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:43.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.438+0000 7f77f9b65700 1 -- 172.21.15.161:0/1900017791 shutdown_connections 2024-02-13T03:24:43.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.438+0000 7f77f9b65700 1 -- 172.21.15.161:0/1900017791 wait complete. 2024-02-13T03:24:43.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.438+0000 7f77f9b65700 1 Processor -- start 2024-02-13T03:24:43.446 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.438+0000 7f77f9b65700 1 -- start start 2024-02-13T03:24:43.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.438+0000 7f77f9b65700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f77f4116980 con 0x7f77f4103360 2024-02-13T03:24:43.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.438+0000 7f77f9b65700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f77f4117b20 con 0x7f77f40ff850 2024-02-13T03:24:43.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.438+0000 7f77f9b65700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f77f4118cc0 con 0x7f77f40fca00 2024-02-13T03:24:43.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.438+0000 7f77f37fe700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f77f40fca00 0x7f77f410cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:40764/0 (socket says 172.21.15.161:40764) 2024-02-13T03:24:43.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.438+0000 7f77f37fe700 1 -- 172.21.15.161:0/1789265693 learned_addr learned my addr 172.21.15.161:0/1789265693 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:24:43.447 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.438+0000 7f77f3fff700 1 --1- 172.21.15.161:0/1789265693 >> v1:172.21.15.161:6789/0 conn(0x7f77f4103360 0x7f77f4115260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:55200/0 (socket says 172.21.15.161:55200) 2024-02-13T03:24:43.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281133078 0 0) 0x7f77f4116980 con 0x7f77f4103360 2024-02-13T03:24:43.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77dc0033f0 con 0x7f77f4103360 2024-02-13T03:24:43.448 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978894769 0 0) 0x7f77f4118cc0 con 0x7f77f40fca00 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77f4116980 con 0x7f77f40fca00 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559987169 0 0) 0x7f77f4117b20 con 0x7f77f40ff850 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77f4118cc0 con 0x7f77f40ff850 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1518094534 0 0) 0x7f77dc0033f0 con 0x7f77f4103360 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f77f4117b20 con 0x7f77f4103360 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f77ec002c60 con 0x7f77f4103360 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2567053367 0 0) 0x7f77f4116980 con 0x7f77f40fca00 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f77dc0033f0 con 0x7f77f40fca00 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f77e4001c80 con 0x7f77f40fca00 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1391830057 0 0) 0x7f77f4118cc0 con 0x7f77f40ff850 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f77f4116980 con 0x7f77f40ff850 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2099765465 0 0) 0x7f77f4117b20 con 0x7f77f4103360 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 >> v1:172.21.15.161:6790/0 conn(0x7f77f40fca00 legacy=0x7f77f410cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 >> v1:172.21.15.183:6789/0 conn(0x7f77f40ff850 legacy=0x7f77f4111b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:43.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4119e60 con 0x7f77f4103360 2024-02-13T03:24:43.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f9b65700 1 -- 172.21.15.161:0/1789265693 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f4116b70 con 0x7f77f4103360 2024-02-13T03:24:43.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f9b65700 1 -- 172.21.15.161:0/1789265693 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f4116fe0 con 0x7f77f4103360 2024-02-13T03:24:43.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f77ec002fa0 con 0x7f77f4103360 2024-02-13T03:24:43.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f9b65700 1 -- 172.21.15.161:0/1789265693 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f405f6a0 con 0x7f77f4103360 2024-02-13T03:24:43.450 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f77ec003880 con 0x7f77f4103360 2024-02-13T03:24:43.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.442+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f77ec003b00 con 0x7f77f4103360 2024-02-13T03:24:43.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.446+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f77ec078a10 con 0x7f77f4103360 2024-02-13T03:24:43.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.454+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f77ec04d3d0 con 0x7f77f4103360 2024-02-13T03:24:43.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.634+0000 7f77f9b65700 1 -- 172.21.15.161:0/1789265693 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f77f40f8280 con 0x7f77f4103360 2024-02-13T03:24:43.642 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.634+0000 7f77f0ff9700 1 -- 172.21.15.161:0/1789265693 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 3717047276) 0x7f77ec04b840 con 0x7f77f4103360 2024-02-13T03:24:43.642 INFO:teuthology.orchestra.run.smithi161.stdout:55834574943 2024-02-13T03:24:43.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.638+0000 7f77f9b65700 1 -- 172.21.15.161:0/1789265693 >> v1:172.21.15.161:6800/4114337139 conn(0x7f77dc062350 legacy=0x7f77dc064810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:43.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.638+0000 7f77f9b65700 1 -- 172.21.15.161:0/1789265693 >> v1:172.21.15.161:6789/0 conn(0x7f77f4103360 legacy=0x7f77f4115260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:43.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.638+0000 7f77f9b65700 1 -- 172.21.15.161:0/1789265693 shutdown_connections 2024-02-13T03:24:43.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.638+0000 7f77f9b65700 1 -- 172.21.15.161:0/1789265693 >> 172.21.15.161:0/1789265693 conn(0x7f77f40f2630 msgr2=0x7f77f41067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:43.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.638+0000 7f77f9b65700 1 -- 172.21.15.161:0/1789265693 shutdown_connections 2024-02-13T03:24:43.645 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:43.638+0000 7f77f9b65700 1 -- 172.21.15.161:0/1789265693 wait complete. 2024-02-13T03:24:43.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:43 smithi161 bash[19769]: cluster 2024-02-13T03:24:42.631405+0000 mgr.y (mgr.24473) 140 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:43 smithi161 bash[27879]: cluster 2024-02-13T03:24:42.631405+0000 mgr.y (mgr.24473) 140 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:43.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:43 smithi183 bash[20824]: cluster 2024-02-13T03:24:42.631405+0000 mgr.y (mgr.24473) 140 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:44.316 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574930 got 55834574943 for osd.1 2024-02-13T03:24:44.316 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-13T03:24:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:44 smithi161 bash[27879]: audit 2024-02-13T03:24:43.640386+0000 mon.a (mon.0) 685 : audit [DBG] from='client.? v1:172.21.15.161:0/1789265693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-13T03:24:44.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:44 smithi161 bash[19769]: audit 2024-02-13T03:24:43.640386+0000 mon.a (mon.0) 685 : audit [DBG] from='client.? v1:172.21.15.161:0/1789265693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-13T03:24:44.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:44 smithi183 bash[20824]: audit 2024-02-13T03:24:43.640386+0000 mon.a (mon.0) 685 : audit [DBG] from='client.? v1:172.21.15.161:0/1789265693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-13T03:24:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:45 smithi161 bash[27879]: cluster 2024-02-13T03:24:44.632222+0000 mgr.y (mgr.24473) 141 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:45.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:45 smithi161 bash[19769]: cluster 2024-02-13T03:24:44.632222+0000 mgr.y (mgr.24473) 141 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:45.682 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:45 smithi183 bash[20824]: cluster 2024-02-13T03:24:44.632222+0000 mgr.y (mgr.24473) 141 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:45.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:24:45 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:24:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:46 smithi161 bash[27879]: audit 2024-02-13T03:24:45.679517+0000 mgr.y (mgr.24473) 142 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:46.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:46 smithi161 bash[19769]: audit 2024-02-13T03:24:45.679517+0000 mgr.y (mgr.24473) 142 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:46.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:46 smithi183 bash[20824]: audit 2024-02-13T03:24:45.679517+0000 mgr.y (mgr.24473) 142 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:47 smithi161 bash[27879]: cluster 2024-02-13T03:24:46.633802+0000 mgr.y (mgr.24473) 143 : cluster [DBG] pgmap v101: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:47 smithi161 bash[19769]: cluster 2024-02-13T03:24:46.633802+0000 mgr.y (mgr.24473) 143 : cluster [DBG] pgmap v101: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:47.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:47 smithi183 bash[20824]: cluster 2024-02-13T03:24:46.633802+0000 mgr.y (mgr.24473) 143 : cluster [DBG] pgmap v101: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:49.155 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:24:49.698 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:24:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:24:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:24:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:49 smithi183 bash[20824]: cluster 2024-02-13T03:24:48.634892+0000 mgr.y (mgr.24473) 144 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:50.030 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:49 smithi161 bash[27879]: cluster 2024-02-13T03:24:48.634892+0000 mgr.y (mgr.24473) 144 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:50.031 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:49 smithi161 bash[19769]: cluster 2024-02-13T03:24:48.634892+0000 mgr.y (mgr.24473) 144 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:51.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:24:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:24:51] "GET /metrics HTTP/1.1" 200 172160 "" "Prometheus/2.33.4" 2024-02-13T03:24:51.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.838+0000 7f55589fa700 1 -- 172.21.15.161:0/3326397754 >> v1:172.21.15.161:6790/0 conn(0x7f55540f95b0 legacy=0x7f5554102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:51.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.838+0000 7f55589fa700 1 -- 172.21.15.161:0/3326397754 shutdown_connections 2024-02-13T03:24:51.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.838+0000 7f55589fa700 1 -- 172.21.15.161:0/3326397754 >> 172.21.15.161:0/3326397754 conn(0x7f55540f2590 msgr2=0x7f55540f49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:51.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.838+0000 7f55589fa700 1 -- 172.21.15.161:0/3326397754 shutdown_connections 2024-02-13T03:24:51.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.838+0000 7f55589fa700 1 -- 172.21.15.161:0/3326397754 wait complete. 2024-02-13T03:24:51.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.838+0000 7f55589fa700 1 Processor -- start 2024-02-13T03:24:51.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.838+0000 7f55589fa700 1 -- start start 2024-02-13T03:24:51.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.838+0000 7f55589fa700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f55541174a0 con 0x7f55540f95b0 2024-02-13T03:24:51.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.838+0000 7f55589fa700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5554118680 con 0x7f55541043c0 2024-02-13T03:24:51.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.838+0000 7f55589fa700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5554119880 con 0x7f55540f8ac0 2024-02-13T03:24:51.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.838+0000 7f555259c700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f55540f8ac0 0x7f55540ff830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:37520/0 (socket says 172.21.15.161:37520) 2024-02-13T03:24:51.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f5551d9b700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f55540f95b0 0x7f55540fff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39786/0 (socket says 172.21.15.161:39786) 2024-02-13T03:24:51.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f555259c700 1 -- 172.21.15.161:0/2302499448 learned_addr learned my addr 172.21.15.161:0/2302499448 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:24:51.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885058825 0 0) 0x7f55541174a0 con 0x7f55540f95b0 2024-02-13T03:24:51.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f553c0033f0 con 0x7f55540f95b0 2024-02-13T03:24:51.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948303558 0 0) 0x7f5554119880 con 0x7f55540f8ac0 2024-02-13T03:24:51.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55541174a0 con 0x7f55540f8ac0 2024-02-13T03:24:51.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620602764 0 0) 0x7f5554118680 con 0x7f55541043c0 2024-02-13T03:24:51.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5554119880 con 0x7f55541043c0 2024-02-13T03:24:51.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2052018722 0 0) 0x7f553c0033f0 con 0x7f55540f95b0 2024-02-13T03:24:51.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5554118680 con 0x7f55540f95b0 2024-02-13T03:24:51.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5544002d20 con 0x7f55540f95b0 2024-02-13T03:24:51.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1229559595 0 0) 0x7f5554119880 con 0x7f55541043c0 2024-02-13T03:24:51.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f553c0033f0 con 0x7f55541043c0 2024-02-13T03:24:51.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f554c001c40 con 0x7f55541043c0 2024-02-13T03:24:51.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1012217696 0 0) 0x7f55541174a0 con 0x7f55540f8ac0 2024-02-13T03:24:51.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5554119880 con 0x7f55540f8ac0 2024-02-13T03:24:51.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5548001940 con 0x7f55540f8ac0 2024-02-13T03:24:51.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 86232538 0 0) 0x7f5554118680 con 0x7f55540f95b0 2024-02-13T03:24:51.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 >> v1:172.21.15.161:6790/0 conn(0x7f55540f8ac0 legacy=0x7f55540ff830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:51.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 >> v1:172.21.15.183:6789/0 conn(0x7f55541043c0 legacy=0x7f5554115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:51.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555411aa80 con 0x7f55540f95b0 2024-02-13T03:24:51.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55589fa700 1 -- 172.21.15.161:0/2302499448 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5554119a70 con 0x7f55540f95b0 2024-02-13T03:24:51.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55589fa700 1 -- 172.21.15.161:0/2302499448 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5554119ee0 con 0x7f55540f95b0 2024-02-13T03:24:51.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f5544002fe0 con 0x7f55540f95b0 2024-02-13T03:24:51.850 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f55440036c0 con 0x7f55540f95b0 2024-02-13T03:24:51.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.842+0000 7f55589fa700 1 -- 172.21.15.161:0/2302499448 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5534004f20 con 0x7f55540f95b0 2024-02-13T03:24:51.851 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.846+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f5544003900 con 0x7f55540f95b0 2024-02-13T03:24:51.852 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.846+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f5544078750 con 0x7f55540f95b0 2024-02-13T03:24:51.859 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:51.850+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f554404c380 con 0x7f55540f95b0 2024-02-13T03:24:51.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:51 smithi161 bash[27879]: cluster 2024-02-13T03:24:50.636285+0000 mgr.y (mgr.24473) 145 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:51.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:51 smithi161 bash[19769]: cluster 2024-02-13T03:24:50.636285+0000 mgr.y (mgr.24473) 145 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:51.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:51 smithi183 bash[20824]: cluster 2024-02-13T03:24:50.636285+0000 mgr.y (mgr.24473) 145 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:52.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:52.034+0000 7f55589fa700 1 -- 172.21.15.161:0/2302499448 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f55340051d0 con 0x7f55540f95b0 2024-02-13T03:24:52.041 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:52.034+0000 7f55437fe700 1 -- 172.21.15.161:0/2302499448 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 2983928745) 0x7f554404a7f0 con 0x7f55540f95b0 2024-02-13T03:24:52.042 INFO:teuthology.orchestra.run.smithi161.stdout:77309411419 2024-02-13T03:24:52.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:52.038+0000 7f55589fa700 1 -- 172.21.15.161:0/2302499448 >> v1:172.21.15.161:6800/4114337139 conn(0x7f553c0622f0 legacy=0x7f553c0647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:52.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:52.038+0000 7f55589fa700 1 -- 172.21.15.161:0/2302499448 >> v1:172.21.15.161:6789/0 conn(0x7f55540f95b0 legacy=0x7f55540fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:24:52.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:52.038+0000 7f55589fa700 1 -- 172.21.15.161:0/2302499448 shutdown_connections 2024-02-13T03:24:52.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:52.038+0000 7f55589fa700 1 -- 172.21.15.161:0/2302499448 >> 172.21.15.161:0/2302499448 conn(0x7f55540f2590 msgr2=0x7f5554107830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:24:52.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:52.038+0000 7f55589fa700 1 -- 172.21.15.161:0/2302499448 shutdown_connections 2024-02-13T03:24:52.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:24:52.038+0000 7f55589fa700 1 -- 172.21.15.161:0/2302499448 wait complete. 2024-02-13T03:24:52.681 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411405 got 77309411419 for osd.2 2024-02-13T03:24:52.682 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-13T03:24:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:52 smithi183 bash[20824]: audit 2024-02-13T03:24:52.039754+0000 mon.a (mon.0) 686 : audit [DBG] from='client.? v1:172.21.15.161:0/2302499448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-13T03:24:53.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:52 smithi161 bash[19769]: audit 2024-02-13T03:24:52.039754+0000 mon.a (mon.0) 686 : audit [DBG] from='client.? v1:172.21.15.161:0/2302499448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-13T03:24:53.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:52 smithi161 bash[27879]: audit 2024-02-13T03:24:52.039754+0000 mon.a (mon.0) 686 : audit [DBG] from='client.? v1:172.21.15.161:0/2302499448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-13T03:24:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:53 smithi183 bash[20824]: cluster 2024-02-13T03:24:52.637125+0000 mgr.y (mgr.24473) 146 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:54.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:53 smithi161 bash[27879]: cluster 2024-02-13T03:24:52.637125+0000 mgr.y (mgr.24473) 146 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:53 smithi161 bash[19769]: cluster 2024-02-13T03:24:52.637125+0000 mgr.y (mgr.24473) 146 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:55.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:55 smithi183 bash[20824]: cluster 2024-02-13T03:24:54.637918+0000 mgr.y (mgr.24473) 147 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:55.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:24:55 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:24:56.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:55 smithi161 bash[27879]: cluster 2024-02-13T03:24:54.637918+0000 mgr.y (mgr.24473) 147 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:56.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:55 smithi161 bash[19769]: cluster 2024-02-13T03:24:54.637918+0000 mgr.y (mgr.24473) 147 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:24:56.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:56 smithi183 bash[20824]: audit 2024-02-13T03:24:55.687562+0000 mgr.y (mgr.24473) 148 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:57.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:56 smithi161 bash[19769]: audit 2024-02-13T03:24:55.687562+0000 mgr.y (mgr.24473) 148 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:57.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:56 smithi161 bash[27879]: audit 2024-02-13T03:24:55.687562+0000 mgr.y (mgr.24473) 148 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:24:57.515 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:24:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:57 smithi183 bash[20824]: cluster 2024-02-13T03:24:56.639484+0000 mgr.y (mgr.24473) 149 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:58.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:57 smithi161 bash[27879]: cluster 2024-02-13T03:24:56.639484+0000 mgr.y (mgr.24473) 149 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:58.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:57 smithi161 bash[19769]: cluster 2024-02-13T03:24:56.639484+0000 mgr.y (mgr.24473) 149 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:24:59.729 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:24:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:24:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:24:59.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:24:59 smithi183 bash[20824]: cluster 2024-02-13T03:24:58.640180+0000 mgr.y (mgr.24473) 150 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:00.029 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:24:59 smithi161 bash[27879]: cluster 2024-02-13T03:24:58.640180+0000 mgr.y (mgr.24473) 150 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:00.029 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:24:59 smithi161 bash[19769]: cluster 2024-02-13T03:24:58.640180+0000 mgr.y (mgr.24473) 150 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:00.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.237+0000 7f481ed8f700 1 -- 172.21.15.161:0/3332468076 >> v1:172.21.15.161:6789/0 conn(0x7f4818103360 legacy=0x7f48181057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:00.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.237+0000 7f481ed8f700 1 -- 172.21.15.161:0/3332468076 shutdown_connections 2024-02-13T03:25:00.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.237+0000 7f481ed8f700 1 -- 172.21.15.161:0/3332468076 >> 172.21.15.161:0/3332468076 conn(0x7f48180f2630 msgr2=0x7f48180f4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:00.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.237+0000 7f481ed8f700 1 -- 172.21.15.161:0/3332468076 shutdown_connections 2024-02-13T03:25:00.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.237+0000 7f481ed8f700 1 -- 172.21.15.161:0/3332468076 wait complete. 2024-02-13T03:25:00.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f481ed8f700 1 Processor -- start 2024-02-13T03:25:00.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f481ed8f700 1 -- start start 2024-02-13T03:25:00.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f481ed8f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4818116420 con 0x7f48180fca00 2024-02-13T03:25:00.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f481ed8f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4818117600 con 0x7f4818103360 2024-02-13T03:25:00.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f481cb2b700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f48180fca00 0x7f48180f8c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:55260/0 (socket says 172.21.15.161:55260) 2024-02-13T03:25:00.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f481cb2b700 1 -- 172.21.15.161:0/3012069282 learned_addr learned my addr 172.21.15.161:0/3012069282 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:25:00.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f481ed8f700 1 -- 172.21.15.161:0/3012069282 --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4818118800 con 0x7f48180ff850 2024-02-13T03:25:00.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029007405 0 0) 0x7f4818116420 con 0x7f48180fca00 2024-02-13T03:25:00.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4800003110 con 0x7f48180fca00 2024-02-13T03:25:00.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212733450 0 0) 0x7f4818118800 con 0x7f48180ff850 2024-02-13T03:25:00.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4818116420 con 0x7f48180ff850 2024-02-13T03:25:00.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287102791 0 0) 0x7f4818117600 con 0x7f4818103360 2024-02-13T03:25:00.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4818118800 con 0x7f4818103360 2024-02-13T03:25:00.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2562026121 0 0) 0x7f4800003110 con 0x7f48180fca00 2024-02-13T03:25:00.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4818117600 con 0x7f48180fca00 2024-02-13T03:25:00.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f4808001a70 con 0x7f48180fca00 2024-02-13T03:25:00.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 91313281 0 0) 0x7f4818116420 con 0x7f48180ff850 2024-02-13T03:25:00.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4800003110 con 0x7f48180ff850 2024-02-13T03:25:00.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f480c001840 con 0x7f48180ff850 2024-02-13T03:25:00.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3754468400 0 0) 0x7f4818117600 con 0x7f48180fca00 2024-02-13T03:25:00.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 >> v1:172.21.15.161:6790/0 conn(0x7f48180ff850 legacy=0x7f48180f9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:00.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 >> v1:172.21.15.183:6789/0 conn(0x7f4818103360 legacy=0x7f4818114ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:00.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818119a00 con 0x7f48180fca00 2024-02-13T03:25:00.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f481ed8f700 1 -- 172.21.15.161:0/3012069282 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4818116610 con 0x7f48180fca00 2024-02-13T03:25:00.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f481ed8f700 1 -- 172.21.15.161:0/3012069282 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4818116b00 con 0x7f48180fca00 2024-02-13T03:25:00.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f48080024b0 con 0x7f48180fca00 2024-02-13T03:25:00.251 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.241+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f4808005a10 con 0x7f48180fca00 2024-02-13T03:25:00.252 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.245+0000 7f481ed8f700 1 -- 172.21.15.161:0/3012069282 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4804004f20 con 0x7f48180fca00 2024-02-13T03:25:00.253 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.245+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f4808002620 con 0x7f48180fca00 2024-02-13T03:25:00.254 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.245+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f4808078400 con 0x7f48180fca00 2024-02-13T03:25:00.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.253+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f480804bfd0 con 0x7f48180fca00 2024-02-13T03:25:00.439 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.429+0000 7f481ed8f700 1 -- 172.21.15.161:0/3012069282 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f48040051d0 con 0x7f48180fca00 2024-02-13T03:25:00.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.433+0000 7f4815ffb700 1 -- 172.21.15.161:0/3012069282 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (unknown 383520633 0 2832570704) 0x7f480804a440 con 0x7f48180fca00 2024-02-13T03:25:00.440 INFO:teuthology.orchestra.run.smithi161.stdout:107374182484 2024-02-13T03:25:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.433+0000 7f481ed8f700 1 -- 172.21.15.161:0/3012069282 >> v1:172.21.15.161:6800/4114337139 conn(0x7f4800062000 legacy=0x7f48000644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.433+0000 7f481ed8f700 1 -- 172.21.15.161:0/3012069282 >> v1:172.21.15.161:6789/0 conn(0x7f48180fca00 legacy=0x7f48180f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.433+0000 7f481ed8f700 1 -- 172.21.15.161:0/3012069282 shutdown_connections 2024-02-13T03:25:00.442 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.433+0000 7f481ed8f700 1 -- 172.21.15.161:0/3012069282 >> 172.21.15.161:0/3012069282 conn(0x7f48180f2630 msgr2=0x7f48180f2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:00.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.433+0000 7f481ed8f700 1 -- 172.21.15.161:0/3012069282 shutdown_connections 2024-02-13T03:25:00.443 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:00.433+0000 7f481ed8f700 1 -- 172.21.15.161:0/3012069282 wait complete. 2024-02-13T03:25:00.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:00 smithi161 bash[27879]: audit 2024-02-13T03:25:00.437786+0000 mon.a (mon.0) 687 : audit [DBG] from='client.? v1:172.21.15.161:0/3012069282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-13T03:25:00.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:00 smithi161 bash[19769]: audit 2024-02-13T03:25:00.437786+0000 mon.a (mon.0) 687 : audit [DBG] from='client.? v1:172.21.15.161:0/3012069282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-13T03:25:00.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:00 smithi183 bash[20824]: audit 2024-02-13T03:25:00.437786+0000 mon.a (mon.0) 687 : audit [DBG] from='client.? v1:172.21.15.161:0/3012069282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-13T03:25:01.077 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182471 got 107374182484 for osd.3 2024-02-13T03:25:01.077 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-13T03:25:01.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:25:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:25:01] "GET /metrics HTTP/1.1" 200 172160 "" "Prometheus/2.33.4" 2024-02-13T03:25:01.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:01 smithi183 bash[20824]: cluster 2024-02-13T03:25:00.641747+0000 mgr.y (mgr.24473) 151 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:02.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:01 smithi161 bash[27879]: cluster 2024-02-13T03:25:00.641747+0000 mgr.y (mgr.24473) 151 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:02.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:01 smithi161 bash[19769]: cluster 2024-02-13T03:25:00.641747+0000 mgr.y (mgr.24473) 151 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:03 smithi161 bash[19769]: cluster 2024-02-13T03:25:02.642430+0000 mgr.y (mgr.24473) 152 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:03 smithi161 bash[19769]: audit 2024-02-13T03:25:02.899316+0000 mon.c (mon.2) 67 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:25:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:03 smithi161 bash[19769]: audit 2024-02-13T03:25:03.217433+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:25:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:03 smithi161 bash[27879]: cluster 2024-02-13T03:25:02.642430+0000 mgr.y (mgr.24473) 152 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:03 smithi161 bash[27879]: audit 2024-02-13T03:25:02.899316+0000 mon.c (mon.2) 67 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:25:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:03 smithi161 bash[27879]: audit 2024-02-13T03:25:03.217433+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:25:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:03 smithi183 bash[20824]: cluster 2024-02-13T03:25:02.642430+0000 mgr.y (mgr.24473) 152 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:03 smithi183 bash[20824]: audit 2024-02-13T03:25:02.899316+0000 mon.c (mon.2) 67 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:25:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:03 smithi183 bash[20824]: audit 2024-02-13T03:25:03.217433+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:25:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:04 smithi161 bash[27879]: audit 2024-02-13T03:25:03.504995+0000 mon.c (mon.2) 68 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:25:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:04 smithi161 bash[27879]: audit 2024-02-13T03:25:03.507033+0000 mon.c (mon.2) 69 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:25:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:04 smithi161 bash[27879]: audit 2024-02-13T03:25:03.517941+0000 mon.a (mon.0) 689 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:25:04.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:04 smithi161 bash[19769]: audit 2024-02-13T03:25:03.504995+0000 mon.c (mon.2) 68 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:25:04.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:04 smithi161 bash[19769]: audit 2024-02-13T03:25:03.507033+0000 mon.c (mon.2) 69 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:25:04.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:04 smithi161 bash[19769]: audit 2024-02-13T03:25:03.517941+0000 mon.a (mon.0) 689 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:25:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:04 smithi183 bash[20824]: audit 2024-02-13T03:25:03.504995+0000 mon.c (mon.2) 68 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:25:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:04 smithi183 bash[20824]: audit 2024-02-13T03:25:03.507033+0000 mon.c (mon.2) 69 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:25:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:04 smithi183 bash[20824]: audit 2024-02-13T03:25:03.517941+0000 mon.a (mon.0) 689 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:25:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:05 smithi161 bash[27879]: cluster 2024-02-13T03:25:04.643117+0000 mgr.y (mgr.24473) 153 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:05 smithi161 bash[19769]: cluster 2024-02-13T03:25:04.643117+0000 mgr.y (mgr.24473) 153 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:05.698 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:05 smithi183 bash[20824]: cluster 2024-02-13T03:25:04.643117+0000 mgr.y (mgr.24473) 153 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:05.899 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:25:05.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:25:05 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:25:06.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:06 smithi161 bash[27879]: audit 2024-02-13T03:25:05.695506+0000 mgr.y (mgr.24473) 154 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:06.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:06 smithi161 bash[19769]: audit 2024-02-13T03:25:05.695506+0000 mgr.y (mgr.24473) 154 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:06.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:06 smithi183 bash[20824]: audit 2024-02-13T03:25:05.695506+0000 mgr.y (mgr.24473) 154 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:07 smithi161 bash[19769]: cluster 2024-02-13T03:25:06.644569+0000 mgr.y (mgr.24473) 155 : cluster [DBG] pgmap v111: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:07.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:07 smithi161 bash[27879]: cluster 2024-02-13T03:25:06.644569+0000 mgr.y (mgr.24473) 155 : cluster [DBG] pgmap v111: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:07.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:07 smithi183 bash[20824]: cluster 2024-02-13T03:25:06.644569+0000 mgr.y (mgr.24473) 155 : cluster [DBG] pgmap v111: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:08.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.805+0000 7f25f0942700 1 -- 172.21.15.161:0/1462282193 >> v1:172.21.15.161:6790/0 conn(0x7f25ec0f8ad0 legacy=0x7f25ec0f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:08.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.805+0000 7f25f0942700 1 -- 172.21.15.161:0/1462282193 shutdown_connections 2024-02-13T03:25:08.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.805+0000 7f25f0942700 1 -- 172.21.15.161:0/1462282193 >> 172.21.15.161:0/1462282193 conn(0x7f25ec0f2620 msgr2=0x7f25ec0f4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:08.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25f0942700 1 -- 172.21.15.161:0/1462282193 shutdown_connections 2024-02-13T03:25:08.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25f0942700 1 -- 172.21.15.161:0/1462282193 wait complete. 2024-02-13T03:25:08.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25f0942700 1 Processor -- start 2024-02-13T03:25:08.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25f0942700 1 -- start start 2024-02-13T03:25:08.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25f0942700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f25ec102e00 con 0x7f25ec0f9680 2024-02-13T03:25:08.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25f0942700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f25ec102fd0 con 0x7f25ec0f8ad0 2024-02-13T03:25:08.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25ead9d700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f25ec104460 0x7f25ec1185e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:41518/0 (socket says 172.21.15.161:41518) 2024-02-13T03:25:08.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25ead9d700 1 -- 172.21.15.161:0/1629844684 learned_addr learned my addr 172.21.15.161:0/1629844684 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:25:08.817 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25f0942700 1 -- 172.21.15.161:0/1629844684 --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f25ec11bd20 con 0x7f25ec104460 2024-02-13T03:25:08.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408517166 0 0) 0x7f25ec102e00 con 0x7f25ec0f9680 2024-02-13T03:25:08.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25d8003030 con 0x7f25ec0f9680 2024-02-13T03:25:08.818 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594644292 0 0) 0x7f25ec11bd20 con 0x7f25ec104460 2024-02-13T03:25:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ec102e00 con 0x7f25ec104460 2024-02-13T03:25:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2444204195 0 0) 0x7f25ec102fd0 con 0x7f25ec0f8ad0 2024-02-13T03:25:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ec11bd20 con 0x7f25ec0f8ad0 2024-02-13T03:25:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3446172090 0 0) 0x7f25d8003030 con 0x7f25ec0f9680 2024-02-13T03:25:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f25ec102fd0 con 0x7f25ec0f9680 2024-02-13T03:25:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f25d40017f0 con 0x7f25ec0f9680 2024-02-13T03:25:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 354151568 0 0) 0x7f25ec102e00 con 0x7f25ec104460 2024-02-13T03:25:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f25d8003030 con 0x7f25ec104460 2024-02-13T03:25:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f25dc0018c0 con 0x7f25ec104460 2024-02-13T03:25:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2168718556 0 0) 0x7f25ec11bd20 con 0x7f25ec0f8ad0 2024-02-13T03:25:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f25ec102e00 con 0x7f25ec0f8ad0 2024-02-13T03:25:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f25e4001950 con 0x7f25ec0f8ad0 2024-02-13T03:25:08.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 752265304 0 0) 0x7f25ec102fd0 con 0x7f25ec0f9680 2024-02-13T03:25:08.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 >> v1:172.21.15.161:6790/0 conn(0x7f25ec104460 legacy=0x7f25ec1185e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:08.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 >> v1:172.21.15.183:6789/0 conn(0x7f25ec0f8ad0 legacy=0x7f25ec102660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:08.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec11cec0 con 0x7f25ec0f9680 2024-02-13T03:25:08.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25f0942700 1 -- 172.21.15.161:0/1629844684 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ec11bf10 con 0x7f25ec0f9680 2024-02-13T03:25:08.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25f0942700 1 -- 172.21.15.161:0/1629844684 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ec11c380 con 0x7f25ec0f9680 2024-02-13T03:25:08.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f25d4001f00 con 0x7f25ec0f9680 2024-02-13T03:25:08.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.809+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f25d40031c0 con 0x7f25ec0f9680 2024-02-13T03:25:08.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.813+0000 7f25f0942700 1 -- 172.21.15.161:0/1629844684 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25cc004f20 con 0x7f25ec0f9680 2024-02-13T03:25:08.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.813+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f25d4001a60 con 0x7f25ec0f9680 2024-02-13T03:25:08.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.813+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f25d4075d20 con 0x7f25ec0f9680 2024-02-13T03:25:08.828 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:08.821+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f25d4049890 con 0x7f25ec0f9680 2024-02-13T03:25:09.011 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:09.001+0000 7f25f0942700 1 -- 172.21.15.161:0/1629844684 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f25cc0051d0 con 0x7f25ec0f9680 2024-02-13T03:25:09.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:09.005+0000 7f25e37fe700 1 -- 172.21.15.161:0/1629844684 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (unknown 1823589948 0 2489990547) 0x7f25d4047ca0 con 0x7f25ec0f9680 2024-02-13T03:25:09.012 INFO:teuthology.orchestra.run.smithi161.stdout:128849018957 2024-02-13T03:25:09.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:09.005+0000 7f25f0942700 1 -- 172.21.15.161:0/1629844684 >> v1:172.21.15.161:6800/4114337139 conn(0x7f25d8062390 legacy=0x7f25d8064850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:09.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:09.005+0000 7f25f0942700 1 -- 172.21.15.161:0/1629844684 >> v1:172.21.15.161:6789/0 conn(0x7f25ec0f9680 legacy=0x7f25ec114eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:09.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:09.005+0000 7f25f0942700 1 -- 172.21.15.161:0/1629844684 shutdown_connections 2024-02-13T03:25:09.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:09.005+0000 7f25f0942700 1 -- 172.21.15.161:0/1629844684 >> 172.21.15.161:0/1629844684 conn(0x7f25ec0f2620 msgr2=0x7f25ec1078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:09.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:09.005+0000 7f25f0942700 1 -- 172.21.15.161:0/1629844684 shutdown_connections 2024-02-13T03:25:09.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:09.005+0000 7f25f0942700 1 -- 172.21.15.161:0/1629844684 wait complete. 2024-02-13T03:25:09.603 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018944 got 128849018957 for osd.4 2024-02-13T03:25:09.604 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-13T03:25:09.705 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:25:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:25:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:25:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:09 smithi161 bash[27879]: cluster 2024-02-13T03:25:08.645068+0000 mgr.y (mgr.24473) 156 : cluster [DBG] pgmap v112: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:09 smithi161 bash[27879]: audit 2024-02-13T03:25:09.009695+0000 mon.a (mon.0) 690 : audit [DBG] from='client.? v1:172.21.15.161:0/1629844684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-13T03:25:09.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:09 smithi161 bash[19769]: cluster 2024-02-13T03:25:08.645068+0000 mgr.y (mgr.24473) 156 : cluster [DBG] pgmap v112: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:09.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:09 smithi161 bash[19769]: audit 2024-02-13T03:25:09.009695+0000 mon.a (mon.0) 690 : audit [DBG] from='client.? v1:172.21.15.161:0/1629844684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-13T03:25:09.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:09 smithi183 bash[20824]: cluster 2024-02-13T03:25:08.645068+0000 mgr.y (mgr.24473) 156 : cluster [DBG] pgmap v112: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:09.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:09 smithi183 bash[20824]: audit 2024-02-13T03:25:09.009695+0000 mon.a (mon.0) 690 : audit [DBG] from='client.? v1:172.21.15.161:0/1629844684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-13T03:25:11.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:25:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:25:11] "GET /metrics HTTP/1.1" 200 172158 "" "Prometheus/2.33.4" 2024-02-13T03:25:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:11 smithi183 bash[20824]: cluster 2024-02-13T03:25:10.646449+0000 mgr.y (mgr.24473) 157 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:11 smithi161 bash[27879]: cluster 2024-02-13T03:25:10.646449+0000 mgr.y (mgr.24473) 157 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:12.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:11 smithi161 bash[19769]: cluster 2024-02-13T03:25:10.646449+0000 mgr.y (mgr.24473) 157 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:13.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:13 smithi161 bash[19769]: cluster 2024-02-13T03:25:12.647338+0000 mgr.y (mgr.24473) 158 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:13 smithi161 bash[27879]: cluster 2024-02-13T03:25:12.647338+0000 mgr.y (mgr.24473) 158 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:13 smithi183 bash[20824]: cluster 2024-02-13T03:25:12.647338+0000 mgr.y (mgr.24473) 158 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:14.453 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:25:15.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:15 smithi161 bash[19769]: cluster 2024-02-13T03:25:14.648081+0000 mgr.y (mgr.24473) 159 : cluster [DBG] pgmap v115: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:15.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:15 smithi161 bash[27879]: cluster 2024-02-13T03:25:14.648081+0000 mgr.y (mgr.24473) 159 : cluster [DBG] pgmap v115: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:15.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:15 smithi183 bash[20824]: cluster 2024-02-13T03:25:14.648081+0000 mgr.y (mgr.24473) 159 : cluster [DBG] pgmap v115: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:15.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:25:15 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:25:16.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:16 smithi183 bash[20824]: audit 2024-02-13T03:25:15.706871+0000 mgr.y (mgr.24473) 160 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:17.019 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:16 smithi161 bash[27879]: audit 2024-02-13T03:25:15.706871+0000 mgr.y (mgr.24473) 160 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:17.019 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:16 smithi161 bash[19769]: audit 2024-02-13T03:25:15.706871+0000 mgr.y (mgr.24473) 160 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:17.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6549f76700 1 -- 172.21.15.161:0/394028474 >> v1:172.21.15.161:6790/0 conn(0x7f65440f68b0 legacy=0x7f65440f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:17.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6549f76700 1 -- 172.21.15.161:0/394028474 shutdown_connections 2024-02-13T03:25:17.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6549f76700 1 -- 172.21.15.161:0/394028474 >> 172.21.15.161:0/394028474 conn(0x7f65440f2610 msgr2=0x7f65440f4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:17.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6549f76700 1 -- 172.21.15.161:0/394028474 shutdown_connections 2024-02-13T03:25:17.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6549f76700 1 -- 172.21.15.161:0/394028474 wait complete. 2024-02-13T03:25:17.161 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6549f76700 1 Processor -- start 2024-02-13T03:25:17.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6549f76700 1 -- start start 2024-02-13T03:25:17.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6549f76700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6544119830 con 0x7f6544106780 2024-02-13T03:25:17.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6549f76700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f654411aa10 con 0x7f65440f68b0 2024-02-13T03:25:17.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6549f76700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f654411bc10 con 0x7f65440f73a0 2024-02-13T03:25:17.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6542ffd700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f65440f73a0 0x7f6544102430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:55608/0 (socket says 172.21.15.161:55608) 2024-02-13T03:25:17.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6542ffd700 1 -- 172.21.15.161:0/489450373 learned_addr learned my addr 172.21.15.161:0/489450373 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:25:17.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178848851 0 0) 0x7f654411bc10 con 0x7f65440f73a0 2024-02-13T03:25:17.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65300033f0 con 0x7f65440f73a0 2024-02-13T03:25:17.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673863280 0 0) 0x7f6544119830 con 0x7f6544106780 2024-02-13T03:25:17.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.157+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f654411bc10 con 0x7f6544106780 2024-02-13T03:25:17.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591753543 0 0) 0x7f654411aa10 con 0x7f65440f68b0 2024-02-13T03:25:17.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6544119830 con 0x7f65440f68b0 2024-02-13T03:25:17.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 935432009 0 0) 0x7f654411bc10 con 0x7f6544106780 2024-02-13T03:25:17.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f654411aa10 con 0x7f6544106780 2024-02-13T03:25:17.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f65340018c0 con 0x7f6544106780 2024-02-13T03:25:17.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2016877973 0 0) 0x7f65300033f0 con 0x7f65440f73a0 2024-02-13T03:25:17.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f654411bc10 con 0x7f65440f73a0 2024-02-13T03:25:17.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f652c001980 con 0x7f65440f73a0 2024-02-13T03:25:17.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1480974504 0 0) 0x7f6544119830 con 0x7f65440f68b0 2024-02-13T03:25:17.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f65300033f0 con 0x7f65440f68b0 2024-02-13T03:25:17.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f653c002f60 con 0x7f65440f68b0 2024-02-13T03:25:17.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3824256338 0 0) 0x7f654411aa10 con 0x7f6544106780 2024-02-13T03:25:17.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 >> v1:172.21.15.161:6790/0 conn(0x7f65440f73a0 legacy=0x7f6544102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:17.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 >> v1:172.21.15.183:6789/0 conn(0x7f65440f68b0 legacy=0x7f6544101d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:17.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654411ce10 con 0x7f6544106780 2024-02-13T03:25:17.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6549f76700 1 -- 172.21.15.161:0/489450373 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6544119a20 con 0x7f6544106780 2024-02-13T03:25:17.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6549f76700 1 -- 172.21.15.161:0/489450373 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6544119e90 con 0x7f6544106780 2024-02-13T03:25:17.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f6534001ec0 con 0x7f6544106780 2024-02-13T03:25:17.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f6534003070 con 0x7f6544106780 2024-02-13T03:25:17.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6549f76700 1 -- 172.21.15.161:0/489450373 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6524004f20 con 0x7f6544106780 2024-02-13T03:25:17.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.161+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f6534001a20 con 0x7f6544106780 2024-02-13T03:25:17.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.165+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f6534075a90 con 0x7f6544106780 2024-02-13T03:25:17.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.169+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f6534049660 con 0x7f6544106780 2024-02-13T03:25:17.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.357+0000 7f6549f76700 1 -- 172.21.15.161:0/489450373 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f65240051d0 con 0x7f6544106780 2024-02-13T03:25:17.361 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.357+0000 7f6540ff9700 1 -- 172.21.15.161:0/489450373 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (unknown 2131975755 0 3825321978) 0x7f6534047a70 con 0x7f6544106780 2024-02-13T03:25:17.361 INFO:teuthology.orchestra.run.smithi161.stdout:154618822727 2024-02-13T03:25:17.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.361+0000 7f6549f76700 1 -- 172.21.15.161:0/489450373 >> v1:172.21.15.161:6800/4114337139 conn(0x7f6530062320 legacy=0x7f65300647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:17.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.361+0000 7f6549f76700 1 -- 172.21.15.161:0/489450373 >> v1:172.21.15.161:6789/0 conn(0x7f6544106780 legacy=0x7f6544117ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:17.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.361+0000 7f6549f76700 1 -- 172.21.15.161:0/489450373 shutdown_connections 2024-02-13T03:25:17.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.361+0000 7f6549f76700 1 -- 172.21.15.161:0/489450373 >> 172.21.15.161:0/489450373 conn(0x7f65440f2610 msgr2=0x7f6544109c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:17.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.361+0000 7f6549f76700 1 -- 172.21.15.161:0/489450373 shutdown_connections 2024-02-13T03:25:17.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:17.361+0000 7f6549f76700 1 -- 172.21.15.161:0/489450373 wait complete. 2024-02-13T03:25:17.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:17 smithi183 bash[20824]: cluster 2024-02-13T03:25:16.649628+0000 mgr.y (mgr.24473) 161 : cluster [DBG] pgmap v116: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:17.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:17 smithi183 bash[20824]: audit 2024-02-13T03:25:17.359497+0000 mon.a (mon.0) 691 : audit [DBG] from='client.? v1:172.21.15.161:0/489450373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-13T03:25:17.994 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:17 smithi161 bash[19769]: cluster 2024-02-13T03:25:16.649628+0000 mgr.y (mgr.24473) 161 : cluster [DBG] pgmap v116: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:17.994 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:17 smithi161 bash[19769]: audit 2024-02-13T03:25:17.359497+0000 mon.a (mon.0) 691 : audit [DBG] from='client.? v1:172.21.15.161:0/489450373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-13T03:25:17.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:17 smithi161 bash[27879]: cluster 2024-02-13T03:25:16.649628+0000 mgr.y (mgr.24473) 161 : cluster [DBG] pgmap v116: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:17.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:17 smithi161 bash[27879]: audit 2024-02-13T03:25:17.359497+0000 mon.a (mon.0) 691 : audit [DBG] from='client.? v1:172.21.15.161:0/489450373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-13T03:25:17.995 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822713 got 154618822727 for osd.5 2024-02-13T03:25:17.995 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-13T03:25:19.724 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:25:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:25:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:25:19.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:19 smithi183 bash[20824]: cluster 2024-02-13T03:25:18.650151+0000 mgr.y (mgr.24473) 162 : cluster [DBG] pgmap v117: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:20.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:19 smithi161 bash[27879]: cluster 2024-02-13T03:25:18.650151+0000 mgr.y (mgr.24473) 162 : cluster [DBG] pgmap v117: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:20.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:19 smithi161 bash[19769]: cluster 2024-02-13T03:25:18.650151+0000 mgr.y (mgr.24473) 162 : cluster [DBG] pgmap v117: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:21.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:25:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:25:21] "GET /metrics HTTP/1.1" 200 172187 "" "Prometheus/2.33.4" 2024-02-13T03:25:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:21 smithi183 bash[20824]: cluster 2024-02-13T03:25:20.651521+0000 mgr.y (mgr.24473) 163 : cluster [DBG] pgmap v118: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:22.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:21 smithi161 bash[27879]: cluster 2024-02-13T03:25:20.651521+0000 mgr.y (mgr.24473) 163 : cluster [DBG] pgmap v118: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:22.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:21 smithi161 bash[19769]: cluster 2024-02-13T03:25:20.651521+0000 mgr.y (mgr.24473) 163 : cluster [DBG] pgmap v118: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:22.834 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:25:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:23 smithi161 bash[27879]: cluster 2024-02-13T03:25:22.652246+0000 mgr.y (mgr.24473) 164 : cluster [DBG] pgmap v119: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:23.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:23 smithi161 bash[19769]: cluster 2024-02-13T03:25:22.652246+0000 mgr.y (mgr.24473) 164 : cluster [DBG] pgmap v119: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:23 smithi183 bash[20824]: cluster 2024-02-13T03:25:22.652246+0000 mgr.y (mgr.24473) 164 : cluster [DBG] pgmap v119: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:25.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.705+0000 7fdf1c28d700 1 -- 172.21.15.161:0/2167114354 >> v1:172.21.15.161:6790/0 conn(0x7fdf140f8980 legacy=0x7fdf140f8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:25.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.705+0000 7fdf1c28d700 1 -- 172.21.15.161:0/2167114354 shutdown_connections 2024-02-13T03:25:25.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.705+0000 7fdf1c28d700 1 -- 172.21.15.161:0/2167114354 >> 172.21.15.161:0/2167114354 conn(0x7fdf1406c1e0 msgr2=0x7fdf1406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:25.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.705+0000 7fdf1c28d700 1 -- 172.21.15.161:0/2167114354 shutdown_connections 2024-02-13T03:25:25.709 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.705+0000 7fdf1c28d700 1 -- 172.21.15.161:0/2167114354 wait complete. 2024-02-13T03:25:25.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.705+0000 7fdf1c28d700 1 Processor -- start 2024-02-13T03:25:25.710 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.705+0000 7fdf1c28d700 1 -- start start 2024-02-13T03:25:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.709+0000 7fdf1c28d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdf141196d0 con 0x7fdf14106640 2024-02-13T03:25:25.711 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.709+0000 7fdf1c28d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdf1411a8b0 con 0x7fdf140f8980 2024-02-13T03:25:25.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.709+0000 7fdf19828700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7fdf14102970 0x7fdf140ff450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:55614/0 (socket says 172.21.15.161:55614) 2024-02-13T03:25:25.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.709+0000 7fdf1a029700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fdf140f8980 0x7fdf140fed20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.161:40970/0 (socket says 172.21.15.161:40970) 2024-02-13T03:25:25.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.709+0000 7fdf19828700 1 -- 172.21.15.161:0/167710766 learned_addr learned my addr 172.21.15.161:0/167710766 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:25:25.712 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.709+0000 7fdf1c28d700 1 -- 172.21.15.161:0/167710766 --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdf1411bab0 con 0x7fdf14102970 2024-02-13T03:25:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.709+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149159179 0 0) 0x7fdf1411bab0 con 0x7fdf14102970 2024-02-13T03:25:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.709+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdefc0033f0 con 0x7fdf14102970 2024-02-13T03:25:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.709+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917642474 0 0) 0x7fdf141196d0 con 0x7fdf14106640 2024-02-13T03:25:25.715 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.709+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf1411bab0 con 0x7fdf14106640 2024-02-13T03:25:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.709+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1604247665 0 0) 0x7fdefc0033f0 con 0x7fdf14102970 2024-02-13T03:25:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdf141196d0 con 0x7fdf14102970 2024-02-13T03:25:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fdf08002990 con 0x7fdf14102970 2024-02-13T03:25:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890020606 0 0) 0x7fdf1411a8b0 con 0x7fdf140f8980 2024-02-13T03:25:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdefc0033f0 con 0x7fdf140f8980 2024-02-13T03:25:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3710058122 0 0) 0x7fdf1411bab0 con 0x7fdf14106640 2024-02-13T03:25:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdf1411a8b0 con 0x7fdf14106640 2024-02-13T03:25:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fdf0c0018c0 con 0x7fdf14106640 2024-02-13T03:25:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1691794644 0 0) 0x7fdf141196d0 con 0x7fdf14102970 2024-02-13T03:25:25.716 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 >> v1:172.21.15.183:6789/0 conn(0x7fdf140f8980 legacy=0x7fdf140fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 >> v1:172.21.15.161:6789/0 conn(0x7fdf14106640 legacy=0x7fdf14117d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1411ccb0 con 0x7fdf14102970 2024-02-13T03:25:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf1c28d700 1 -- 172.21.15.161:0/167710766 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1411aaa0 con 0x7fdf14102970 2024-02-13T03:25:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf1c28d700 1 -- 172.21.15.161:0/167710766 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1411af90 con 0x7fdf14102970 2024-02-13T03:25:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fdf08002d20 con 0x7fdf14102970 2024-02-13T03:25:25.717 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fdf08005b50 con 0x7fdf14102970 2024-02-13T03:25:25.718 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf1c28d700 1 -- 172.21.15.161:0/167710766 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf00004f20 con 0x7fdf14102970 2024-02-13T03:25:25.719 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.713+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fdf080189e0 con 0x7fdf14102970 2024-02-13T03:25:25.720 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.717+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7fdf080785e0 con 0x7fdf14102970 2024-02-13T03:25:25.725 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.721+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fdf0804c1b0 con 0x7fdf14102970 2024-02-13T03:25:25.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.905+0000 7fdf1c28d700 1 -- 172.21.15.161:0/167710766 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fdf000051d0 con 0x7fdf14102970 2024-02-13T03:25:25.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.905+0000 7fdf13779700 1 -- 172.21.15.161:0/167710766 <== mon.2 v1:172.21.15.161:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (unknown 1274345170 0 864586964) 0x7fdf0804a5c0 con 0x7fdf14102970 2024-02-13T03:25:25.909 INFO:teuthology.orchestra.run.smithi161.stdout:180388626494 2024-02-13T03:25:25.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.905+0000 7fdf1c28d700 1 -- 172.21.15.161:0/167710766 >> v1:172.21.15.161:6800/4114337139 conn(0x7fdefc062340 legacy=0x7fdefc064800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:25.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.905+0000 7fdf1c28d700 1 -- 172.21.15.161:0/167710766 >> v1:172.21.15.161:6790/0 conn(0x7fdf14102970 legacy=0x7fdf140ff450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:25.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.905+0000 7fdf1c28d700 1 -- 172.21.15.161:0/167710766 shutdown_connections 2024-02-13T03:25:25.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.905+0000 7fdf1c28d700 1 -- 172.21.15.161:0/167710766 >> 172.21.15.161:0/167710766 conn(0x7fdf1406c1e0 msgr2=0x7fdf14105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:25.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.905+0000 7fdf1c28d700 1 -- 172.21.15.161:0/167710766 shutdown_connections 2024-02-13T03:25:25.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:25.909+0000 7fdf1c28d700 1 -- 172.21.15.161:0/167710766 wait complete. 2024-02-13T03:25:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:25 smithi161 bash[27879]: cluster 2024-02-13T03:25:24.652964+0000 mgr.y (mgr.24473) 165 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:25 smithi161 bash[19769]: cluster 2024-02-13T03:25:24.652964+0000 mgr.y (mgr.24473) 165 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:25 smithi183 bash[20824]: cluster 2024-02-13T03:25:24.652964+0000 mgr.y (mgr.24473) 165 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:25.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:25:25 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:25:26.568 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626481 got 180388626494 for osd.6 2024-02-13T03:25:26.568 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-13T03:25:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:26 smithi161 bash[27879]: audit 2024-02-13T03:25:25.715417+0000 mgr.y (mgr.24473) 166 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:26 smithi161 bash[27879]: audit 2024-02-13T03:25:25.906507+0000 mon.c (mon.2) 70 : audit [DBG] from='client.? v1:172.21.15.161:0/167710766' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-13T03:25:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:26 smithi161 bash[19769]: audit 2024-02-13T03:25:25.715417+0000 mgr.y (mgr.24473) 166 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:26 smithi161 bash[19769]: audit 2024-02-13T03:25:25.906507+0000 mon.c (mon.2) 70 : audit [DBG] from='client.? v1:172.21.15.161:0/167710766' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-13T03:25:26.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:26 smithi183 bash[20824]: audit 2024-02-13T03:25:25.715417+0000 mgr.y (mgr.24473) 166 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:26 smithi183 bash[20824]: audit 2024-02-13T03:25:25.906507+0000 mon.c (mon.2) 70 : audit [DBG] from='client.? v1:172.21.15.161:0/167710766' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-13T03:25:27.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:27 smithi183 bash[20824]: cluster 2024-02-13T03:25:26.654460+0000 mgr.y (mgr.24473) 167 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:28.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:27 smithi161 bash[27879]: cluster 2024-02-13T03:25:26.654460+0000 mgr.y (mgr.24473) 167 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:28.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:27 smithi161 bash[19769]: cluster 2024-02-13T03:25:26.654460+0000 mgr.y (mgr.24473) 167 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:29.726 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:25:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:25:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:25:29.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:29 smithi183 bash[20824]: cluster 2024-02-13T03:25:28.654943+0000 mgr.y (mgr.24473) 168 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:30.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:29 smithi161 bash[27879]: cluster 2024-02-13T03:25:28.654943+0000 mgr.y (mgr.24473) 168 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:30.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:29 smithi161 bash[19769]: cluster 2024-02-13T03:25:28.654943+0000 mgr.y (mgr.24473) 168 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:31.407 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:25:31.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:25:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:25:31] "GET /metrics HTTP/1.1" 200 172187 "" "Prometheus/2.33.4" 2024-02-13T03:25:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:31 smithi183 bash[20824]: cluster 2024-02-13T03:25:30.656323+0000 mgr.y (mgr.24473) 169 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:32.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:31 smithi161 bash[27879]: cluster 2024-02-13T03:25:30.656323+0000 mgr.y (mgr.24473) 169 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:32.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:31 smithi161 bash[19769]: cluster 2024-02-13T03:25:30.656323+0000 mgr.y (mgr.24473) 169 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:33 smithi161 bash[27879]: cluster 2024-02-13T03:25:32.656973+0000 mgr.y (mgr.24473) 170 : cluster [DBG] pgmap v124: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:33.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:33 smithi161 bash[19769]: cluster 2024-02-13T03:25:32.656973+0000 mgr.y (mgr.24473) 170 : cluster [DBG] pgmap v124: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:33.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:33 smithi183 bash[20824]: cluster 2024-02-13T03:25:32.656973+0000 mgr.y (mgr.24473) 170 : cluster [DBG] pgmap v124: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:34.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.333+0000 7f7d0c427700 1 -- 172.21.15.161:0/2217205280 >> v1:172.21.15.161:6790/0 conn(0x7f7d040f8b80 legacy=0x7f7d040f8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:34.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.333+0000 7f7d0c427700 1 -- 172.21.15.161:0/2217205280 shutdown_connections 2024-02-13T03:25:34.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.333+0000 7f7d0c427700 1 -- 172.21.15.161:0/2217205280 >> 172.21.15.161:0/2217205280 conn(0x7f7d040f25d0 msgr2=0x7f7d040f49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:34.339 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.333+0000 7f7d0c427700 1 -- 172.21.15.161:0/2217205280 shutdown_connections 2024-02-13T03:25:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.333+0000 7f7d0c427700 1 -- 172.21.15.161:0/2217205280 wait complete. 2024-02-13T03:25:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7d0c427700 1 Processor -- start 2024-02-13T03:25:34.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7d0c427700 1 -- start start 2024-02-13T03:25:34.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7d0c427700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7d04117280 con 0x7f7d040f8b80 2024-02-13T03:25:34.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7d0c427700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7d04118420 con 0x7f7d04104450 2024-02-13T03:25:34.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7d0c427700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7d04119620 con 0x7f7d040f9670 2024-02-13T03:25:34.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7d0a1c3700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f7d040f8b80 0x7f7d040ff430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39894/0 (socket says 172.21.15.161:39894) 2024-02-13T03:25:34.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7d0a1c3700 1 -- 172.21.15.161:0/3029832373 learned_addr learned my addr 172.21.15.161:0/3029832373 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:25:34.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104877979 0 0) 0x7f7d04117280 con 0x7f7d040f8b80 2024-02-13T03:25:34.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cf0003630 con 0x7f7d040f8b80 2024-02-13T03:25:34.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463458410 0 0) 0x7f7d04119620 con 0x7f7d040f9670 2024-02-13T03:25:34.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d04117280 con 0x7f7d040f9670 2024-02-13T03:25:34.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633397554 0 0) 0x7f7d04118420 con 0x7f7d04104450 2024-02-13T03:25:34.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d04119620 con 0x7f7d04104450 2024-02-13T03:25:34.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4237676232 0 0) 0x7f7cf0003630 con 0x7f7d040f8b80 2024-02-13T03:25:34.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7d04118420 con 0x7f7d040f8b80 2024-02-13T03:25:34.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f7d00002d80 con 0x7f7d040f8b80 2024-02-13T03:25:34.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2666091797 0 0) 0x7f7d04117280 con 0x7f7d040f9670 2024-02-13T03:25:34.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7cf0003630 con 0x7f7d040f9670 2024-02-13T03:25:34.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f7cf40019f0 con 0x7f7d040f9670 2024-02-13T03:25:34.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1750902105 0 0) 0x7f7d04119620 con 0x7f7d04104450 2024-02-13T03:25:34.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7d04117280 con 0x7f7d04104450 2024-02-13T03:25:34.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f7cfc001b40 con 0x7f7d04104450 2024-02-13T03:25:34.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4176650991 0 0) 0x7f7d04118420 con 0x7f7d040f8b80 2024-02-13T03:25:34.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 >> v1:172.21.15.161:6790/0 conn(0x7f7d040f9670 legacy=0x7f7d040ffb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:34.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 >> v1:172.21.15.183:6789/0 conn(0x7f7d04104450 legacy=0x7f7d04115ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:34.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0411a820 con 0x7f7d040f8b80 2024-02-13T03:25:34.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7d0c427700 1 -- 172.21.15.161:0/3029832373 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d04119810 con 0x7f7d040f8b80 2024-02-13T03:25:34.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7d0c427700 1 -- 172.21.15.161:0/3029832373 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d04119c80 con 0x7f7d040f8b80 2024-02-13T03:25:34.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f7d00001b60 con 0x7f7d040f8b80 2024-02-13T03:25:34.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.337+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f7d000038f0 con 0x7f7d040f8b80 2024-02-13T03:25:34.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.341+0000 7f7d0c427700 1 -- 172.21.15.161:0/3029832373 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce8004f20 con 0x7f7d040f8b80 2024-02-13T03:25:34.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.341+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f7d00003190 con 0x7f7d040f8b80 2024-02-13T03:25:34.345 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.341+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f7d00078870 con 0x7f7d040f8b80 2024-02-13T03:25:34.351 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.349+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f7d0004c420 con 0x7f7d040f8b80 2024-02-13T03:25:34.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.533+0000 7f7d0c427700 1 -- 172.21.15.161:0/3029832373 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7ce80051d0 con 0x7f7d040f8b80 2024-02-13T03:25:34.536 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.533+0000 7f7cfb7fe700 1 -- 172.21.15.161:0/3029832373 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (unknown 1482059429 0 2757319680) 0x7f7d0004a890 con 0x7f7d040f8b80 2024-02-13T03:25:34.537 INFO:teuthology.orchestra.run.smithi161.stdout:206158430265 2024-02-13T03:25:34.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.537+0000 7f7d0c427700 1 -- 172.21.15.161:0/3029832373 >> v1:172.21.15.161:6800/4114337139 conn(0x7f7cf00624c0 legacy=0x7f7cf0064980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:34.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.537+0000 7f7d0c427700 1 -- 172.21.15.161:0/3029832373 >> v1:172.21.15.161:6789/0 conn(0x7f7d040f8b80 legacy=0x7f7d040ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:34.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.537+0000 7f7d0c427700 1 -- 172.21.15.161:0/3029832373 shutdown_connections 2024-02-13T03:25:34.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.537+0000 7f7d0c427700 1 -- 172.21.15.161:0/3029832373 >> 172.21.15.161:0/3029832373 conn(0x7f7d040f25d0 msgr2=0x7f7d041068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:34.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.537+0000 7f7d0c427700 1 -- 172.21.15.161:0/3029832373 shutdown_connections 2024-02-13T03:25:34.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:34.537+0000 7f7d0c427700 1 -- 172.21.15.161:0/3029832373 wait complete. 2024-02-13T03:25:34.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:34 smithi161 bash[19769]: audit 2024-02-13T03:25:34.534557+0000 mon.a (mon.0) 692 : audit [DBG] from='client.? v1:172.21.15.161:0/3029832373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-13T03:25:34.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:34 smithi161 bash[27879]: audit 2024-02-13T03:25:34.534557+0000 mon.a (mon.0) 692 : audit [DBG] from='client.? v1:172.21.15.161:0/3029832373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-13T03:25:34.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:34 smithi183 bash[20824]: audit 2024-02-13T03:25:34.534557+0000 mon.a (mon.0) 692 : audit [DBG] from='client.? v1:172.21.15.161:0/3029832373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-13T03:25:35.156 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430252 got 206158430265 for osd.7 2024-02-13T03:25:35.156 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-13T03:25:35.156 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-13T03:25:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:35 smithi161 bash[27879]: cluster 2024-02-13T03:25:34.657464+0000 mgr.y (mgr.24473) 171 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:35.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:35 smithi161 bash[19769]: cluster 2024-02-13T03:25:34.657464+0000 mgr.y (mgr.24473) 171 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:35.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:25:35 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:25:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:35 smithi183 bash[20824]: cluster 2024-02-13T03:25:34.657464+0000 mgr.y (mgr.24473) 171 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:36 smithi161 bash[19769]: audit 2024-02-13T03:25:35.726701+0000 mgr.y (mgr.24473) 172 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:36 smithi161 bash[27879]: audit 2024-02-13T03:25:35.726701+0000 mgr.y (mgr.24473) 172 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:36 smithi183 bash[20824]: audit 2024-02-13T03:25:35.726701+0000 mgr.y (mgr.24473) 172 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:37.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:37 smithi161 bash[19769]: cluster 2024-02-13T03:25:36.658891+0000 mgr.y (mgr.24473) 173 : cluster [DBG] pgmap v126: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:37.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:37 smithi161 bash[27879]: cluster 2024-02-13T03:25:36.658891+0000 mgr.y (mgr.24473) 173 : cluster [DBG] pgmap v126: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:37.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:37 smithi183 bash[20824]: cluster 2024-02-13T03:25:36.658891+0000 mgr.y (mgr.24473) 173 : cluster [DBG] pgmap v126: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:39.721 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:25:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:25:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:25:39.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:39 smithi183 bash[20824]: cluster 2024-02-13T03:25:38.659478+0000 mgr.y (mgr.24473) 174 : cluster [DBG] pgmap v127: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:39.999 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:25:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:39 smithi161 bash[27879]: cluster 2024-02-13T03:25:38.659478+0000 mgr.y (mgr.24473) 174 : cluster [DBG] pgmap v127: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:40.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:39 smithi161 bash[19769]: cluster 2024-02-13T03:25:38.659478+0000 mgr.y (mgr.24473) 174 : cluster [DBG] pgmap v127: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:41.245 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:25:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:25:41] "GET /metrics HTTP/1.1" 200 172185 "" "Prometheus/2.33.4" 2024-02-13T03:25:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:41 smithi183 bash[20824]: cluster 2024-02-13T03:25:40.660857+0000 mgr.y (mgr.24473) 175 : cluster [DBG] pgmap v128: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:42.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:41 smithi161 bash[19769]: cluster 2024-02-13T03:25:40.660857+0000 mgr.y (mgr.24473) 175 : cluster [DBG] pgmap v128: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:42.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:41 smithi161 bash[27879]: cluster 2024-02-13T03:25:40.660857+0000 mgr.y (mgr.24473) 175 : cluster [DBG] pgmap v128: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:42.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.969+0000 7f1058249700 1 -- 172.21.15.161:0/2127789324 >> v1:172.21.15.161:6789/0 conn(0x7f105006b920 legacy=0x7f1050109680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:42.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.969+0000 7f1058249700 1 -- 172.21.15.161:0/2127789324 shutdown_connections 2024-02-13T03:25:42.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.969+0000 7f1058249700 1 -- 172.21.15.161:0/2127789324 >> 172.21.15.161:0/2127789324 conn(0x7f10500f3710 msgr2=0x7f10500f5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:42.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.969+0000 7f1058249700 1 -- 172.21.15.161:0/2127789324 shutdown_connections 2024-02-13T03:25:42.974 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.969+0000 7f1058249700 1 -- 172.21.15.161:0/2127789324 wait complete. 2024-02-13T03:25:42.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1058249700 1 Processor -- start 2024-02-13T03:25:42.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1058249700 1 -- start start 2024-02-13T03:25:42.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1058249700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f105011d920 con 0x7f105010ada0 2024-02-13T03:25:42.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1058249700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f105011eb00 con 0x7f105006ccc0 2024-02-13T03:25:42.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1058249700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f105011fd00 con 0x7f105006b920 2024-02-13T03:25:42.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1055fe5700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f105006b920 0x7f1050106f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:35696/0 (socket says 172.21.15.161:35696) 2024-02-13T03:25:42.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f10567e6700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f105010ada0 0x7f105011c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:37564/0 (socket says 172.21.15.161:37564) 2024-02-13T03:25:42.976 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1055fe5700 1 -- 172.21.15.161:0/361032225 learned_addr learned my addr 172.21.15.161:0/361032225 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:25:42.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287268102 0 0) 0x7f105011fd00 con 0x7f105006b920 2024-02-13T03:25:42.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f103c003630 con 0x7f105006b920 2024-02-13T03:25:42.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1301399536 0 0) 0x7f105011d920 con 0x7f105010ada0 2024-02-13T03:25:42.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f105011fd00 con 0x7f105010ada0 2024-02-13T03:25:42.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547203665 0 0) 0x7f105011eb00 con 0x7f105006ccc0 2024-02-13T03:25:42.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f105011d920 con 0x7f105006ccc0 2024-02-13T03:25:42.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2226166883 0 0) 0x7f103c003630 con 0x7f105006b920 2024-02-13T03:25:42.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f105011eb00 con 0x7f105006b920 2024-02-13T03:25:42.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f104c001d50 con 0x7f105006b920 2024-02-13T03:25:42.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1840555758 0 0) 0x7f105011eb00 con 0x7f105006b920 2024-02-13T03:25:42.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 >> v1:172.21.15.183:6789/0 conn(0x7f105006ccc0 legacy=0x7f1050107660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:42.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 >> v1:172.21.15.161:6789/0 conn(0x7f105010ada0 legacy=0x7f105011c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:42.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1050120f00 con 0x7f105006b920 2024-02-13T03:25:42.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1058249700 1 -- 172.21.15.161:0/361032225 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105011db10 con 0x7f105006b920 2024-02-13T03:25:42.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1058249700 1 -- 172.21.15.161:0/361032225 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f105011df80 con 0x7f105006b920 2024-02-13T03:25:42.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f104c0027b0 con 0x7f105006b920 2024-02-13T03:25:42.978 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f104c005e40 con 0x7f105006b920 2024-02-13T03:25:42.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1058249700 1 -- 172.21.15.161:0/361032225 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1034004f50 con 0x7f105006b920 2024-02-13T03:25:42.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.973+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f104c018cd0 con 0x7f105006b920 2024-02-13T03:25:42.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.977+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f104c078940 con 0x7f105006b920 2024-02-13T03:25:42.986 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:42.981+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f104c048820 con 0x7f105006b920 2024-02-13T03:25:43.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:43.157+0000 7f1058249700 1 -- 172.21.15.161:0/361032225 --> v1:172.21.15.161:6800/4114337139 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1034000bc0 con 0x7f103c061f20 2024-02-13T03:25:43.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:43.181+0000 7f1046ffd700 1 -- 172.21.15.161:0/361032225 <== mgr.24473 v1:172.21.15.161:6800/4114337139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+355168 (unknown 2965378022 0 1320451467) 0x7f1034000bc0 con 0x7f103c061f20 2024-02-13T03:25:43.189 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-02-13T03:25:43.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:43.185+0000 7f1058249700 1 -- 172.21.15.161:0/361032225 >> v1:172.21.15.161:6800/4114337139 conn(0x7f103c061f20 legacy=0x7f103c0643e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:43.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:43.185+0000 7f1058249700 1 -- 172.21.15.161:0/361032225 >> v1:172.21.15.161:6790/0 conn(0x7f105006b920 legacy=0x7f1050106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:43.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:43.185+0000 7f1058249700 1 -- 172.21.15.161:0/361032225 shutdown_connections 2024-02-13T03:25:43.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:43.185+0000 7f1058249700 1 -- 172.21.15.161:0/361032225 >> 172.21.15.161:0/361032225 conn(0x7f10500f3710 msgr2=0x7f10500f4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:43.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:43.185+0000 7f1058249700 1 -- 172.21.15.161:0/361032225 shutdown_connections 2024-02-13T03:25:43.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:43.185+0000 7f1058249700 1 -- 172.21.15.161:0/361032225 wait complete. 2024-02-13T03:25:43.193 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-02-13T03:25:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:43 smithi161 bash[27879]: cluster 2024-02-13T03:25:42.661430+0000 mgr.y (mgr.24473) 176 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:43 smithi161 bash[27879]: audit 2024-02-13T03:25:43.161915+0000 mgr.y (mgr.24473) 177 : audit [DBG] from='client.24671 v1:172.21.15.161:0/361032225' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T03:25:43.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:43 smithi161 bash[19769]: cluster 2024-02-13T03:25:42.661430+0000 mgr.y (mgr.24473) 176 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:43.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:43 smithi161 bash[19769]: audit 2024-02-13T03:25:43.161915+0000 mgr.y (mgr.24473) 177 : audit [DBG] from='client.24671 v1:172.21.15.161:0/361032225' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T03:25:43.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:43 smithi183 bash[20824]: cluster 2024-02-13T03:25:42.661430+0000 mgr.y (mgr.24473) 176 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:43.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:43 smithi183 bash[20824]: audit 2024-02-13T03:25:43.161915+0000 mgr.y (mgr.24473) 177 : audit [DBG] from='client.24671 v1:172.21.15.161:0/361032225' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T03:25:43.819 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-02-13T03:25:42.661178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1368,"num_read_kb":1088,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":659,"ondisk_log_size":659,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2381748,"kb_used_data":4980,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747579468,"statfs":{"total":767960285184,"available":765521375232,"internally_reserved":0,"allocated":5099520,"data_stored":1929553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005939"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990699+0000","last_change":"2024-02-13T03:21:15.714391+0000","last_active":"2024-02-13T03:22:34.990699+0000","last_peered":"2024-02-13T03:22:34.990699+0000","last_clean":"2024-02-13T03:22:34.990699+0000","last_became_active":"2024-02-13T03:21:15.714078+0000","last_became_peered":"2024-02-13T03:21:15.714078+0000","last_unstale":"2024-02-13T03:22:34.990699+0000","last_undegraded":"2024-02-13T03:22:34.990699+0000","last_fullsized":"2024-02-13T03:22:34.990699+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991314+0000","last_change":"2024-02-13T03:21:13.690573+0000","last_active":"2024-02-13T03:22:34.991314+0000","last_peered":"2024-02-13T03:22:34.991314+0000","last_clean":"2024-02-13T03:22:34.991314+0000","last_became_active":"2024-02-13T03:21:13.690185+0000","last_became_peered":"2024-02-13T03:21:13.690185+0000","last_unstale":"2024-02-13T03:22:34.991314+0000","last_undegraded":"2024-02-13T03:22:34.991314+0000","last_fullsized":"2024-02-13T03:22:34.991314+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985536+0000","last_change":"2024-02-13T03:21:09.891039+0000","last_active":"2024-02-13T03:22:34.985536+0000","last_peered":"2024-02-13T03:22:34.985536+0000","last_clean":"2024-02-13T03:22:34.985536+0000","last_became_active":"2024-02-13T03:21:09.890276+0000","last_became_peered":"2024-02-13T03:21:09.890276+0000","last_unstale":"2024-02-13T03:22:34.985536+0000","last_undegraded":"2024-02-13T03:22:34.985536+0000","last_fullsized":"2024-02-13T03:22:34.985536+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"60'15","reported_seq":48,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990605+0000","last_change":"2024-02-13T03:21:11.682044+0000","last_active":"2024-02-13T03:22:34.990605+0000","last_peered":"2024-02-13T03:22:34.990605+0000","last_clean":"2024-02-13T03:22:34.990605+0000","last_became_active":"2024-02-13T03:21:11.681785+0000","last_became_peered":"2024-02-13T03:21:11.681785+0000","last_unstale":"2024-02-13T03:22:34.990605+0000","last_undegraded":"2024-02-13T03:22:34.990605+0000","last_fullsized":"2024-02-13T03:22:34.990605+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991900+0000","last_change":"2024-02-13T03:21:15.705996+0000","last_active":"2024-02-13T03:22:34.991900+0000","last_peered":"2024-02-13T03:22:34.991900+0000","last_clean":"2024-02-13T03:22:34.991900+0000","last_became_active":"2024-02-13T03:21:15.705646+0000","last_became_peered":"2024-02-13T03:21:15.705646+0000","last_unstale":"2024-02-13T03:22:34.991900+0000","last_undegraded":"2024-02-13T03:22:34.991900+0000","last_fullsized":"2024-02-13T03:22:34.991900+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990548+0000","last_change":"2024-02-13T03:21:13.687658+0000","last_active":"2024-02-13T03:22:34.990548+0000","last_peered":"2024-02-13T03:22:34.990548+0000","last_clean":"2024-02-13T03:22:34.990548+0000","last_became_active":"2024-02-13T03:21:13.687359+0000","last_became_peered":"2024-02-13T03:21:13.687359+0000","last_unstale":"2024-02-13T03:22:34.990548+0000","last_undegraded":"2024-02-13T03:22:34.990548+0000","last_fullsized":"2024-02-13T03:22:34.990548+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"53'1","reported_seq":26,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992041+0000","last_change":"2024-02-13T03:21:09.890345+0000","last_active":"2024-02-13T03:22:34.992041+0000","last_peered":"2024-02-13T03:22:34.992041+0000","last_clean":"2024-02-13T03:22:34.992041+0000","last_became_active":"2024-02-13T03:21:09.890016+0000","last_became_peered":"2024-02-13T03:21:09.890016+0000","last_unstale":"2024-02-13T03:22:34.992041+0000","last_undegraded":"2024-02-13T03:22:34.992041+0000","last_fullsized":"2024-02-13T03:22:34.992041+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"60'39","reported_seq":102,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992506+0000","last_change":"2024-02-13T03:21:11.681664+0000","last_active":"2024-02-13T03:22:34.992506+0000","last_peered":"2024-02-13T03:22:34.992506+0000","last_clean":"2024-02-13T03:22:34.992506+0000","last_became_active":"2024-02-13T03:21:11.680975+0000","last_became_peered":"2024-02-13T03:21:11.680975+0000","last_unstale":"2024-02-13T03:22:34.992506+0000","last_undegraded":"2024-02-13T03:22:34.992506+0000","last_fullsized":"2024-02-13T03:22:34.992506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.983790+0000","last_change":"2024-02-13T03:21:15.709923+0000","last_active":"2024-02-13T03:22:34.983790+0000","last_peered":"2024-02-13T03:22:34.983790+0000","last_clean":"2024-02-13T03:22:34.983790+0000","last_became_active":"2024-02-13T03:21:15.709397+0000","last_became_peered":"2024-02-13T03:21:15.709397+0000","last_unstale":"2024-02-13T03:22:34.983790+0000","last_undegraded":"2024-02-13T03:22:34.983790+0000","last_fullsized":"2024-02-13T03:22:34.983790+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.995548+0000","last_change":"2024-02-13T03:21:13.699585+0000","last_active":"2024-02-13T03:22:34.995548+0000","last_peered":"2024-02-13T03:22:34.995548+0000","last_clean":"2024-02-13T03:22:34.995548+0000","last_became_active":"2024-02-13T03:21:13.699118+0000","last_became_peered":"2024-02-13T03:21:13.699118+0000","last_unstale":"2024-02-13T03:22:34.995548+0000","last_undegraded":"2024-02-13T03:22:34.995548+0000","last_fullsized":"2024-02-13T03:22:34.995548+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996579+0000","last_change":"2024-02-13T03:21:09.682172+0000","last_active":"2024-02-13T03:22:34.996579+0000","last_peered":"2024-02-13T03:22:34.996579+0000","last_clean":"2024-02-13T03:22:34.996579+0000","last_became_active":"2024-02-13T03:21:09.681813+0000","last_became_peered":"2024-02-13T03:21:09.681813+0000","last_unstale":"2024-02-13T03:22:34.996579+0000","last_undegraded":"2024-02-13T03:22:34.996579+0000","last_fullsized":"2024-02-13T03:22:34.996579+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"60'21","reported_seq":63,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991060+0000","last_change":"2024-02-13T03:21:11.677665+0000","last_active":"2024-02-13T03:22:34.991060+0000","last_peered":"2024-02-13T03:22:34.991060+0000","last_clean":"2024-02-13T03:22:34.991060+0000","last_became_active":"2024-02-13T03:21:11.677317+0000","last_became_peered":"2024-02-13T03:21:11.677317+0000","last_unstale":"2024-02-13T03:22:34.991060+0000","last_undegraded":"2024-02-13T03:22:34.991060+0000","last_fullsized":"2024-02-13T03:22:34.991060+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.994910+0000","last_change":"2024-02-13T03:21:15.719993+0000","last_active":"2024-02-13T03:22:34.994910+0000","last_peered":"2024-02-13T03:22:34.994910+0000","last_clean":"2024-02-13T03:22:34.994910+0000","last_became_active":"2024-02-13T03:21:15.719676+0000","last_became_peered":"2024-02-13T03:21:15.719676+0000","last_unstale":"2024-02-13T03:22:34.994910+0000","last_undegraded":"2024-02-13T03:22:34.994910+0000","last_fullsized":"2024-02-13T03:22:34.994910+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987529+0000","last_change":"2024-02-13T03:21:13.699675+0000","last_active":"2024-02-13T03:22:34.987529+0000","last_peered":"2024-02-13T03:22:34.987529+0000","last_clean":"2024-02-13T03:22:34.987529+0000","last_became_active":"2024-02-13T03:21:13.699297+0000","last_became_peered":"2024-02-13T03:21:13.699297+0000","last_unstale":"2024-02-13T03:22:34.987529+0000","last_undegraded":"2024-02-13T03:22:34.987529+0000","last_fullsized":"2024-02-13T03:22:34.987529+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.995450+0000","last_change":"2024-02-13T03:21:09.681823+0000","last_active":"2024-02-13T03:22:34.995450+0000","last_peered":"2024-02-13T03:22:34.995450+0000","last_clean":"2024-02-13T03:22:34.995450+0000","last_became_active":"2024-02-13T03:21:09.681494+0000","last_became_peered":"2024-02-13T03:21:09.681494+0000","last_unstale":"2024-02-13T03:22:34.995450+0000","last_undegraded":"2024-02-13T03:22:34.995450+0000","last_fullsized":"2024-02-13T03:22:34.995450+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"60'11","reported_seq":59,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:22:36.696008+0000","last_change":"2024-02-13T03:22:36.696008+0000","last_active":"2024-02-13T03:22:36.696008+0000","last_peered":"2024-02-13T03:22:36.696008+0000","last_clean":"2024-02-13T03:22:36.696008+0000","last_became_active":"2024-02-13T03:22:36.001245+0000","last_became_peered":"2024-02-13T03:22:36.001245+0000","last_unstale":"2024-02-13T03:22:36.696008+0000","last_undegraded":"2024-02-13T03:22:36.696008+0000","last_fullsized":"2024-02-13T03:22:36.696008+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991941+0000","last_change":"2024-02-13T03:21:15.719021+0000","last_active":"2024-02-13T03:22:34.991941+0000","last_peered":"2024-02-13T03:22:34.991941+0000","last_clean":"2024-02-13T03:22:34.991941+0000","last_became_active":"2024-02-13T03:21:15.718779+0000","last_became_peered":"2024-02-13T03:21:15.718779+0000","last_unstale":"2024-02-13T03:22:34.991941+0000","last_undegraded":"2024-02-13T03:22:34.991941+0000","last_fullsized":"2024-02-13T03:22:34.991941+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992029+0000","last_change":"2024-02-13T03:21:13.686451+0000","last_active":"2024-02-13T03:22:34.992029+0000","last_peered":"2024-02-13T03:22:34.992029+0000","last_clean":"2024-02-13T03:22:34.992029+0000","last_became_active":"2024-02-13T03:21:13.686100+0000","last_became_peered":"2024-02-13T03:21:13.686100+0000","last_unstale":"2024-02-13T03:22:34.992029+0000","last_undegraded":"2024-02-13T03:22:34.992029+0000","last_fullsized":"2024-02-13T03:22:34.992029+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985141+0000","last_change":"2024-02-13T03:21:09.672561+0000","last_active":"2024-02-13T03:22:34.985141+0000","last_peered":"2024-02-13T03:22:34.985141+0000","last_clean":"2024-02-13T03:22:34.985141+0000","last_became_active":"2024-02-13T03:21:09.672137+0000","last_became_peered":"2024-02-13T03:21:09.672137+0000","last_unstale":"2024-02-13T03:22:34.985141+0000","last_undegraded":"2024-02-13T03:22:34.985141+0000","last_fullsized":"2024-02-13T03:22:34.985141+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"60'15","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.989691+0000","last_change":"2024-02-13T03:21:11.684010+0000","last_active":"2024-02-13T03:22:34.989691+0000","last_peered":"2024-02-13T03:22:34.989691+0000","last_clean":"2024-02-13T03:22:34.989691+0000","last_became_active":"2024-02-13T03:21:11.683615+0000","last_became_peered":"2024-02-13T03:21:11.683615+0000","last_unstale":"2024-02-13T03:22:34.989691+0000","last_undegraded":"2024-02-13T03:22:34.989691+0000","last_fullsized":"2024-02-13T03:22:34.989691+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991745+0000","last_change":"2024-02-13T03:21:15.713802+0000","last_active":"2024-02-13T03:22:34.991745+0000","last_peered":"2024-02-13T03:22:34.991745+0000","last_clean":"2024-02-13T03:22:34.991745+0000","last_became_active":"2024-02-13T03:21:15.713415+0000","last_became_peered":"2024-02-13T03:21:15.713415+0000","last_unstale":"2024-02-13T03:22:34.991745+0000","last_undegraded":"2024-02-13T03:22:34.991745+0000","last_fullsized":"2024-02-13T03:22:34.991745+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991265+0000","last_change":"2024-02-13T03:21:13.698026+0000","last_active":"2024-02-13T03:22:34.991265+0000","last_peered":"2024-02-13T03:22:34.991265+0000","last_clean":"2024-02-13T03:22:34.991265+0000","last_became_active":"2024-02-13T03:21:13.697702+0000","last_became_peered":"2024-02-13T03:21:13.697702+0000","last_unstale":"2024-02-13T03:22:34.991265+0000","last_undegraded":"2024-02-13T03:22:34.991265+0000","last_fullsized":"2024-02-13T03:22:34.991265+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991177+0000","last_change":"2024-02-13T03:21:09.891507+0000","last_active":"2024-02-13T03:22:34.991177+0000","last_peered":"2024-02-13T03:22:34.991177+0000","last_clean":"2024-02-13T03:22:34.991177+0000","last_became_active":"2024-02-13T03:21:09.891057+0000","last_became_peered":"2024-02-13T03:21:09.891057+0000","last_unstale":"2024-02-13T03:22:34.991177+0000","last_undegraded":"2024-02-13T03:22:34.991177+0000","last_fullsized":"2024-02-13T03:22:34.991177+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"60'18","reported_seq":55,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991486+0000","last_change":"2024-02-13T03:21:11.676921+0000","last_active":"2024-02-13T03:22:34.991486+0000","last_peered":"2024-02-13T03:22:34.991486+0000","last_clean":"2024-02-13T03:22:34.991486+0000","last_became_active":"2024-02-13T03:21:11.674876+0000","last_became_peered":"2024-02-13T03:21:11.674876+0000","last_unstale":"2024-02-13T03:22:34.991486+0000","last_undegraded":"2024-02-13T03:22:34.991486+0000","last_fullsized":"2024-02-13T03:22:34.991486+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988155+0000","last_change":"2024-02-13T03:21:15.719273+0000","last_active":"2024-02-13T03:22:34.988155+0000","last_peered":"2024-02-13T03:22:34.988155+0000","last_clean":"2024-02-13T03:22:34.988155+0000","last_became_active":"2024-02-13T03:21:15.718838+0000","last_became_peered":"2024-02-13T03:21:15.718838+0000","last_unstale":"2024-02-13T03:22:34.988155+0000","last_undegraded":"2024-02-13T03:22:34.988155+0000","last_fullsized":"2024-02-13T03:22:34.988155+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"60'8","reported_seq":79,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:32.890092+0000","last_change":"2024-02-13T03:21:13.697995+0000","last_active":"2024-02-13T03:25:32.890092+0000","last_peered":"2024-02-13T03:25:32.890092+0000","last_clean":"2024-02-13T03:25:32.890092+0000","last_became_active":"2024-02-13T03:21:13.696479+0000","last_became_peered":"2024-02-13T03:21:13.696479+0000","last_unstale":"2024-02-13T03:25:32.890092+0000","last_undegraded":"2024-02-13T03:25:32.890092+0000","last_fullsized":"2024-02-13T03:25:32.890092+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991152+0000","last_change":"2024-02-13T03:21:09.672365+0000","last_active":"2024-02-13T03:22:34.991152+0000","last_peered":"2024-02-13T03:22:34.991152+0000","last_clean":"2024-02-13T03:22:34.991152+0000","last_became_active":"2024-02-13T03:21:09.671995+0000","last_became_peered":"2024-02-13T03:21:09.671995+0000","last_unstale":"2024-02-13T03:22:34.991152+0000","last_undegraded":"2024-02-13T03:22:34.991152+0000","last_fullsized":"2024-02-13T03:22:34.991152+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"60'16","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985274+0000","last_change":"2024-02-13T03:21:12.019391+0000","last_active":"2024-02-13T03:22:34.985274+0000","last_peered":"2024-02-13T03:22:34.985274+0000","last_clean":"2024-02-13T03:22:34.985274+0000","last_became_active":"2024-02-13T03:21:12.018959+0000","last_became_peered":"2024-02-13T03:21:12.018959+0000","last_unstale":"2024-02-13T03:22:34.985274+0000","last_undegraded":"2024-02-13T03:22:34.985274+0000","last_fullsized":"2024-02-13T03:22:34.985274+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991751+0000","last_change":"2024-02-13T03:21:15.716547+0000","last_active":"2024-02-13T03:22:34.991751+0000","last_peered":"2024-02-13T03:22:34.991751+0000","last_clean":"2024-02-13T03:22:34.991751+0000","last_became_active":"2024-02-13T03:21:15.716194+0000","last_became_peered":"2024-02-13T03:21:15.716194+0000","last_unstale":"2024-02-13T03:22:34.991751+0000","last_undegraded":"2024-02-13T03:22:34.991751+0000","last_fullsized":"2024-02-13T03:22:34.991751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985537+0000","last_change":"2024-02-13T03:21:13.691442+0000","last_active":"2024-02-13T03:22:34.985537+0000","last_peered":"2024-02-13T03:22:34.985537+0000","last_clean":"2024-02-13T03:22:34.985537+0000","last_became_active":"2024-02-13T03:21:13.691023+0000","last_became_peered":"2024-02-13T03:21:13.691023+0000","last_unstale":"2024-02-13T03:22:34.985537+0000","last_undegraded":"2024-02-13T03:22:34.985537+0000","last_fullsized":"2024-02-13T03:22:34.985537+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992030+0000","last_change":"2024-02-13T03:21:09.666829+0000","last_active":"2024-02-13T03:22:34.992030+0000","last_peered":"2024-02-13T03:22:34.992030+0000","last_clean":"2024-02-13T03:22:34.992030+0000","last_became_active":"2024-02-13T03:21:09.666424+0000","last_became_peered":"2024-02-13T03:21:09.666424+0000","last_unstale":"2024-02-13T03:22:34.992030+0000","last_undegraded":"2024-02-13T03:22:34.992030+0000","last_fullsized":"2024-02-13T03:22:34.992030+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'17","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.997217+0000","last_change":"2024-02-13T03:21:11.681384+0000","last_active":"2024-02-13T03:22:34.997217+0000","last_peered":"2024-02-13T03:22:34.997217+0000","last_clean":"2024-02-13T03:22:34.997217+0000","last_became_active":"2024-02-13T03:21:11.681073+0000","last_became_peered":"2024-02-13T03:21:11.681073+0000","last_unstale":"2024-02-13T03:22:34.997217+0000","last_undegraded":"2024-02-13T03:22:34.997217+0000","last_fullsized":"2024-02-13T03:22:34.997217+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987374+0000","last_change":"2024-02-13T03:21:15.719544+0000","last_active":"2024-02-13T03:22:34.987374+0000","last_peered":"2024-02-13T03:22:34.987374+0000","last_clean":"2024-02-13T03:22:34.987374+0000","last_became_active":"2024-02-13T03:21:15.719320+0000","last_became_peered":"2024-02-13T03:21:15.719320+0000","last_unstale":"2024-02-13T03:22:34.987374+0000","last_undegraded":"2024-02-13T03:22:34.987374+0000","last_fullsized":"2024-02-13T03:22:34.987374+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.995789+0000","last_change":"2024-02-13T03:21:13.692644+0000","last_active":"2024-02-13T03:22:34.995789+0000","last_peered":"2024-02-13T03:22:34.995789+0000","last_clean":"2024-02-13T03:22:34.995789+0000","last_became_active":"2024-02-13T03:21:13.692203+0000","last_became_peered":"2024-02-13T03:21:13.692203+0000","last_unstale":"2024-02-13T03:22:34.995789+0000","last_undegraded":"2024-02-13T03:22:34.995789+0000","last_fullsized":"2024-02-13T03:22:34.995789+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985430+0000","last_change":"2024-02-13T03:21:09.675350+0000","last_active":"2024-02-13T03:22:34.985430+0000","last_peered":"2024-02-13T03:22:34.985430+0000","last_clean":"2024-02-13T03:22:34.985430+0000","last_became_active":"2024-02-13T03:21:09.674981+0000","last_became_peered":"2024-02-13T03:21:09.674981+0000","last_unstale":"2024-02-13T03:22:34.985430+0000","last_undegraded":"2024-02-13T03:22:34.985430+0000","last_fullsized":"2024-02-13T03:22:34.985430+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"60'4","reported_seq":23,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992136+0000","last_change":"2024-02-13T03:21:11.675505+0000","last_active":"2024-02-13T03:22:34.992136+0000","last_peered":"2024-02-13T03:22:34.992136+0000","last_clean":"2024-02-13T03:22:34.992136+0000","last_became_active":"2024-02-13T03:21:11.675052+0000","last_became_peered":"2024-02-13T03:21:11.675052+0000","last_unstale":"2024-02-13T03:22:34.992136+0000","last_undegraded":"2024-02-13T03:22:34.992136+0000","last_fullsized":"2024-02-13T03:22:34.992136+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991222+0000","last_change":"2024-02-13T03:21:15.701933+0000","last_active":"2024-02-13T03:22:34.991222+0000","last_peered":"2024-02-13T03:22:34.991222+0000","last_clean":"2024-02-13T03:22:34.991222+0000","last_became_active":"2024-02-13T03:21:15.701606+0000","last_became_peered":"2024-02-13T03:21:15.701606+0000","last_unstale":"2024-02-13T03:22:34.991222+0000","last_undegraded":"2024-02-13T03:22:34.991222+0000","last_fullsized":"2024-02-13T03:22:34.991222+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987582+0000","last_change":"2024-02-13T03:21:13.698530+0000","last_active":"2024-02-13T03:22:34.987582+0000","last_peered":"2024-02-13T03:22:34.987582+0000","last_clean":"2024-02-13T03:22:34.987582+0000","last_became_active":"2024-02-13T03:21:13.698185+0000","last_became_peered":"2024-02-13T03:21:13.698185+0000","last_unstale":"2024-02-13T03:22:34.987582+0000","last_undegraded":"2024-02-13T03:22:34.987582+0000","last_fullsized":"2024-02-13T03:22:34.987582+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996883+0000","last_change":"2024-02-13T03:21:09.679180+0000","last_active":"2024-02-13T03:22:34.996883+0000","last_peered":"2024-02-13T03:22:34.996883+0000","last_clean":"2024-02-13T03:22:34.996883+0000","last_became_active":"2024-02-13T03:21:09.678859+0000","last_became_peered":"2024-02-13T03:21:09.678859+0000","last_unstale":"2024-02-13T03:22:34.996883+0000","last_undegraded":"2024-02-13T03:22:34.996883+0000","last_fullsized":"2024-02-13T03:22:34.996883+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"60'17","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992408+0000","last_change":"2024-02-13T03:21:12.018803+0000","last_active":"2024-02-13T03:22:34.992408+0000","last_peered":"2024-02-13T03:22:34.992408+0000","last_clean":"2024-02-13T03:22:34.992408+0000","last_became_active":"2024-02-13T03:21:12.017648+0000","last_became_peered":"2024-02-13T03:21:12.017648+0000","last_unstale":"2024-02-13T03:22:34.992408+0000","last_undegraded":"2024-02-13T03:22:34.992408+0000","last_fullsized":"2024-02-13T03:22:34.992408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"60'1","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.983488+0000","last_change":"2024-02-13T03:21:15.710918+0000","last_active":"2024-02-13T03:22:34.983488+0000","last_peered":"2024-02-13T03:22:34.983488+0000","last_clean":"2024-02-13T03:22:34.983488+0000","last_became_active":"2024-02-13T03:21:15.710532+0000","last_became_peered":"2024-02-13T03:21:15.710532+0000","last_unstale":"2024-02-13T03:22:34.983488+0000","last_undegraded":"2024-02-13T03:22:34.983488+0000","last_fullsized":"2024-02-13T03:22:34.983488+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992650+0000","last_change":"2024-02-13T03:21:13.694780+0000","last_active":"2024-02-13T03:22:34.992650+0000","last_peered":"2024-02-13T03:22:34.992650+0000","last_clean":"2024-02-13T03:22:34.992650+0000","last_became_active":"2024-02-13T03:21:13.694381+0000","last_became_peered":"2024-02-13T03:21:13.694381+0000","last_unstale":"2024-02-13T03:22:34.992650+0000","last_undegraded":"2024-02-13T03:22:34.992650+0000","last_fullsized":"2024-02-13T03:22:34.992650+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988175+0000","last_change":"2024-02-13T03:21:09.673670+0000","last_active":"2024-02-13T03:22:34.988175+0000","last_peered":"2024-02-13T03:22:34.988175+0000","last_clean":"2024-02-13T03:22:34.988175+0000","last_became_active":"2024-02-13T03:21:09.672121+0000","last_became_peered":"2024-02-13T03:21:09.672121+0000","last_unstale":"2024-02-13T03:22:34.988175+0000","last_undegraded":"2024-02-13T03:22:34.988175+0000","last_fullsized":"2024-02-13T03:22:34.988175+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"60'9","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:22:36.006223+0000","last_change":"2024-02-13T03:22:36.006223+0000","last_active":"2024-02-13T03:22:36.006223+0000","last_peered":"2024-02-13T03:22:36.006223+0000","last_clean":"2024-02-13T03:22:36.006223+0000","last_became_active":"2024-02-13T03:22:35.999802+0000","last_became_peered":"2024-02-13T03:22:35.999802+0000","last_unstale":"2024-02-13T03:22:36.006223+0000","last_undegraded":"2024-02-13T03:22:36.006223+0000","last_fullsized":"2024-02-13T03:22:36.006223+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991728+0000","last_change":"2024-02-13T03:21:15.712517+0000","last_active":"2024-02-13T03:22:34.991728+0000","last_peered":"2024-02-13T03:22:34.991728+0000","last_clean":"2024-02-13T03:22:34.991728+0000","last_became_active":"2024-02-13T03:21:15.712193+0000","last_became_peered":"2024-02-13T03:21:15.712193+0000","last_unstale":"2024-02-13T03:22:34.991728+0000","last_undegraded":"2024-02-13T03:22:34.991728+0000","last_fullsized":"2024-02-13T03:22:34.991728+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988444+0000","last_change":"2024-02-13T03:21:13.686338+0000","last_active":"2024-02-13T03:22:34.988444+0000","last_peered":"2024-02-13T03:22:34.988444+0000","last_clean":"2024-02-13T03:22:34.988444+0000","last_became_active":"2024-02-13T03:21:13.686016+0000","last_became_peered":"2024-02-13T03:21:13.686016+0000","last_unstale":"2024-02-13T03:22:34.988444+0000","last_undegraded":"2024-02-13T03:22:34.988444+0000","last_fullsized":"2024-02-13T03:22:34.988444+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'1","reported_seq":25,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991704+0000","last_change":"2024-02-13T03:21:09.677024+0000","last_active":"2024-02-13T03:22:34.991704+0000","last_peered":"2024-02-13T03:22:34.991704+0000","last_clean":"2024-02-13T03:22:34.991704+0000","last_became_active":"2024-02-13T03:21:09.676705+0000","last_became_peered":"2024-02-13T03:21:09.676705+0000","last_unstale":"2024-02-13T03:22:34.991704+0000","last_undegraded":"2024-02-13T03:22:34.991704+0000","last_fullsized":"2024-02-13T03:22:34.991704+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"60'11","reported_seq":36,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991429+0000","last_change":"2024-02-13T03:21:11.680974+0000","last_active":"2024-02-13T03:22:34.991429+0000","last_peered":"2024-02-13T03:22:34.991429+0000","last_clean":"2024-02-13T03:22:34.991429+0000","last_became_active":"2024-02-13T03:21:11.680688+0000","last_became_peered":"2024-02-13T03:21:11.680688+0000","last_unstale":"2024-02-13T03:22:34.991429+0000","last_undegraded":"2024-02-13T03:22:34.991429+0000","last_fullsized":"2024-02-13T03:22:34.991429+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985011+0000","last_change":"2024-02-13T03:21:15.712764+0000","last_active":"2024-02-13T03:22:34.985011+0000","last_peered":"2024-02-13T03:22:34.985011+0000","last_clean":"2024-02-13T03:22:34.985011+0000","last_became_active":"2024-02-13T03:21:15.712423+0000","last_became_peered":"2024-02-13T03:21:15.712423+0000","last_unstale":"2024-02-13T03:22:34.985011+0000","last_undegraded":"2024-02-13T03:22:34.985011+0000","last_fullsized":"2024-02-13T03:22:34.985011+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"60'8","reported_seq":79,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:32.890185+0000","last_change":"2024-02-13T03:21:13.700498+0000","last_active":"2024-02-13T03:25:32.890185+0000","last_peered":"2024-02-13T03:25:32.890185+0000","last_clean":"2024-02-13T03:25:32.890185+0000","last_became_active":"2024-02-13T03:21:13.700072+0000","last_became_peered":"2024-02-13T03:21:13.700072+0000","last_unstale":"2024-02-13T03:25:32.890185+0000","last_undegraded":"2024-02-13T03:25:32.890185+0000","last_fullsized":"2024-02-13T03:25:32.890185+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987655+0000","last_change":"2024-02-13T03:21:09.893368+0000","last_active":"2024-02-13T03:22:34.987655+0000","last_peered":"2024-02-13T03:22:34.987655+0000","last_clean":"2024-02-13T03:22:34.987655+0000","last_became_active":"2024-02-13T03:21:09.892998+0000","last_became_peered":"2024-02-13T03:21:09.892998+0000","last_unstale":"2024-02-13T03:22:34.987655+0000","last_undegraded":"2024-02-13T03:22:34.987655+0000","last_fullsized":"2024-02-13T03:22:34.987655+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"60'22","reported_seq":62,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992520+0000","last_change":"2024-02-13T03:21:12.018509+0000","last_active":"2024-02-13T03:22:34.992520+0000","last_peered":"2024-02-13T03:22:34.992520+0000","last_clean":"2024-02-13T03:22:34.992520+0000","last_became_active":"2024-02-13T03:21:12.017459+0000","last_became_peered":"2024-02-13T03:21:12.017459+0000","last_unstale":"2024-02-13T03:22:34.992520+0000","last_undegraded":"2024-02-13T03:22:34.992520+0000","last_fullsized":"2024-02-13T03:22:34.992520+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992009+0000","last_change":"2024-02-13T03:21:15.719839+0000","last_active":"2024-02-13T03:22:34.992009+0000","last_peered":"2024-02-13T03:22:34.992009+0000","last_clean":"2024-02-13T03:22:34.992009+0000","last_became_active":"2024-02-13T03:21:15.719549+0000","last_became_peered":"2024-02-13T03:21:15.719549+0000","last_unstale":"2024-02-13T03:22:34.992009+0000","last_undegraded":"2024-02-13T03:22:34.992009+0000","last_fullsized":"2024-02-13T03:22:34.992009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"60'8","reported_seq":76,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:32.886631+0000","last_change":"2024-02-13T03:21:13.690254+0000","last_active":"2024-02-13T03:25:32.886631+0000","last_peered":"2024-02-13T03:25:32.886631+0000","last_clean":"2024-02-13T03:25:32.886631+0000","last_became_active":"2024-02-13T03:21:13.689829+0000","last_became_peered":"2024-02-13T03:21:13.689829+0000","last_unstale":"2024-02-13T03:25:32.886631+0000","last_undegraded":"2024-02-13T03:25:32.886631+0000","last_fullsized":"2024-02-13T03:25:32.886631+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.984299+0000","last_change":"2024-02-13T03:21:09.667086+0000","last_active":"2024-02-13T03:22:34.984299+0000","last_peered":"2024-02-13T03:22:34.984299+0000","last_clean":"2024-02-13T03:22:34.984299+0000","last_became_active":"2024-02-13T03:21:09.666585+0000","last_became_peered":"2024-02-13T03:21:09.666585+0000","last_unstale":"2024-02-13T03:22:34.984299+0000","last_undegraded":"2024-02-13T03:22:34.984299+0000","last_fullsized":"2024-02-13T03:22:34.984299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"60'15","reported_seq":48,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996963+0000","last_change":"2024-02-13T03:21:12.025158+0000","last_active":"2024-02-13T03:22:34.996963+0000","last_peered":"2024-02-13T03:22:34.996963+0000","last_clean":"2024-02-13T03:22:34.996963+0000","last_became_active":"2024-02-13T03:21:12.024786+0000","last_became_peered":"2024-02-13T03:21:12.024786+0000","last_unstale":"2024-02-13T03:22:34.996963+0000","last_undegraded":"2024-02-13T03:22:34.996963+0000","last_fullsized":"2024-02-13T03:22:34.996963+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990062+0000","last_change":"2024-02-13T03:21:15.719431+0000","last_active":"2024-02-13T03:22:34.990062+0000","last_peered":"2024-02-13T03:22:34.990062+0000","last_clean":"2024-02-13T03:22:34.990062+0000","last_became_active":"2024-02-13T03:21:15.719060+0000","last_became_peered":"2024-02-13T03:21:15.719060+0000","last_unstale":"2024-02-13T03:22:34.990062+0000","last_undegraded":"2024-02-13T03:22:34.990062+0000","last_fullsized":"2024-02-13T03:22:34.990062+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992205+0000","last_change":"2024-02-13T03:21:13.692813+0000","last_active":"2024-02-13T03:22:34.992205+0000","last_peered":"2024-02-13T03:22:34.992205+0000","last_clean":"2024-02-13T03:22:34.992205+0000","last_became_active":"2024-02-13T03:21:13.692461+0000","last_became_peered":"2024-02-13T03:21:13.692461+0000","last_unstale":"2024-02-13T03:22:34.992205+0000","last_undegraded":"2024-02-13T03:22:34.992205+0000","last_fullsized":"2024-02-13T03:22:34.992205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987681+0000","last_change":"2024-02-13T03:21:09.673501+0000","last_active":"2024-02-13T03:22:34.987681+0000","last_peered":"2024-02-13T03:22:34.987681+0000","last_clean":"2024-02-13T03:22:34.987681+0000","last_became_active":"2024-02-13T03:21:09.673173+0000","last_became_peered":"2024-02-13T03:21:09.673173+0000","last_unstale":"2024-02-13T03:22:34.987681+0000","last_undegraded":"2024-02-13T03:22:34.987681+0000","last_fullsized":"2024-02-13T03:22:34.987681+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"60'21","reported_seq":63,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985684+0000","last_change":"2024-02-13T03:21:12.022273+0000","last_active":"2024-02-13T03:22:34.985684+0000","last_peered":"2024-02-13T03:22:34.985684+0000","last_clean":"2024-02-13T03:22:34.985684+0000","last_became_active":"2024-02-13T03:21:12.021613+0000","last_became_peered":"2024-02-13T03:21:12.021613+0000","last_unstale":"2024-02-13T03:22:34.985684+0000","last_undegraded":"2024-02-13T03:22:34.985684+0000","last_fullsized":"2024-02-13T03:22:34.985684+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988276+0000","last_change":"2024-02-13T03:21:15.717901+0000","last_active":"2024-02-13T03:22:34.988276+0000","last_peered":"2024-02-13T03:22:34.988276+0000","last_clean":"2024-02-13T03:22:34.988276+0000","last_became_active":"2024-02-13T03:21:15.717303+0000","last_became_peered":"2024-02-13T03:21:15.717303+0000","last_unstale":"2024-02-13T03:22:34.988276+0000","last_undegraded":"2024-02-13T03:22:34.988276+0000","last_fullsized":"2024-02-13T03:22:34.988276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996925+0000","last_change":"2024-02-13T03:21:13.694546+0000","last_active":"2024-02-13T03:22:34.996925+0000","last_peered":"2024-02-13T03:22:34.996925+0000","last_clean":"2024-02-13T03:22:34.996925+0000","last_became_active":"2024-02-13T03:21:13.694163+0000","last_became_peered":"2024-02-13T03:21:13.694163+0000","last_unstale":"2024-02-13T03:22:34.996925+0000","last_undegraded":"2024-02-13T03:22:34.996925+0000","last_fullsized":"2024-02-13T03:22:34.996925+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.995629+0000","last_change":"2024-02-13T03:21:09.677907+0000","last_active":"2024-02-13T03:22:34.995629+0000","last_peered":"2024-02-13T03:22:34.995629+0000","last_clean":"2024-02-13T03:22:34.995629+0000","last_became_active":"2024-02-13T03:21:09.677552+0000","last_became_peered":"2024-02-13T03:21:09.677552+0000","last_unstale":"2024-02-13T03:22:34.995629+0000","last_undegraded":"2024-02-13T03:22:34.995629+0000","last_fullsized":"2024-02-13T03:22:34.995629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"60'12","reported_seq":41,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992094+0000","last_change":"2024-02-13T03:21:12.019452+0000","last_active":"2024-02-13T03:22:34.992094+0000","last_peered":"2024-02-13T03:22:34.992094+0000","last_clean":"2024-02-13T03:22:34.992094+0000","last_became_active":"2024-02-13T03:21:12.019132+0000","last_became_peered":"2024-02-13T03:21:12.019132+0000","last_unstale":"2024-02-13T03:22:34.992094+0000","last_undegraded":"2024-02-13T03:22:34.992094+0000","last_fullsized":"2024-02-13T03:22:34.992094+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.995381+0000","last_change":"2024-02-13T03:21:15.721807+0000","last_active":"2024-02-13T03:22:34.995381+0000","last_peered":"2024-02-13T03:22:34.995381+0000","last_clean":"2024-02-13T03:22:34.995381+0000","last_became_active":"2024-02-13T03:21:15.721490+0000","last_became_peered":"2024-02-13T03:21:15.721490+0000","last_unstale":"2024-02-13T03:22:34.995381+0000","last_undegraded":"2024-02-13T03:22:34.995381+0000","last_fullsized":"2024-02-13T03:22:34.995381+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"53'2","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988761+0000","last_change":"2024-02-13T03:21:09.675843+0000","last_active":"2024-02-13T03:22:34.988761+0000","last_peered":"2024-02-13T03:22:34.988761+0000","last_clean":"2024-02-13T03:22:34.988761+0000","last_became_active":"2024-02-13T03:21:09.675404+0000","last_became_peered":"2024-02-13T03:21:09.675404+0000","last_unstale":"2024-02-13T03:22:34.988761+0000","last_undegraded":"2024-02-13T03:22:34.988761+0000","last_fullsized":"2024-02-13T03:22:34.988761+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"60'27","reported_seq":72,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.994717+0000","last_change":"2024-02-13T03:21:12.025133+0000","last_active":"2024-02-13T03:22:34.994717+0000","last_peered":"2024-02-13T03:22:34.994717+0000","last_clean":"2024-02-13T03:22:34.994717+0000","last_became_active":"2024-02-13T03:21:12.024766+0000","last_became_peered":"2024-02-13T03:21:12.024766+0000","last_unstale":"2024-02-13T03:22:34.994717+0000","last_undegraded":"2024-02-13T03:22:34.994717+0000","last_fullsized":"2024-02-13T03:22:34.994717+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"60'8","reported_seq":76,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:32.890296+0000","last_change":"2024-02-13T03:21:13.701953+0000","last_active":"2024-02-13T03:25:32.890296+0000","last_peered":"2024-02-13T03:25:32.890296+0000","last_clean":"2024-02-13T03:25:32.890296+0000","last_became_active":"2024-02-13T03:21:13.701595+0000","last_became_peered":"2024-02-13T03:21:13.701595+0000","last_unstale":"2024-02-13T03:25:32.890296+0000","last_undegraded":"2024-02-13T03:25:32.890296+0000","last_fullsized":"2024-02-13T03:25:32.890296+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991754+0000","last_change":"2024-02-13T03:21:15.708758+0000","last_active":"2024-02-13T03:22:34.991754+0000","last_peered":"2024-02-13T03:22:34.991754+0000","last_clean":"2024-02-13T03:22:34.991754+0000","last_became_active":"2024-02-13T03:21:15.708444+0000","last_became_peered":"2024-02-13T03:21:15.708444+0000","last_unstale":"2024-02-13T03:22:34.991754+0000","last_undegraded":"2024-02-13T03:22:34.991754+0000","last_fullsized":"2024-02-13T03:22:34.991754+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988555+0000","last_change":"2024-02-13T03:21:09.675874+0000","last_active":"2024-02-13T03:22:34.988555+0000","last_peered":"2024-02-13T03:22:34.988555+0000","last_clean":"2024-02-13T03:22:34.988555+0000","last_became_active":"2024-02-13T03:21:09.674990+0000","last_became_peered":"2024-02-13T03:21:09.674990+0000","last_unstale":"2024-02-13T03:22:34.988555+0000","last_undegraded":"2024-02-13T03:22:34.988555+0000","last_fullsized":"2024-02-13T03:22:34.988555+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"60'24","reported_seq":70,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990905+0000","last_change":"2024-02-13T03:21:11.680985+0000","last_active":"2024-02-13T03:22:34.990905+0000","last_peered":"2024-02-13T03:22:34.990905+0000","last_clean":"2024-02-13T03:22:34.990905+0000","last_became_active":"2024-02-13T03:21:11.680602+0000","last_became_peered":"2024-02-13T03:21:11.680602+0000","last_unstale":"2024-02-13T03:22:34.990905+0000","last_undegraded":"2024-02-13T03:22:34.990905+0000","last_fullsized":"2024-02-13T03:22:34.990905+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988012+0000","last_change":"2024-02-13T03:21:13.683804+0000","last_active":"2024-02-13T03:22:34.988012+0000","last_peered":"2024-02-13T03:22:34.988012+0000","last_clean":"2024-02-13T03:22:34.988012+0000","last_became_active":"2024-02-13T03:21:13.683415+0000","last_became_peered":"2024-02-13T03:21:13.683415+0000","last_unstale":"2024-02-13T03:22:34.988012+0000","last_undegraded":"2024-02-13T03:22:34.988012+0000","last_fullsized":"2024-02-13T03:22:34.988012+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987259+0000","last_change":"2024-02-13T03:21:15.716870+0000","last_active":"2024-02-13T03:22:34.987259+0000","last_peered":"2024-02-13T03:22:34.987259+0000","last_clean":"2024-02-13T03:22:34.987259+0000","last_became_active":"2024-02-13T03:21:15.714813+0000","last_became_peered":"2024-02-13T03:21:15.714813+0000","last_unstale":"2024-02-13T03:22:34.987259+0000","last_undegraded":"2024-02-13T03:22:34.987259+0000","last_fullsized":"2024-02-13T03:22:34.987259+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.989897+0000","last_change":"2024-02-13T03:21:13.688574+0000","last_active":"2024-02-13T03:22:34.989897+0000","last_peered":"2024-02-13T03:22:34.989897+0000","last_clean":"2024-02-13T03:22:34.989897+0000","last_became_active":"2024-02-13T03:21:13.688224+0000","last_became_peered":"2024-02-13T03:21:13.688224+0000","last_unstale":"2024-02-13T03:22:34.989897+0000","last_undegraded":"2024-02-13T03:22:34.989897+0000","last_fullsized":"2024-02-13T03:22:34.989897+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988460+0000","last_change":"2024-02-13T03:21:09.679650+0000","last_active":"2024-02-13T03:22:34.988460+0000","last_peered":"2024-02-13T03:22:34.988460+0000","last_clean":"2024-02-13T03:22:34.988460+0000","last_became_active":"2024-02-13T03:21:09.679184+0000","last_became_peered":"2024-02-13T03:21:09.679184+0000","last_unstale":"2024-02-13T03:22:34.988460+0000","last_undegraded":"2024-02-13T03:22:34.988460+0000","last_fullsized":"2024-02-13T03:22:34.988460+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"60'25","reported_seq":63,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996196+0000","last_change":"2024-02-13T03:21:12.021923+0000","last_active":"2024-02-13T03:22:34.996196+0000","last_peered":"2024-02-13T03:22:34.996196+0000","last_clean":"2024-02-13T03:22:34.996196+0000","last_became_active":"2024-02-13T03:21:12.021003+0000","last_became_peered":"2024-02-13T03:21:12.021003+0000","last_unstale":"2024-02-13T03:22:34.996196+0000","last_undegraded":"2024-02-13T03:22:34.996196+0000","last_fullsized":"2024-02-13T03:22:34.996196+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.983033+0000","last_change":"2024-02-13T03:21:15.714837+0000","last_active":"2024-02-13T03:22:34.983033+0000","last_peered":"2024-02-13T03:22:34.983033+0000","last_clean":"2024-02-13T03:22:34.983033+0000","last_became_active":"2024-02-13T03:21:15.714514+0000","last_became_peered":"2024-02-13T03:21:15.714514+0000","last_unstale":"2024-02-13T03:22:34.983033+0000","last_undegraded":"2024-02-13T03:22:34.983033+0000","last_fullsized":"2024-02-13T03:22:34.983033+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988681+0000","last_change":"2024-02-13T03:21:13.685173+0000","last_active":"2024-02-13T03:22:34.988681+0000","last_peered":"2024-02-13T03:22:34.988681+0000","last_clean":"2024-02-13T03:22:34.988681+0000","last_became_active":"2024-02-13T03:21:13.684888+0000","last_became_peered":"2024-02-13T03:21:13.684888+0000","last_unstale":"2024-02-13T03:22:34.988681+0000","last_undegraded":"2024-02-13T03:22:34.988681+0000","last_fullsized":"2024-02-13T03:22:34.988681+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996481+0000","last_change":"2024-02-13T03:21:09.678441+0000","last_active":"2024-02-13T03:22:34.996481+0000","last_peered":"2024-02-13T03:22:34.996481+0000","last_clean":"2024-02-13T03:22:34.996481+0000","last_became_active":"2024-02-13T03:21:09.677981+0000","last_became_peered":"2024-02-13T03:21:09.677981+0000","last_unstale":"2024-02-13T03:22:34.996481+0000","last_undegraded":"2024-02-13T03:22:34.996481+0000","last_fullsized":"2024-02-13T03:22:34.996481+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"60'15","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:22:36.433302+0000","last_change":"2024-02-13T03:22:36.433302+0000","last_active":"2024-02-13T03:22:36.433302+0000","last_peered":"2024-02-13T03:22:36.433302+0000","last_clean":"2024-02-13T03:22:36.433302+0000","last_became_active":"2024-02-13T03:22:35.997847+0000","last_became_peered":"2024-02-13T03:22:35.997847+0000","last_unstale":"2024-02-13T03:22:36.433302+0000","last_undegraded":"2024-02-13T03:22:36.433302+0000","last_fullsized":"2024-02-13T03:22:36.433302+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991457+0000","last_change":"2024-02-13T03:21:15.716004+0000","last_active":"2024-02-13T03:22:34.991457+0000","last_peered":"2024-02-13T03:22:34.991457+0000","last_clean":"2024-02-13T03:22:34.991457+0000","last_became_active":"2024-02-13T03:21:15.715549+0000","last_became_peered":"2024-02-13T03:21:15.715549+0000","last_unstale":"2024-02-13T03:22:34.991457+0000","last_undegraded":"2024-02-13T03:22:34.991457+0000","last_fullsized":"2024-02-13T03:22:34.991457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.995154+0000","last_change":"2024-02-13T03:21:08.660483+0000","last_active":"2024-02-13T03:22:34.995154+0000","last_peered":"2024-02-13T03:22:34.995154+0000","last_clean":"2024-02-13T03:22:34.995154+0000","last_became_active":"2024-02-13T03:21:08.660084+0000","last_became_peered":"2024-02-13T03:21:08.660084+0000","last_unstale":"2024-02-13T03:22:34.995154+0000","last_undegraded":"2024-02-13T03:22:34.995154+0000","last_fullsized":"2024-02-13T03:22:34.995154+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:07.620086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:07.620086+0000","last_clean_scrub_stamp":"2024-02-13T03:21:07.620086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991160+0000","last_change":"2024-02-13T03:21:09.673377+0000","last_active":"2024-02-13T03:22:34.991160+0000","last_peered":"2024-02-13T03:22:34.991160+0000","last_clean":"2024-02-13T03:22:34.991160+0000","last_became_active":"2024-02-13T03:21:09.672635+0000","last_became_peered":"2024-02-13T03:21:09.672635+0000","last_unstale":"2024-02-13T03:22:34.991160+0000","last_undegraded":"2024-02-13T03:22:34.991160+0000","last_fullsized":"2024-02-13T03:22:34.991160+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"60'34","reported_seq":133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:32.886743+0000","last_change":"2024-02-13T03:21:11.680288+0000","last_active":"2024-02-13T03:25:32.886743+0000","last_peered":"2024-02-13T03:25:32.886743+0000","last_clean":"2024-02-13T03:25:32.886743+0000","last_became_active":"2024-02-13T03:21:11.679879+0000","last_became_peered":"2024-02-13T03:21:11.679879+0000","last_unstale":"2024-02-13T03:25:32.886743+0000","last_undegraded":"2024-02-13T03:25:32.886743+0000","last_fullsized":"2024-02-13T03:25:32.886743+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985631+0000","last_change":"2024-02-13T03:21:13.690578+0000","last_active":"2024-02-13T03:22:34.985631+0000","last_peered":"2024-02-13T03:22:34.985631+0000","last_clean":"2024-02-13T03:22:34.985631+0000","last_became_active":"2024-02-13T03:21:13.690236+0000","last_became_peered":"2024-02-13T03:21:13.690236+0000","last_unstale":"2024-02-13T03:22:34.985631+0000","last_undegraded":"2024-02-13T03:22:34.985631+0000","last_fullsized":"2024-02-13T03:22:34.985631+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988543+0000","last_change":"2024-02-13T03:21:15.718127+0000","last_active":"2024-02-13T03:22:34.988543+0000","last_peered":"2024-02-13T03:22:34.988543+0000","last_clean":"2024-02-13T03:22:34.988543+0000","last_became_active":"2024-02-13T03:21:15.717847+0000","last_became_peered":"2024-02-13T03:21:15.717847+0000","last_unstale":"2024-02-13T03:22:34.988543+0000","last_undegraded":"2024-02-13T03:22:34.988543+0000","last_fullsized":"2024-02-13T03:22:34.988543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988180+0000","last_change":"2024-02-13T03:21:09.889381+0000","last_active":"2024-02-13T03:22:34.988180+0000","last_peered":"2024-02-13T03:22:34.988180+0000","last_clean":"2024-02-13T03:22:34.988180+0000","last_became_active":"2024-02-13T03:21:09.889030+0000","last_became_peered":"2024-02-13T03:21:09.889030+0000","last_unstale":"2024-02-13T03:22:34.988180+0000","last_undegraded":"2024-02-13T03:22:34.988180+0000","last_fullsized":"2024-02-13T03:22:34.988180+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988303+0000","last_change":"2024-02-13T03:21:08.660768+0000","last_active":"2024-02-13T03:22:34.988303+0000","last_peered":"2024-02-13T03:22:34.988303+0000","last_clean":"2024-02-13T03:22:34.988303+0000","last_became_active":"2024-02-13T03:21:08.660413+0000","last_became_peered":"2024-02-13T03:21:08.660413+0000","last_unstale":"2024-02-13T03:22:34.988303+0000","last_undegraded":"2024-02-13T03:22:34.988303+0000","last_fullsized":"2024-02-13T03:22:34.988303+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:07.620086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:07.620086+0000","last_clean_scrub_stamp":"2024-02-13T03:21:07.620086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"60'19","reported_seq":59,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992443+0000","last_change":"2024-02-13T03:21:11.676843+0000","last_active":"2024-02-13T03:22:34.992443+0000","last_peered":"2024-02-13T03:22:34.992443+0000","last_clean":"2024-02-13T03:22:34.992443+0000","last_became_active":"2024-02-13T03:21:11.676528+0000","last_became_peered":"2024-02-13T03:21:11.676528+0000","last_unstale":"2024-02-13T03:22:34.992443+0000","last_undegraded":"2024-02-13T03:22:34.992443+0000","last_fullsized":"2024-02-13T03:22:34.992443+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.989830+0000","last_change":"2024-02-13T03:21:13.696632+0000","last_active":"2024-02-13T03:22:34.989830+0000","last_peered":"2024-02-13T03:22:34.989830+0000","last_clean":"2024-02-13T03:22:34.989830+0000","last_became_active":"2024-02-13T03:21:13.696353+0000","last_became_peered":"2024-02-13T03:21:13.696353+0000","last_unstale":"2024-02-13T03:22:34.989830+0000","last_undegraded":"2024-02-13T03:22:34.989830+0000","last_fullsized":"2024-02-13T03:22:34.989830+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.997143+0000","last_change":"2024-02-13T03:21:15.722832+0000","last_active":"2024-02-13T03:22:34.997143+0000","last_peered":"2024-02-13T03:22:34.997143+0000","last_clean":"2024-02-13T03:22:34.997143+0000","last_became_active":"2024-02-13T03:21:15.722574+0000","last_became_peered":"2024-02-13T03:21:15.722574+0000","last_unstale":"2024-02-13T03:22:34.997143+0000","last_undegraded":"2024-02-13T03:22:34.997143+0000","last_fullsized":"2024-02-13T03:22:34.997143+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":23,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987424+0000","last_change":"2024-02-13T03:20:57.120214+0000","last_active":"2024-02-13T03:22:34.987424+0000","last_peered":"2024-02-13T03:22:34.987424+0000","last_clean":"2024-02-13T03:22:34.987424+0000","last_became_active":"2024-02-13T03:20:57.119905+0000","last_became_peered":"2024-02-13T03:20:57.119905+0000","last_unstale":"2024-02-13T03:22:34.987424+0000","last_undegraded":"2024-02-13T03:22:34.987424+0000","last_fullsized":"2024-02-13T03:22:34.987424+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:17:31.987382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:17:31.987382+0000","last_clean_scrub_stamp":"2024-02-13T03:17:31.987382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991452+0000","last_change":"2024-02-13T03:21:09.671092+0000","last_active":"2024-02-13T03:22:34.991452+0000","last_peered":"2024-02-13T03:22:34.991452+0000","last_clean":"2024-02-13T03:22:34.991452+0000","last_became_active":"2024-02-13T03:21:09.670767+0000","last_became_peered":"2024-02-13T03:21:09.670767+0000","last_unstale":"2024-02-13T03:22:34.991452+0000","last_undegraded":"2024-02-13T03:22:34.991452+0000","last_fullsized":"2024-02-13T03:22:34.991452+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"60'14","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996742+0000","last_change":"2024-02-13T03:21:12.022092+0000","last_active":"2024-02-13T03:22:34.996742+0000","last_peered":"2024-02-13T03:22:34.996742+0000","last_clean":"2024-02-13T03:22:34.996742+0000","last_became_active":"2024-02-13T03:21:12.021274+0000","last_became_peered":"2024-02-13T03:21:12.021274+0000","last_unstale":"2024-02-13T03:22:34.996742+0000","last_undegraded":"2024-02-13T03:22:34.996742+0000","last_fullsized":"2024-02-13T03:22:34.996742+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988246+0000","last_change":"2024-02-13T03:21:13.702783+0000","last_active":"2024-02-13T03:22:34.988246+0000","last_peered":"2024-02-13T03:22:34.988246+0000","last_clean":"2024-02-13T03:22:34.988246+0000","last_became_active":"2024-02-13T03:21:13.702442+0000","last_became_peered":"2024-02-13T03:21:13.702442+0000","last_unstale":"2024-02-13T03:22:34.988246+0000","last_undegraded":"2024-02-13T03:22:34.988246+0000","last_fullsized":"2024-02-13T03:22:34.988246+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990323+0000","last_change":"2024-02-13T03:21:15.715398+0000","last_active":"2024-02-13T03:22:34.990323+0000","last_peered":"2024-02-13T03:22:34.990323+0000","last_clean":"2024-02-13T03:22:34.990323+0000","last_became_active":"2024-02-13T03:21:15.715080+0000","last_became_peered":"2024-02-13T03:21:15.715080+0000","last_unstale":"2024-02-13T03:22:34.990323+0000","last_undegraded":"2024-02-13T03:22:34.990323+0000","last_fullsized":"2024-02-13T03:22:34.990323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.983368+0000","last_change":"2024-02-13T03:21:09.671105+0000","last_active":"2024-02-13T03:22:34.983368+0000","last_peered":"2024-02-13T03:22:34.983368+0000","last_clean":"2024-02-13T03:22:34.983368+0000","last_became_active":"2024-02-13T03:21:09.669540+0000","last_became_peered":"2024-02-13T03:21:09.669540+0000","last_unstale":"2024-02-13T03:22:34.983368+0000","last_undegraded":"2024-02-13T03:22:34.983368+0000","last_fullsized":"2024-02-13T03:22:34.983368+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"60'5","reported_seq":287,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:39.461057+0000","last_change":"2024-02-13T03:21:08.659798+0000","last_active":"2024-02-13T03:25:39.461057+0000","last_peered":"2024-02-13T03:25:39.461057+0000","last_clean":"2024-02-13T03:25:39.461057+0000","last_became_active":"2024-02-13T03:21:08.659452+0000","last_became_peered":"2024-02-13T03:21:08.659452+0000","last_unstale":"2024-02-13T03:25:39.461057+0000","last_undegraded":"2024-02-13T03:25:39.461057+0000","last_fullsized":"2024-02-13T03:25:39.461057+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:07.620086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:07.620086+0000","last_clean_scrub_stamp":"2024-02-13T03:21:07.620086+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":264,"num_read_kb":259,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"60'12","reported_seq":35,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985117+0000","last_change":"2024-02-13T03:21:11.677377+0000","last_active":"2024-02-13T03:22:34.985117+0000","last_peered":"2024-02-13T03:22:34.985117+0000","last_clean":"2024-02-13T03:22:34.985117+0000","last_became_active":"2024-02-13T03:21:11.677036+0000","last_became_peered":"2024-02-13T03:21:11.677036+0000","last_unstale":"2024-02-13T03:22:34.985117+0000","last_undegraded":"2024-02-13T03:22:34.985117+0000","last_fullsized":"2024-02-13T03:22:34.985117+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.997331+0000","last_change":"2024-02-13T03:21:13.699240+0000","last_active":"2024-02-13T03:22:34.997331+0000","last_peered":"2024-02-13T03:22:34.997331+0000","last_clean":"2024-02-13T03:22:34.997331+0000","last_became_active":"2024-02-13T03:21:13.698912+0000","last_became_peered":"2024-02-13T03:21:13.698912+0000","last_unstale":"2024-02-13T03:22:34.997331+0000","last_undegraded":"2024-02-13T03:22:34.997331+0000","last_fullsized":"2024-02-13T03:22:34.997331+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987420+0000","last_change":"2024-02-13T03:21:15.709016+0000","last_active":"2024-02-13T03:22:34.987420+0000","last_peered":"2024-02-13T03:22:34.987420+0000","last_clean":"2024-02-13T03:22:34.987420+0000","last_became_active":"2024-02-13T03:21:15.708609+0000","last_became_peered":"2024-02-13T03:21:15.708609+0000","last_unstale":"2024-02-13T03:22:34.987420+0000","last_undegraded":"2024-02-13T03:22:34.987420+0000","last_fullsized":"2024-02-13T03:22:34.987420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991353+0000","last_change":"2024-02-13T03:21:09.890251+0000","last_active":"2024-02-13T03:22:34.991353+0000","last_peered":"2024-02-13T03:22:34.991353+0000","last_clean":"2024-02-13T03:22:34.991353+0000","last_became_active":"2024-02-13T03:21:09.889879+0000","last_became_peered":"2024-02-13T03:21:09.889879+0000","last_unstale":"2024-02-13T03:22:34.991353+0000","last_undegraded":"2024-02-13T03:22:34.991353+0000","last_fullsized":"2024-02-13T03:22:34.991353+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"60'25","reported_seq":68,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985306+0000","last_change":"2024-02-13T03:21:12.018603+0000","last_active":"2024-02-13T03:22:34.985306+0000","last_peered":"2024-02-13T03:22:34.985306+0000","last_clean":"2024-02-13T03:22:34.985306+0000","last_became_active":"2024-02-13T03:21:12.017722+0000","last_became_peered":"2024-02-13T03:21:12.017722+0000","last_unstale":"2024-02-13T03:22:34.985306+0000","last_undegraded":"2024-02-13T03:22:34.985306+0000","last_fullsized":"2024-02-13T03:22:34.985306+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996430+0000","last_change":"2024-02-13T03:21:13.695336+0000","last_active":"2024-02-13T03:22:34.996430+0000","last_peered":"2024-02-13T03:22:34.996430+0000","last_clean":"2024-02-13T03:22:34.996430+0000","last_became_active":"2024-02-13T03:21:13.694984+0000","last_became_peered":"2024-02-13T03:21:13.694984+0000","last_unstale":"2024-02-13T03:22:34.996430+0000","last_undegraded":"2024-02-13T03:22:34.996430+0000","last_fullsized":"2024-02-13T03:22:34.996430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.982980+0000","last_change":"2024-02-13T03:21:15.711017+0000","last_active":"2024-02-13T03:22:34.982980+0000","last_peered":"2024-02-13T03:22:34.982980+0000","last_clean":"2024-02-13T03:22:34.982980+0000","last_became_active":"2024-02-13T03:21:15.710708+0000","last_became_peered":"2024-02-13T03:21:15.710708+0000","last_unstale":"2024-02-13T03:22:34.982980+0000","last_undegraded":"2024-02-13T03:22:34.982980+0000","last_fullsized":"2024-02-13T03:22:34.982980+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996771+0000","last_change":"2024-02-13T03:21:09.892530+0000","last_active":"2024-02-13T03:22:34.996771+0000","last_peered":"2024-02-13T03:22:34.996771+0000","last_clean":"2024-02-13T03:22:34.996771+0000","last_became_active":"2024-02-13T03:21:09.891560+0000","last_became_peered":"2024-02-13T03:21:09.891560+0000","last_unstale":"2024-02-13T03:22:34.996771+0000","last_undegraded":"2024-02-13T03:22:34.996771+0000","last_fullsized":"2024-02-13T03:22:34.996771+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"60'16","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988563+0000","last_change":"2024-02-13T03:21:11.675808+0000","last_active":"2024-02-13T03:22:34.988563+0000","last_peered":"2024-02-13T03:22:34.988563+0000","last_clean":"2024-02-13T03:22:34.988563+0000","last_became_active":"2024-02-13T03:21:11.675329+0000","last_became_peered":"2024-02-13T03:21:11.675329+0000","last_unstale":"2024-02-13T03:22:34.988563+0000","last_undegraded":"2024-02-13T03:22:34.988563+0000","last_fullsized":"2024-02-13T03:22:34.988563+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"60'8","reported_seq":76,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:32.886482+0000","last_change":"2024-02-13T03:21:13.693739+0000","last_active":"2024-02-13T03:25:32.886482+0000","last_peered":"2024-02-13T03:25:32.886482+0000","last_clean":"2024-02-13T03:25:32.886482+0000","last_became_active":"2024-02-13T03:21:13.692363+0000","last_became_peered":"2024-02-13T03:21:13.692363+0000","last_unstale":"2024-02-13T03:25:32.886482+0000","last_undegraded":"2024-02-13T03:25:32.886482+0000","last_fullsized":"2024-02-13T03:25:32.886482+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988055+0000","last_change":"2024-02-13T03:21:15.719024+0000","last_active":"2024-02-13T03:22:34.988055+0000","last_peered":"2024-02-13T03:22:34.988055+0000","last_clean":"2024-02-13T03:22:34.988055+0000","last_became_active":"2024-02-13T03:21:15.718682+0000","last_became_peered":"2024-02-13T03:21:15.718682+0000","last_unstale":"2024-02-13T03:22:34.988055+0000","last_undegraded":"2024-02-13T03:22:34.988055+0000","last_fullsized":"2024-02-13T03:22:34.988055+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985373+0000","last_change":"2024-02-13T03:21:09.669986+0000","last_active":"2024-02-13T03:22:34.985373+0000","last_peered":"2024-02-13T03:22:34.985373+0000","last_clean":"2024-02-13T03:22:34.985373+0000","last_became_active":"2024-02-13T03:21:09.669635+0000","last_became_peered":"2024-02-13T03:21:09.669635+0000","last_unstale":"2024-02-13T03:22:34.985373+0000","last_undegraded":"2024-02-13T03:22:34.985373+0000","last_fullsized":"2024-02-13T03:22:34.985373+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"60'21","reported_seq":54,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990912+0000","last_change":"2024-02-13T03:21:11.678542+0000","last_active":"2024-02-13T03:22:34.990912+0000","last_peered":"2024-02-13T03:22:34.990912+0000","last_clean":"2024-02-13T03:22:34.990912+0000","last_became_active":"2024-02-13T03:21:11.678249+0000","last_became_peered":"2024-02-13T03:21:11.678249+0000","last_unstale":"2024-02-13T03:22:34.990912+0000","last_undegraded":"2024-02-13T03:22:34.990912+0000","last_fullsized":"2024-02-13T03:22:34.990912+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"60'8","reported_seq":76,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:32.886739+0000","last_change":"2024-02-13T03:21:13.686685+0000","last_active":"2024-02-13T03:25:32.886739+0000","last_peered":"2024-02-13T03:25:32.886739+0000","last_clean":"2024-02-13T03:25:32.886739+0000","last_became_active":"2024-02-13T03:21:13.686154+0000","last_became_peered":"2024-02-13T03:21:13.686154+0000","last_unstale":"2024-02-13T03:25:32.886739+0000","last_undegraded":"2024-02-13T03:25:32.886739+0000","last_fullsized":"2024-02-13T03:25:32.886739+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990935+0000","last_change":"2024-02-13T03:21:15.712051+0000","last_active":"2024-02-13T03:22:34.990935+0000","last_peered":"2024-02-13T03:22:34.990935+0000","last_clean":"2024-02-13T03:22:34.990935+0000","last_became_active":"2024-02-13T03:21:15.711786+0000","last_became_peered":"2024-02-13T03:21:15.711786+0000","last_unstale":"2024-02-13T03:22:34.990935+0000","last_undegraded":"2024-02-13T03:22:34.990935+0000","last_fullsized":"2024-02-13T03:22:34.990935+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991874+0000","last_change":"2024-02-13T03:21:09.668411+0000","last_active":"2024-02-13T03:22:34.991874+0000","last_peered":"2024-02-13T03:22:34.991874+0000","last_clean":"2024-02-13T03:22:34.991874+0000","last_became_active":"2024-02-13T03:21:09.668081+0000","last_became_peered":"2024-02-13T03:21:09.668081+0000","last_unstale":"2024-02-13T03:22:34.991874+0000","last_undegraded":"2024-02-13T03:22:34.991874+0000","last_fullsized":"2024-02-13T03:22:34.991874+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"60'30","reported_seq":79,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992317+0000","last_change":"2024-02-13T03:21:12.020181+0000","last_active":"2024-02-13T03:22:34.992317+0000","last_peered":"2024-02-13T03:22:34.992317+0000","last_clean":"2024-02-13T03:22:34.992317+0000","last_became_active":"2024-02-13T03:21:12.019872+0000","last_became_peered":"2024-02-13T03:21:12.019872+0000","last_unstale":"2024-02-13T03:22:34.992317+0000","last_undegraded":"2024-02-13T03:22:34.992317+0000","last_fullsized":"2024-02-13T03:22:34.992317+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990723+0000","last_change":"2024-02-13T03:21:13.694344+0000","last_active":"2024-02-13T03:22:34.990723+0000","last_peered":"2024-02-13T03:22:34.990723+0000","last_clean":"2024-02-13T03:22:34.990723+0000","last_became_active":"2024-02-13T03:21:13.693831+0000","last_became_peered":"2024-02-13T03:21:13.693831+0000","last_unstale":"2024-02-13T03:22:34.990723+0000","last_undegraded":"2024-02-13T03:22:34.990723+0000","last_fullsized":"2024-02-13T03:22:34.990723+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996504+0000","last_change":"2024-02-13T03:21:15.720324+0000","last_active":"2024-02-13T03:22:34.996504+0000","last_peered":"2024-02-13T03:22:34.996504+0000","last_clean":"2024-02-13T03:22:34.996504+0000","last_became_active":"2024-02-13T03:21:15.719862+0000","last_became_peered":"2024-02-13T03:21:15.719862+0000","last_unstale":"2024-02-13T03:22:34.996504+0000","last_undegraded":"2024-02-13T03:22:34.996504+0000","last_fullsized":"2024-02-13T03:22:34.996504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"60'8","reported_seq":76,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:32.890269+0000","last_change":"2024-02-13T03:21:13.700639+0000","last_active":"2024-02-13T03:25:32.890269+0000","last_peered":"2024-02-13T03:25:32.890269+0000","last_clean":"2024-02-13T03:25:32.890269+0000","last_became_active":"2024-02-13T03:21:13.700316+0000","last_became_peered":"2024-02-13T03:21:13.700316+0000","last_unstale":"2024-02-13T03:25:32.890269+0000","last_undegraded":"2024-02-13T03:25:32.890269+0000","last_fullsized":"2024-02-13T03:25:32.890269+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992220+0000","last_change":"2024-02-13T03:21:09.668351+0000","last_active":"2024-02-13T03:22:34.992220+0000","last_peered":"2024-02-13T03:22:34.992220+0000","last_clean":"2024-02-13T03:22:34.992220+0000","last_became_active":"2024-02-13T03:21:09.668010+0000","last_became_peered":"2024-02-13T03:21:09.668010+0000","last_unstale":"2024-02-13T03:22:34.992220+0000","last_undegraded":"2024-02-13T03:22:34.992220+0000","last_fullsized":"2024-02-13T03:22:34.992220+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"60'15","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992387+0000","last_change":"2024-02-13T03:21:11.680636+0000","last_active":"2024-02-13T03:22:34.992387+0000","last_peered":"2024-02-13T03:22:34.992387+0000","last_clean":"2024-02-13T03:22:34.992387+0000","last_became_active":"2024-02-13T03:21:11.680328+0000","last_became_peered":"2024-02-13T03:21:11.680328+0000","last_unstale":"2024-02-13T03:22:34.992387+0000","last_undegraded":"2024-02-13T03:22:34.992387+0000","last_fullsized":"2024-02-13T03:22:34.992387+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991564+0000","last_change":"2024-02-13T03:21:15.713297+0000","last_active":"2024-02-13T03:22:34.991564+0000","last_peered":"2024-02-13T03:22:34.991564+0000","last_clean":"2024-02-13T03:22:34.991564+0000","last_became_active":"2024-02-13T03:21:15.712900+0000","last_became_peered":"2024-02-13T03:21:15.712900+0000","last_unstale":"2024-02-13T03:22:34.991564+0000","last_undegraded":"2024-02-13T03:22:34.991564+0000","last_fullsized":"2024-02-13T03:22:34.991564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.997273+0000","last_change":"2024-02-13T03:21:13.693915+0000","last_active":"2024-02-13T03:22:34.997273+0000","last_peered":"2024-02-13T03:22:34.997273+0000","last_clean":"2024-02-13T03:22:34.997273+0000","last_became_active":"2024-02-13T03:21:13.693569+0000","last_became_peered":"2024-02-13T03:21:13.693569+0000","last_unstale":"2024-02-13T03:22:34.997273+0000","last_undegraded":"2024-02-13T03:22:34.997273+0000","last_fullsized":"2024-02-13T03:22:34.997273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990666+0000","last_change":"2024-02-13T03:21:09.892621+0000","last_active":"2024-02-13T03:22:34.990666+0000","last_peered":"2024-02-13T03:22:34.990666+0000","last_clean":"2024-02-13T03:22:34.990666+0000","last_became_active":"2024-02-13T03:21:09.892095+0000","last_became_peered":"2024-02-13T03:21:09.892095+0000","last_unstale":"2024-02-13T03:22:34.990666+0000","last_undegraded":"2024-02-13T03:22:34.990666+0000","last_fullsized":"2024-02-13T03:22:34.990666+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"60'17","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991802+0000","last_change":"2024-02-13T03:21:11.680077+0000","last_active":"2024-02-13T03:22:34.991802+0000","last_peered":"2024-02-13T03:22:34.991802+0000","last_clean":"2024-02-13T03:22:34.991802+0000","last_became_active":"2024-02-13T03:21:11.679776+0000","last_became_peered":"2024-02-13T03:21:11.679776+0000","last_unstale":"2024-02-13T03:22:34.991802+0000","last_undegraded":"2024-02-13T03:22:34.991802+0000","last_fullsized":"2024-02-13T03:22:34.991802+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.989851+0000","last_change":"2024-02-13T03:21:15.712029+0000","last_active":"2024-02-13T03:22:34.989851+0000","last_peered":"2024-02-13T03:22:34.989851+0000","last_clean":"2024-02-13T03:22:34.989851+0000","last_became_active":"2024-02-13T03:21:15.711652+0000","last_became_peered":"2024-02-13T03:21:15.711652+0000","last_unstale":"2024-02-13T03:22:34.989851+0000","last_undegraded":"2024-02-13T03:22:34.989851+0000","last_fullsized":"2024-02-13T03:22:34.989851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991588+0000","last_change":"2024-02-13T03:21:13.686808+0000","last_active":"2024-02-13T03:22:34.991588+0000","last_peered":"2024-02-13T03:22:34.991588+0000","last_clean":"2024-02-13T03:22:34.991588+0000","last_became_active":"2024-02-13T03:21:13.685760+0000","last_became_peered":"2024-02-13T03:21:13.685760+0000","last_unstale":"2024-02-13T03:22:34.991588+0000","last_undegraded":"2024-02-13T03:22:34.991588+0000","last_fullsized":"2024-02-13T03:22:34.991588+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996919+0000","last_change":"2024-02-13T03:21:09.674794+0000","last_active":"2024-02-13T03:22:34.996919+0000","last_peered":"2024-02-13T03:22:34.996919+0000","last_clean":"2024-02-13T03:22:34.996919+0000","last_became_active":"2024-02-13T03:21:09.674260+0000","last_became_peered":"2024-02-13T03:21:09.674260+0000","last_unstale":"2024-02-13T03:22:34.996919+0000","last_undegraded":"2024-02-13T03:22:34.996919+0000","last_fullsized":"2024-02-13T03:22:34.996919+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"60'17","reported_seq":45,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991621+0000","last_change":"2024-02-13T03:21:11.682859+0000","last_active":"2024-02-13T03:22:34.991621+0000","last_peered":"2024-02-13T03:22:34.991621+0000","last_clean":"2024-02-13T03:22:34.991621+0000","last_became_active":"2024-02-13T03:21:11.682576+0000","last_became_peered":"2024-02-13T03:21:11.682576+0000","last_unstale":"2024-02-13T03:22:34.991621+0000","last_undegraded":"2024-02-13T03:22:34.991621+0000","last_fullsized":"2024-02-13T03:22:34.991621+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991991+0000","last_change":"2024-02-13T03:21:15.714493+0000","last_active":"2024-02-13T03:22:34.991991+0000","last_peered":"2024-02-13T03:22:34.991991+0000","last_clean":"2024-02-13T03:22:34.991991+0000","last_became_active":"2024-02-13T03:21:15.714169+0000","last_became_peered":"2024-02-13T03:21:15.714169+0000","last_unstale":"2024-02-13T03:22:34.991991+0000","last_undegraded":"2024-02-13T03:22:34.991991+0000","last_fullsized":"2024-02-13T03:22:34.991991+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"60'8","reported_seq":76,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:32.886789+0000","last_change":"2024-02-13T03:21:13.696276+0000","last_active":"2024-02-13T03:25:32.886789+0000","last_peered":"2024-02-13T03:25:32.886789+0000","last_clean":"2024-02-13T03:25:32.886789+0000","last_became_active":"2024-02-13T03:21:13.695849+0000","last_became_peered":"2024-02-13T03:21:13.695849+0000","last_unstale":"2024-02-13T03:25:32.886789+0000","last_undegraded":"2024-02-13T03:25:32.886789+0000","last_fullsized":"2024-02-13T03:25:32.886789+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992534+0000","last_change":"2024-02-13T03:21:09.673847+0000","last_active":"2024-02-13T03:22:34.992534+0000","last_peered":"2024-02-13T03:22:34.992534+0000","last_clean":"2024-02-13T03:22:34.992534+0000","last_became_active":"2024-02-13T03:21:09.673525+0000","last_became_peered":"2024-02-13T03:21:09.673525+0000","last_unstale":"2024-02-13T03:22:34.992534+0000","last_undegraded":"2024-02-13T03:22:34.992534+0000","last_fullsized":"2024-02-13T03:22:34.992534+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"60'10","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991519+0000","last_change":"2024-02-13T03:21:11.679116+0000","last_active":"2024-02-13T03:22:34.991519+0000","last_peered":"2024-02-13T03:22:34.991519+0000","last_clean":"2024-02-13T03:22:34.991519+0000","last_became_active":"2024-02-13T03:21:11.678788+0000","last_became_peered":"2024-02-13T03:21:11.678788+0000","last_unstale":"2024-02-13T03:22:34.991519+0000","last_undegraded":"2024-02-13T03:22:34.991519+0000","last_fullsized":"2024-02-13T03:22:34.991519+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":584,"ondisk_log_size":584,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":264,"num_read_kb":259,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430267,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297724,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":643072,"data_stored":241758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91700000000000004,"15min":0.90100000000000002},"min":{"1min":0.61799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3029999999999999,"5min":1.347,"15min":1.347},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.878,"15min":0.86499999999999999},"min":{"1min":0.69399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.234,"5min":1.363,"15min":1.363},"last":0.81000000000000005}]},{"osd":1,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.94299999999999995,"15min":0.94799999999999995},"min":{"1min":0.625,"5min":0.505,"15min":0.505},"max":{"1min":1.1859999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93500000000000005,"15min":0.94099999999999995},"min":{"1min":0.65500000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.288,"5min":1.381,"15min":1.381},"last":1.345}]},{"osd":2,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.96099999999999997,"15min":0.93500000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.272,"5min":1.452,"15min":1.452},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.95399999999999996,"15min":0.93000000000000005},"min":{"1min":0.746,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3959999999999999,"5min":1.528,"15min":1.528},"last":0.97999999999999998}]},{"osd":3,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91400000000000003,"15min":0.90800000000000003},"min":{"1min":0.60299999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.0700000000000001,"5min":1.389,"15min":1.389},"last":1.141},{"interface":"front","average":{"1min":1.052,"5min":1.0129999999999999,"15min":1.0109999999999999},"min":{"1min":0.79600000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3100000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.90900000000000003}]},{"osd":4,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.96799999999999997,"15min":0.98599999999999999},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2130000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.014,"5min":0.97699999999999998,"15min":0.96099999999999997},"min":{"1min":0.79300000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.252,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.05}]},{"osd":5,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0369999999999999,"15min":1.0569999999999999},"min":{"1min":0.72999999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3340000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.111,"15min":1.0840000000000001},"min":{"1min":0.75600000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.3959999999999999,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.1120000000000001}]},{"osd":6,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1419999999999999,"15min":1.1200000000000001},"min":{"1min":0.77300000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.399,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.278},{"interface":"front","average":{"1min":1.137,"5min":1.095,"15min":1.05},"min":{"1min":0.73299999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.431,"5min":1.534,"15min":1.534},"last":0.95899999999999996}]}]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297704,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447448,"statfs":{"total":95995035648,"available":95690186752,"internally_reserved":0,"allocated":622592,"data_stored":241561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.198,"5min":1.262,"15min":1.262},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.88800000000000001,"15min":0.86399999999999999},"min":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.82899999999999996}]},{"osd":1,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.89300000000000002,"15min":0.93000000000000005},"min":{"1min":0.55500000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.3460000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90000000000000002,"15min":0.89800000000000002},"min":{"1min":0.60399999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.133,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.871}]},{"osd":2,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.93999999999999995,"15min":0.91500000000000004},"min":{"1min":0.752,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.165,"5min":1.292,"15min":1.292},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94399999999999995,"15min":0.94599999999999995},"min":{"1min":0.65700000000000003,"5min":0.627,"15min":0.627},"max":{"1min":1.2010000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93300000000000005,"15min":0.91100000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2589999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.996},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0309999999999999,"15min":1.0069999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3029999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.004}]},{"osd":4,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.91800000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2729999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.998,"5min":0.97799999999999998,"15min":0.91300000000000003},"min":{"1min":0.53100000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.3129999999999999,"5min":1.607,"15min":1.607},"last":0.79100000000000004}]},{"osd":5,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.99099999999999999,"15min":0.997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.244,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0700000000000001,"15min":1.056},"min":{"1min":0.76100000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.355,"5min":1.883,"15min":1.883},"last":0.92900000000000005}]},{"osd":7,"last update":"Tue Feb 13 03:24:59 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.101,"15min":1.0600000000000001},"min":{"1min":0.621,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3979999999999999,"5min":1.575,"15min":1.575},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1639999999999999,"15min":1.1359999999999999},"min":{"1min":0.73599999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.48,"5min":1.931,"15min":1.931},"last":1.159}]}]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297732,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447420,"statfs":{"total":95995035648,"available":95690158080,"internally_reserved":0,"allocated":651264,"data_stored":240761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.71699999999999997,"15min":0.59099999999999997},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.376},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.745,"15min":0.63},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.41399999999999998},"max":{"1min":1.157,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.92800000000000005}]},{"osd":2,"last update":"Tue Feb 13 03:24:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.752,"15min":0.68899999999999995},"min":{"1min":0.52700000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94299999999999995,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.77300000000000002,"15min":0.70999999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.026,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.018}]},{"osd":3,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.83599999999999997,"15min":0.81200000000000006},"min":{"1min":0.45700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.319,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.242},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.84099999999999997,"15min":0.81200000000000006},"min":{"1min":0.63500000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.093}]},{"osd":4,"last update":"Tue Feb 13 03:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.0089999999999999,"15min":0.97099999999999997},"min":{"1min":0.61199999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2230000000000001,"5min":1.587,"15min":1.587},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.159,"5min":1.079,"15min":1.034},"min":{"1min":0.79600000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.1819999999999999}]},{"osd":5,"last update":"Tue Feb 13 03:24:42 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.022,"15min":0.996},"min":{"1min":0.77000000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.26,"5min":1.581,"15min":1.581},"last":1.444},{"interface":"front","average":{"1min":1.091,"5min":1.1140000000000001,"15min":1.0800000000000001},"min":{"1min":0.72299999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.405,"5min":1.675,"15min":1.675},"last":1.4970000000000001}]},{"osd":6,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1779999999999999,"15min":1.1699999999999999},"min":{"1min":0.90000000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.417,"5min":1.734,"15min":1.734},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.1850000000000001,"15min":1.157},"min":{"1min":1.0109999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":1.524}]},{"osd":7,"last update":"Tue Feb 13 03:25:05 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.25,"15min":1.2609999999999999},"min":{"1min":0.95499999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.621,"5min":1.742,"15min":1.742},"last":1.627},{"interface":"front","average":{"1min":1.319,"5min":1.262,"15min":1.268},"min":{"1min":1.0720000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.3700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738484,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297724,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":643072,"data_stored":241110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Feb 13 03:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73799999999999999,"15min":1.075},"min":{"1min":0.44900000000000001,"5min":0.35399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0940000000000001,"5min":1.4690000000000001,"15min":20.219999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76100000000000001,"15min":0.60599999999999998},"min":{"1min":0.55600000000000005,"5min":0.34699999999999998,"15min":0.32000000000000001},"max":{"1min":1.0700000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.82799999999999996}]},{"osd":2,"last update":"Tue Feb 13 03:24:40 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77100000000000002,"15min":0.66700000000000004},"min":{"1min":0.53900000000000003,"5min":0.47399999999999998,"15min":0.28399999999999997},"max":{"1min":1.1299999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":1.032},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.78000000000000003,"15min":0.68500000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.39000000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.3899999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Tue Feb 13 03:25:23 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.83799999999999997,"15min":0.80700000000000005},"min":{"1min":0.64300000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.125,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.008},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88,"15min":0.82199999999999995},"min":{"1min":0.498,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.181,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.67900000000000005}]},{"osd":4,"last update":"Tue Feb 13 03:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89200000000000002,"15min":0.873},"min":{"1min":0.67700000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.26,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1080000000000001,"15min":1.056},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.403,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.133}]},{"osd":5,"last update":"Tue Feb 13 03:24:40 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0600000000000001,"15min":1.0469999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.4390000000000001,"5min":1.617,"15min":1.617},"last":1.256},{"interface":"front","average":{"1min":1.133,"5min":1.107,"15min":1.1319999999999999},"min":{"1min":0.77100000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4850000000000001,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.93000000000000005}]},{"osd":6,"last update":"Tue Feb 13 03:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.133,"15min":1.131},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4510000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.077},{"interface":"front","average":{"1min":1.135,"5min":1.1599999999999999,"15min":1.1930000000000001},"min":{"1min":0.67000000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5549999999999999,"5min":1.718,"15min":1.718},"last":1.3320000000000001}]},{"osd":7,"last update":"Tue Feb 13 03:25:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.044,"15min":1.024},"min":{"1min":0.82199999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2310000000000001,"15min":1.2110000000000001},"min":{"1min":0.78400000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.4399999999999999,"5min":1.661,"15min":1.661},"last":1.198}]}]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297704,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447448,"statfs":{"total":95995035648,"available":95690186752,"internally_reserved":0,"allocated":622592,"data_stored":240928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.69799999999999995,"15min":0.61399999999999999},"min":{"1min":0.55400000000000005,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.996,"5min":1.111,"15min":1.111},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79700000000000004,"15min":0.67900000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.02,"5min":1.794,"15min":1.794},"last":0.81100000000000005}]},{"osd":1,"last update":"Tue Feb 13 03:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.74399999999999999,"15min":0.66400000000000003},"min":{"1min":0.53600000000000003,"5min":0.48499999999999999,"15min":0.376},"max":{"1min":1.1160000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.84999999999999998,"15min":0.73599999999999999},"min":{"1min":0.58299999999999996,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.387,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.1319999999999999}]},{"osd":3,"last update":"Tue Feb 13 03:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.89600000000000002,"15min":0.82999999999999996},"min":{"1min":0.60399999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.444,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.879,"15min":0.80100000000000005},"min":{"1min":0.70099999999999996,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.256,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":0.89500000000000002}]},{"osd":4,"last update":"Tue Feb 13 03:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.097,"15min":0.999},"min":{"1min":0.79400000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.8660000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0640000000000001,"15min":0.97399999999999998},"min":{"1min":0.77700000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.548,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.3919999999999999}]},{"osd":5,"last update":"Tue Feb 13 03:24:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.069,"15min":1.012},"min":{"1min":0.68100000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.724,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.1930000000000001,"15min":1.139},"min":{"1min":0.76300000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":1.2130000000000001}]},{"osd":6,"last update":"Tue Feb 13 03:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.113,"15min":1.0629999999999999},"min":{"1min":0.877,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.53,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.099},{"interface":"front","average":{"1min":1.296,"5min":1.2649999999999999,"15min":1.2310000000000001},"min":{"1min":1.0429999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5920000000000001,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.2889999999999999}]},{"osd":7,"last update":"Tue Feb 13 03:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.145,"15min":1.113},"min":{"1min":0.95299999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.637,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.23},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2749999999999999,"15min":1.2470000000000001},"min":{"1min":0.72499999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.345}]}]},{"osd":3,"up_from":25,"seq":107374182493,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297732,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447420,"statfs":{"total":95995035648,"available":95690158080,"internally_reserved":0,"allocated":651264,"data_stored":241752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:25:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.73399999999999999,"15min":0.67800000000000005},"min":{"1min":0.42599999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96899999999999997,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.54800000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1259999999999999,"5min":1.208,"15min":1.208},"last":1.7609999999999999}]},{"osd":1,"last update":"Tue Feb 13 03:25:30 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.751,"15min":0.79300000000000004},"min":{"1min":0.52000000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.2949999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78600000000000003,"15min":0.76600000000000001},"min":{"1min":0.58599999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1719999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.71499999999999997}]},{"osd":2,"last update":"Tue Feb 13 03:25:30 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85099999999999998,"15min":0.83899999999999997},"min":{"1min":0.61599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2529999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":1.6870000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.48899999999999999,"5min":0.38,"15min":0.38},"max":{"1min":1.042,"5min":1.214,"15min":1.214},"last":1.5860000000000001}]},{"osd":4,"last update":"Tue Feb 13 03:24:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.92700000000000005,"15min":0.89700000000000002},"min":{"1min":0.66400000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.2},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.97599999999999998,"15min":0.92200000000000004},"min":{"1min":0.748,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.234,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.0369999999999999}]},{"osd":5,"last update":"Tue Feb 13 03:24:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0620000000000001,"15min":1.034},"min":{"1min":0.76100000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.397,"5min":2.468,"15min":2.468},"last":1.23},{"interface":"front","average":{"1min":0.996,"5min":1.026,"15min":1.0169999999999999},"min":{"1min":0.746,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.21,"5min":1.486,"15min":1.486},"last":1.292}]},{"osd":6,"last update":"Tue Feb 13 03:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.0720000000000001,"15min":1.0469999999999999},"min":{"1min":0.84799999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":1.258},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.173,"15min":1.234},"min":{"1min":0.79100000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5169999999999999,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":1.077}]},{"osd":7,"last update":"Tue Feb 13 03:25:05 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.093,"15min":1.0720000000000001},"min":{"1min":0.85799999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.133,"15min":1.1359999999999999},"min":{"1min":0.81899999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.379,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.423}]}]},{"osd":4,"up_from":30,"seq":128849018964,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297724,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":643072,"data_stored":241220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.83599999999999997,"15min":0.79400000000000004},"min":{"1min":0.63400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.077,"5min":1.357,"15min":1.357},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.92900000000000005,"15min":0.83499999999999996},"min":{"1min":0.64100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.29}]},{"osd":1,"last update":"Tue Feb 13 03:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.86599999999999999,"15min":0.81000000000000005},"min":{"1min":0.65300000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.1399999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.89100000000000001,"15min":0.86299999999999999},"min":{"1min":0.68799999999999994,"5min":0.372,"15min":0.372},"max":{"1min":1.246,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.131}]},{"osd":2,"last update":"Tue Feb 13 03:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.89300000000000002,"15min":0.86499999999999999},"min":{"1min":0.59599999999999997,"5min":0.314,"15min":0.314},"max":{"1min":1.1699999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.016,"5min":0.97799999999999998,"15min":0.90600000000000003},"min":{"1min":0.73699999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.22,"5min":1.577,"15min":1.577},"last":1.1890000000000001}]},{"osd":3,"last update":"Tue Feb 13 03:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":0.99299999999999999,"15min":0.94699999999999995},"min":{"1min":0.77200000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.393,"5min":1.681,"15min":1.681},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.0209999999999999,"15min":0.93999999999999995},"min":{"1min":0.73099999999999998,"5min":0.622,"15min":0.622},"max":{"1min":1.268,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.94899999999999995}]},{"osd":5,"last update":"Tue Feb 13 03:25:39 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.92900000000000005,"15min":0.80200000000000005},"min":{"1min":0.68000000000000005,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2190000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.91600000000000004,"15min":0.77000000000000002},"min":{"1min":0.69599999999999995,"5min":0.47899999999999998,"15min":0.46000000000000002},"max":{"1min":1.206,"5min":1.252,"15min":1.252},"last":1.165}]},{"osd":6,"last update":"Tue Feb 13 03:25:24 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.95699999999999996,"15min":0.90500000000000003},"min":{"1min":0.81699999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.23,"5min":1.504,"15min":1.504},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.038,"15min":0.97199999999999998},"min":{"1min":0.76000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.997}]},{"osd":7,"last update":"Tue Feb 13 03:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0569999999999999,"15min":1.054},"min":{"1min":0.89100000000000001,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3320000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.083,"15min":1.0680000000000001},"min":{"1min":0.92400000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.329,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.2230000000000001}]}]},{"osd":5,"up_from":36,"seq":154618822732,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297704,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447448,"statfs":{"total":95995035648,"available":95690186752,"internally_reserved":0,"allocated":622592,"data_stored":240463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:24:39 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.86599999999999999,"15min":0.80900000000000005},"min":{"1min":0.67300000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2350000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.89800000000000002,"15min":0.90700000000000003},"min":{"1min":0.69299999999999995,"5min":0.315,"15min":0.315},"max":{"1min":1.2669999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.0640000000000001}]},{"osd":1,"last update":"Tue Feb 13 03:24:39 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.88300000000000001,"15min":0.85899999999999999},"min":{"1min":0.64800000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.1839999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":1.167},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.92100000000000004,"15min":0.90300000000000002},"min":{"1min":0.72499999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.347,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.147}]},{"osd":2,"last update":"Tue Feb 13 03:24:39 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.88400000000000001,"15min":0.86099999999999999},"min":{"1min":0.60299999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.214,"5min":1.385,"15min":1.385},"last":1.2669999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.91300000000000003,"15min":0.88500000000000001},"min":{"1min":0.624,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.163,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.997}]},{"osd":3,"last update":"Tue Feb 13 03:24:39 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.90900000000000003,"15min":0.877},"min":{"1min":0.748,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.2250000000000001,"5min":1.351,"15min":1.351},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.96099999999999997,"15min":0.92300000000000004},"min":{"1min":0.77300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.212,"5min":3.0779999999999998,"15min":3.0779999999999998},"last":0.76800000000000002}]},{"osd":4,"last update":"Tue Feb 13 03:24:39 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.879,"15min":0.81999999999999995},"min":{"1min":0.71299999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.27,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.91100000000000003,"15min":0.84499999999999997},"min":{"1min":0.70299999999999996,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3109999999999999,"5min":1.468,"15min":1.468},"last":1.117}]},{"osd":6,"last update":"Tue Feb 13 03:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.96599999999999997,"15min":0.89600000000000002},"min":{"1min":0.92500000000000004,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.405,"5min":1.52,"15min":1.52},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.999,"15min":0.94299999999999995},"min":{"1min":0.68500000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.091}]},{"osd":7,"last update":"Tue Feb 13 03:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.0569999999999999,"15min":1.012},"min":{"1min":0.90700000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3779999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.0600000000000001,"15min":0.98899999999999999},"min":{"1min":0.89800000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5940000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-13T03:25:43.829 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-13T03:25:45.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:45 smithi183 bash[20824]: cluster 2024-02-13T03:25:44.662056+0000 mgr.y (mgr.24473) 178 : cluster [DBG] pgmap v130: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:45.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:25:45 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:25:46.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:45 smithi161 bash[19769]: cluster 2024-02-13T03:25:44.662056+0000 mgr.y (mgr.24473) 178 : cluster [DBG] pgmap v130: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:46.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:45 smithi161 bash[27879]: cluster 2024-02-13T03:25:44.662056+0000 mgr.y (mgr.24473) 178 : cluster [DBG] pgmap v130: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:46.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:46 smithi183 bash[20824]: audit 2024-02-13T03:25:45.735545+0000 mgr.y (mgr.24473) 179 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:47.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:46 smithi161 bash[27879]: audit 2024-02-13T03:25:45.735545+0000 mgr.y (mgr.24473) 179 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:47.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:46 smithi161 bash[19769]: audit 2024-02-13T03:25:45.735545+0000 mgr.y (mgr.24473) 179 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:47.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:47 smithi183 bash[20824]: cluster 2024-02-13T03:25:46.663507+0000 mgr.y (mgr.24473) 180 : cluster [DBG] pgmap v131: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:48.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:47 smithi161 bash[19769]: cluster 2024-02-13T03:25:46.663507+0000 mgr.y (mgr.24473) 180 : cluster [DBG] pgmap v131: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:48.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:47 smithi161 bash[27879]: cluster 2024-02-13T03:25:46.663507+0000 mgr.y (mgr.24473) 180 : cluster [DBG] pgmap v131: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:48.713 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:25:49.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:25:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:25:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:25:49.905 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:49 smithi161 bash[27879]: cluster 2024-02-13T03:25:48.664082+0000 mgr.y (mgr.24473) 181 : cluster [DBG] pgmap v132: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:49.905 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:49 smithi161 bash[19769]: cluster 2024-02-13T03:25:48.664082+0000 mgr.y (mgr.24473) 181 : cluster [DBG] pgmap v132: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:50.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:49 smithi183 bash[20824]: cluster 2024-02-13T03:25:48.664082+0000 mgr.y (mgr.24473) 181 : cluster [DBG] pgmap v132: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:25:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:25:51] "GET /metrics HTTP/1.1" 200 172184 "" "Prometheus/2.33.4" 2024-02-13T03:25:51.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f262a1d0700 1 -- 172.21.15.161:0/3466514261 >> v1:172.21.15.161:6789/0 conn(0x7f262410ad80 legacy=0x7f262410d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:51.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f262a1d0700 1 -- 172.21.15.161:0/3466514261 shutdown_connections 2024-02-13T03:25:51.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f262a1d0700 1 -- 172.21.15.161:0/3466514261 >> 172.21.15.161:0/3466514261 conn(0x7f26240f36d0 msgr2=0x7f26240f5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:51.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f262a1d0700 1 -- 172.21.15.161:0/3466514261 shutdown_connections 2024-02-13T03:25:51.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f262a1d0700 1 -- 172.21.15.161:0/3466514261 wait complete. 2024-02-13T03:25:51.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f262a1d0700 1 Processor -- start 2024-02-13T03:25:51.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f262a1d0700 1 -- start start 2024-02-13T03:25:51.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f262a1d0700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f262411d9d0 con 0x7f262406b920 2024-02-13T03:25:51.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f262a1d0700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f262411ebb0 con 0x7f262410ad80 2024-02-13T03:25:51.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f262a1d0700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f262411fdb0 con 0x7f262406ccc0 2024-02-13T03:25:51.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f26237fe700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f262406b920 0x7f2624106fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53156/0 (socket says 172.21.15.161:53156) 2024-02-13T03:25:51.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f26237fe700 1 -- 172.21.15.161:0/3054618815 learned_addr learned my addr 172.21.15.161:0/3054618815 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:25:51.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312953114 0 0) 0x7f262411d9d0 con 0x7f262406b920 2024-02-13T03:25:51.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2610003030 con 0x7f262406b920 2024-02-13T03:25:51.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295055047 0 0) 0x7f262411fdb0 con 0x7f262406ccc0 2024-02-13T03:25:51.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f262411d9d0 con 0x7f262406ccc0 2024-02-13T03:25:51.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906949815 0 0) 0x7f262411ebb0 con 0x7f262410ad80 2024-02-13T03:25:51.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f262411fdb0 con 0x7f262410ad80 2024-02-13T03:25:51.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 513550490 0 0) 0x7f2610003030 con 0x7f262406b920 2024-02-13T03:25:51.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f262411ebb0 con 0x7f262406b920 2024-02-13T03:25:51.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f261c002d50 con 0x7f262406b920 2024-02-13T03:25:51.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4141583072 0 0) 0x7f262411d9d0 con 0x7f262406ccc0 2024-02-13T03:25:51.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2610003030 con 0x7f262406ccc0 2024-02-13T03:25:51.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f260c0018e0 con 0x7f262406ccc0 2024-02-13T03:25:51.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 885080554 0 0) 0x7f262411ebb0 con 0x7f262406b920 2024-02-13T03:25:51.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 >> v1:172.21.15.161:6790/0 conn(0x7f262406ccc0 legacy=0x7f26241076f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:51.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.460+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 >> v1:172.21.15.183:6789/0 conn(0x7f262410ad80 legacy=0x7f262411c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:51.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.464+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2624120fb0 con 0x7f262406b920 2024-02-13T03:25:51.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.464+0000 7f262a1d0700 1 -- 172.21.15.161:0/3054618815 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262411ffa0 con 0x7f262406b920 2024-02-13T03:25:51.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.464+0000 7f262a1d0700 1 -- 172.21.15.161:0/3054618815 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2624120410 con 0x7f262406b920 2024-02-13T03:25:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.464+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f261c003120 con 0x7f262406b920 2024-02-13T03:25:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.464+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f261c005ef0 con 0x7f262406b920 2024-02-13T03:25:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.464+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f261c002060 con 0x7f262406b920 2024-02-13T03:25:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.464+0000 7f262a1d0700 1 -- 172.21.15.161:0/3054618815 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2604004f20 con 0x7f262406b920 2024-02-13T03:25:51.470 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.464+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f261c078a90 con 0x7f262406b920 2024-02-13T03:25:51.478 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.472+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f261c04c600 con 0x7f262406b920 2024-02-13T03:25:51.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.648+0000 7f262a1d0700 1 -- 172.21.15.161:0/3054618815 --> v1:172.21.15.161:6800/4114337139 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2604000b90 con 0x7f2610062250 2024-02-13T03:25:51.673 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.668+0000 7f2620ff9700 1 -- 172.21.15.161:0/3054618815 <== mgr.24473 v1:172.21.15.161:6800/4114337139 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+355492 (unknown 2965378022 0 674488417) 0x7f2604000b90 con 0x7f2610062250 2024-02-13T03:25:51.675 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-02-13T03:25:51.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.676+0000 7f262a1d0700 1 -- 172.21.15.161:0/3054618815 >> v1:172.21.15.161:6800/4114337139 conn(0x7f2610062250 legacy=0x7f2610064710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:51.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.676+0000 7f262a1d0700 1 -- 172.21.15.161:0/3054618815 >> v1:172.21.15.161:6789/0 conn(0x7f262406b920 legacy=0x7f2624106fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:25:51.682 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.676+0000 7f262a1d0700 1 -- 172.21.15.161:0/3054618815 shutdown_connections 2024-02-13T03:25:51.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.676+0000 7f262a1d0700 1 -- 172.21.15.161:0/3054618815 >> 172.21.15.161:0/3054618815 conn(0x7f26240f36d0 msgr2=0x7f26240f4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:25:51.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.676+0000 7f262a1d0700 1 -- 172.21.15.161:0/3054618815 shutdown_connections 2024-02-13T03:25:51.683 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:25:51.676+0000 7f262a1d0700 1 -- 172.21.15.161:0/3054618815 wait complete. 2024-02-13T03:25:51.683 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2024-02-13T03:25:52.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:51 smithi161 bash[19769]: cluster 2024-02-13T03:25:50.665449+0000 mgr.y (mgr.24473) 182 : cluster [DBG] pgmap v133: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:52.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:51 smithi161 bash[27879]: cluster 2024-02-13T03:25:50.665449+0000 mgr.y (mgr.24473) 182 : cluster [DBG] pgmap v133: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:52.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:51 smithi183 bash[20824]: cluster 2024-02-13T03:25:50.665449+0000 mgr.y (mgr.24473) 182 : cluster [DBG] pgmap v133: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:52.321 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-02-13T03:25:50.664352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1378,"num_read_kb":1098,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":659,"ondisk_log_size":659,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2381748,"kb_used_data":4980,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747579468,"statfs":{"total":767960285184,"available":765521375232,"internally_reserved":0,"allocated":5099520,"data_stored":1929553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005173"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990699+0000","last_change":"2024-02-13T03:21:15.714391+0000","last_active":"2024-02-13T03:22:34.990699+0000","last_peered":"2024-02-13T03:22:34.990699+0000","last_clean":"2024-02-13T03:22:34.990699+0000","last_became_active":"2024-02-13T03:21:15.714078+0000","last_became_peered":"2024-02-13T03:21:15.714078+0000","last_unstale":"2024-02-13T03:22:34.990699+0000","last_undegraded":"2024-02-13T03:22:34.990699+0000","last_fullsized":"2024-02-13T03:22:34.990699+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991314+0000","last_change":"2024-02-13T03:21:13.690573+0000","last_active":"2024-02-13T03:22:34.991314+0000","last_peered":"2024-02-13T03:22:34.991314+0000","last_clean":"2024-02-13T03:22:34.991314+0000","last_became_active":"2024-02-13T03:21:13.690185+0000","last_became_peered":"2024-02-13T03:21:13.690185+0000","last_unstale":"2024-02-13T03:22:34.991314+0000","last_undegraded":"2024-02-13T03:22:34.991314+0000","last_fullsized":"2024-02-13T03:22:34.991314+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985536+0000","last_change":"2024-02-13T03:21:09.891039+0000","last_active":"2024-02-13T03:22:34.985536+0000","last_peered":"2024-02-13T03:22:34.985536+0000","last_clean":"2024-02-13T03:22:34.985536+0000","last_became_active":"2024-02-13T03:21:09.890276+0000","last_became_peered":"2024-02-13T03:21:09.890276+0000","last_unstale":"2024-02-13T03:22:34.985536+0000","last_undegraded":"2024-02-13T03:22:34.985536+0000","last_fullsized":"2024-02-13T03:22:34.985536+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"60'15","reported_seq":48,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990605+0000","last_change":"2024-02-13T03:21:11.682044+0000","last_active":"2024-02-13T03:22:34.990605+0000","last_peered":"2024-02-13T03:22:34.990605+0000","last_clean":"2024-02-13T03:22:34.990605+0000","last_became_active":"2024-02-13T03:21:11.681785+0000","last_became_peered":"2024-02-13T03:21:11.681785+0000","last_unstale":"2024-02-13T03:22:34.990605+0000","last_undegraded":"2024-02-13T03:22:34.990605+0000","last_fullsized":"2024-02-13T03:22:34.990605+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991900+0000","last_change":"2024-02-13T03:21:15.705996+0000","last_active":"2024-02-13T03:22:34.991900+0000","last_peered":"2024-02-13T03:22:34.991900+0000","last_clean":"2024-02-13T03:22:34.991900+0000","last_became_active":"2024-02-13T03:21:15.705646+0000","last_became_peered":"2024-02-13T03:21:15.705646+0000","last_unstale":"2024-02-13T03:22:34.991900+0000","last_undegraded":"2024-02-13T03:22:34.991900+0000","last_fullsized":"2024-02-13T03:22:34.991900+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990548+0000","last_change":"2024-02-13T03:21:13.687658+0000","last_active":"2024-02-13T03:22:34.990548+0000","last_peered":"2024-02-13T03:22:34.990548+0000","last_clean":"2024-02-13T03:22:34.990548+0000","last_became_active":"2024-02-13T03:21:13.687359+0000","last_became_peered":"2024-02-13T03:21:13.687359+0000","last_unstale":"2024-02-13T03:22:34.990548+0000","last_undegraded":"2024-02-13T03:22:34.990548+0000","last_fullsized":"2024-02-13T03:22:34.990548+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"53'1","reported_seq":26,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992041+0000","last_change":"2024-02-13T03:21:09.890345+0000","last_active":"2024-02-13T03:22:34.992041+0000","last_peered":"2024-02-13T03:22:34.992041+0000","last_clean":"2024-02-13T03:22:34.992041+0000","last_became_active":"2024-02-13T03:21:09.890016+0000","last_became_peered":"2024-02-13T03:21:09.890016+0000","last_unstale":"2024-02-13T03:22:34.992041+0000","last_undegraded":"2024-02-13T03:22:34.992041+0000","last_fullsized":"2024-02-13T03:22:34.992041+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"60'39","reported_seq":102,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992506+0000","last_change":"2024-02-13T03:21:11.681664+0000","last_active":"2024-02-13T03:22:34.992506+0000","last_peered":"2024-02-13T03:22:34.992506+0000","last_clean":"2024-02-13T03:22:34.992506+0000","last_became_active":"2024-02-13T03:21:11.680975+0000","last_became_peered":"2024-02-13T03:21:11.680975+0000","last_unstale":"2024-02-13T03:22:34.992506+0000","last_undegraded":"2024-02-13T03:22:34.992506+0000","last_fullsized":"2024-02-13T03:22:34.992506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.983790+0000","last_change":"2024-02-13T03:21:15.709923+0000","last_active":"2024-02-13T03:22:34.983790+0000","last_peered":"2024-02-13T03:22:34.983790+0000","last_clean":"2024-02-13T03:22:34.983790+0000","last_became_active":"2024-02-13T03:21:15.709397+0000","last_became_peered":"2024-02-13T03:21:15.709397+0000","last_unstale":"2024-02-13T03:22:34.983790+0000","last_undegraded":"2024-02-13T03:22:34.983790+0000","last_fullsized":"2024-02-13T03:22:34.983790+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.995548+0000","last_change":"2024-02-13T03:21:13.699585+0000","last_active":"2024-02-13T03:22:34.995548+0000","last_peered":"2024-02-13T03:22:34.995548+0000","last_clean":"2024-02-13T03:22:34.995548+0000","last_became_active":"2024-02-13T03:21:13.699118+0000","last_became_peered":"2024-02-13T03:21:13.699118+0000","last_unstale":"2024-02-13T03:22:34.995548+0000","last_undegraded":"2024-02-13T03:22:34.995548+0000","last_fullsized":"2024-02-13T03:22:34.995548+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996579+0000","last_change":"2024-02-13T03:21:09.682172+0000","last_active":"2024-02-13T03:22:34.996579+0000","last_peered":"2024-02-13T03:22:34.996579+0000","last_clean":"2024-02-13T03:22:34.996579+0000","last_became_active":"2024-02-13T03:21:09.681813+0000","last_became_peered":"2024-02-13T03:21:09.681813+0000","last_unstale":"2024-02-13T03:22:34.996579+0000","last_undegraded":"2024-02-13T03:22:34.996579+0000","last_fullsized":"2024-02-13T03:22:34.996579+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"60'21","reported_seq":63,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991060+0000","last_change":"2024-02-13T03:21:11.677665+0000","last_active":"2024-02-13T03:22:34.991060+0000","last_peered":"2024-02-13T03:22:34.991060+0000","last_clean":"2024-02-13T03:22:34.991060+0000","last_became_active":"2024-02-13T03:21:11.677317+0000","last_became_peered":"2024-02-13T03:21:11.677317+0000","last_unstale":"2024-02-13T03:22:34.991060+0000","last_undegraded":"2024-02-13T03:22:34.991060+0000","last_fullsized":"2024-02-13T03:22:34.991060+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.994910+0000","last_change":"2024-02-13T03:21:15.719993+0000","last_active":"2024-02-13T03:22:34.994910+0000","last_peered":"2024-02-13T03:22:34.994910+0000","last_clean":"2024-02-13T03:22:34.994910+0000","last_became_active":"2024-02-13T03:21:15.719676+0000","last_became_peered":"2024-02-13T03:21:15.719676+0000","last_unstale":"2024-02-13T03:22:34.994910+0000","last_undegraded":"2024-02-13T03:22:34.994910+0000","last_fullsized":"2024-02-13T03:22:34.994910+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987529+0000","last_change":"2024-02-13T03:21:13.699675+0000","last_active":"2024-02-13T03:22:34.987529+0000","last_peered":"2024-02-13T03:22:34.987529+0000","last_clean":"2024-02-13T03:22:34.987529+0000","last_became_active":"2024-02-13T03:21:13.699297+0000","last_became_peered":"2024-02-13T03:21:13.699297+0000","last_unstale":"2024-02-13T03:22:34.987529+0000","last_undegraded":"2024-02-13T03:22:34.987529+0000","last_fullsized":"2024-02-13T03:22:34.987529+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.995450+0000","last_change":"2024-02-13T03:21:09.681823+0000","last_active":"2024-02-13T03:22:34.995450+0000","last_peered":"2024-02-13T03:22:34.995450+0000","last_clean":"2024-02-13T03:22:34.995450+0000","last_became_active":"2024-02-13T03:21:09.681494+0000","last_became_peered":"2024-02-13T03:21:09.681494+0000","last_unstale":"2024-02-13T03:22:34.995450+0000","last_undegraded":"2024-02-13T03:22:34.995450+0000","last_fullsized":"2024-02-13T03:22:34.995450+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"60'11","reported_seq":59,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:22:36.696008+0000","last_change":"2024-02-13T03:22:36.696008+0000","last_active":"2024-02-13T03:22:36.696008+0000","last_peered":"2024-02-13T03:22:36.696008+0000","last_clean":"2024-02-13T03:22:36.696008+0000","last_became_active":"2024-02-13T03:22:36.001245+0000","last_became_peered":"2024-02-13T03:22:36.001245+0000","last_unstale":"2024-02-13T03:22:36.696008+0000","last_undegraded":"2024-02-13T03:22:36.696008+0000","last_fullsized":"2024-02-13T03:22:36.696008+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991941+0000","last_change":"2024-02-13T03:21:15.719021+0000","last_active":"2024-02-13T03:22:34.991941+0000","last_peered":"2024-02-13T03:22:34.991941+0000","last_clean":"2024-02-13T03:22:34.991941+0000","last_became_active":"2024-02-13T03:21:15.718779+0000","last_became_peered":"2024-02-13T03:21:15.718779+0000","last_unstale":"2024-02-13T03:22:34.991941+0000","last_undegraded":"2024-02-13T03:22:34.991941+0000","last_fullsized":"2024-02-13T03:22:34.991941+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992029+0000","last_change":"2024-02-13T03:21:13.686451+0000","last_active":"2024-02-13T03:22:34.992029+0000","last_peered":"2024-02-13T03:22:34.992029+0000","last_clean":"2024-02-13T03:22:34.992029+0000","last_became_active":"2024-02-13T03:21:13.686100+0000","last_became_peered":"2024-02-13T03:21:13.686100+0000","last_unstale":"2024-02-13T03:22:34.992029+0000","last_undegraded":"2024-02-13T03:22:34.992029+0000","last_fullsized":"2024-02-13T03:22:34.992029+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985141+0000","last_change":"2024-02-13T03:21:09.672561+0000","last_active":"2024-02-13T03:22:34.985141+0000","last_peered":"2024-02-13T03:22:34.985141+0000","last_clean":"2024-02-13T03:22:34.985141+0000","last_became_active":"2024-02-13T03:21:09.672137+0000","last_became_peered":"2024-02-13T03:21:09.672137+0000","last_unstale":"2024-02-13T03:22:34.985141+0000","last_undegraded":"2024-02-13T03:22:34.985141+0000","last_fullsized":"2024-02-13T03:22:34.985141+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"60'15","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.989691+0000","last_change":"2024-02-13T03:21:11.684010+0000","last_active":"2024-02-13T03:22:34.989691+0000","last_peered":"2024-02-13T03:22:34.989691+0000","last_clean":"2024-02-13T03:22:34.989691+0000","last_became_active":"2024-02-13T03:21:11.683615+0000","last_became_peered":"2024-02-13T03:21:11.683615+0000","last_unstale":"2024-02-13T03:22:34.989691+0000","last_undegraded":"2024-02-13T03:22:34.989691+0000","last_fullsized":"2024-02-13T03:22:34.989691+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991745+0000","last_change":"2024-02-13T03:21:15.713802+0000","last_active":"2024-02-13T03:22:34.991745+0000","last_peered":"2024-02-13T03:22:34.991745+0000","last_clean":"2024-02-13T03:22:34.991745+0000","last_became_active":"2024-02-13T03:21:15.713415+0000","last_became_peered":"2024-02-13T03:21:15.713415+0000","last_unstale":"2024-02-13T03:22:34.991745+0000","last_undegraded":"2024-02-13T03:22:34.991745+0000","last_fullsized":"2024-02-13T03:22:34.991745+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991265+0000","last_change":"2024-02-13T03:21:13.698026+0000","last_active":"2024-02-13T03:22:34.991265+0000","last_peered":"2024-02-13T03:22:34.991265+0000","last_clean":"2024-02-13T03:22:34.991265+0000","last_became_active":"2024-02-13T03:21:13.697702+0000","last_became_peered":"2024-02-13T03:21:13.697702+0000","last_unstale":"2024-02-13T03:22:34.991265+0000","last_undegraded":"2024-02-13T03:22:34.991265+0000","last_fullsized":"2024-02-13T03:22:34.991265+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991177+0000","last_change":"2024-02-13T03:21:09.891507+0000","last_active":"2024-02-13T03:22:34.991177+0000","last_peered":"2024-02-13T03:22:34.991177+0000","last_clean":"2024-02-13T03:22:34.991177+0000","last_became_active":"2024-02-13T03:21:09.891057+0000","last_became_peered":"2024-02-13T03:21:09.891057+0000","last_unstale":"2024-02-13T03:22:34.991177+0000","last_undegraded":"2024-02-13T03:22:34.991177+0000","last_fullsized":"2024-02-13T03:22:34.991177+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"60'18","reported_seq":55,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991486+0000","last_change":"2024-02-13T03:21:11.676921+0000","last_active":"2024-02-13T03:22:34.991486+0000","last_peered":"2024-02-13T03:22:34.991486+0000","last_clean":"2024-02-13T03:22:34.991486+0000","last_became_active":"2024-02-13T03:21:11.674876+0000","last_became_peered":"2024-02-13T03:21:11.674876+0000","last_unstale":"2024-02-13T03:22:34.991486+0000","last_undegraded":"2024-02-13T03:22:34.991486+0000","last_fullsized":"2024-02-13T03:22:34.991486+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988155+0000","last_change":"2024-02-13T03:21:15.719273+0000","last_active":"2024-02-13T03:22:34.988155+0000","last_peered":"2024-02-13T03:22:34.988155+0000","last_clean":"2024-02-13T03:22:34.988155+0000","last_became_active":"2024-02-13T03:21:15.718838+0000","last_became_peered":"2024-02-13T03:21:15.718838+0000","last_unstale":"2024-02-13T03:22:34.988155+0000","last_undegraded":"2024-02-13T03:22:34.988155+0000","last_fullsized":"2024-02-13T03:22:34.988155+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"60'8","reported_seq":81,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:42.898094+0000","last_change":"2024-02-13T03:21:13.697995+0000","last_active":"2024-02-13T03:25:42.898094+0000","last_peered":"2024-02-13T03:25:42.898094+0000","last_clean":"2024-02-13T03:25:42.898094+0000","last_became_active":"2024-02-13T03:21:13.696479+0000","last_became_peered":"2024-02-13T03:21:13.696479+0000","last_unstale":"2024-02-13T03:25:42.898094+0000","last_undegraded":"2024-02-13T03:25:42.898094+0000","last_fullsized":"2024-02-13T03:25:42.898094+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991152+0000","last_change":"2024-02-13T03:21:09.672365+0000","last_active":"2024-02-13T03:22:34.991152+0000","last_peered":"2024-02-13T03:22:34.991152+0000","last_clean":"2024-02-13T03:22:34.991152+0000","last_became_active":"2024-02-13T03:21:09.671995+0000","last_became_peered":"2024-02-13T03:21:09.671995+0000","last_unstale":"2024-02-13T03:22:34.991152+0000","last_undegraded":"2024-02-13T03:22:34.991152+0000","last_fullsized":"2024-02-13T03:22:34.991152+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"60'16","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985274+0000","last_change":"2024-02-13T03:21:12.019391+0000","last_active":"2024-02-13T03:22:34.985274+0000","last_peered":"2024-02-13T03:22:34.985274+0000","last_clean":"2024-02-13T03:22:34.985274+0000","last_became_active":"2024-02-13T03:21:12.018959+0000","last_became_peered":"2024-02-13T03:21:12.018959+0000","last_unstale":"2024-02-13T03:22:34.985274+0000","last_undegraded":"2024-02-13T03:22:34.985274+0000","last_fullsized":"2024-02-13T03:22:34.985274+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991751+0000","last_change":"2024-02-13T03:21:15.716547+0000","last_active":"2024-02-13T03:22:34.991751+0000","last_peered":"2024-02-13T03:22:34.991751+0000","last_clean":"2024-02-13T03:22:34.991751+0000","last_became_active":"2024-02-13T03:21:15.716194+0000","last_became_peered":"2024-02-13T03:21:15.716194+0000","last_unstale":"2024-02-13T03:22:34.991751+0000","last_undegraded":"2024-02-13T03:22:34.991751+0000","last_fullsized":"2024-02-13T03:22:34.991751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985537+0000","last_change":"2024-02-13T03:21:13.691442+0000","last_active":"2024-02-13T03:22:34.985537+0000","last_peered":"2024-02-13T03:22:34.985537+0000","last_clean":"2024-02-13T03:22:34.985537+0000","last_became_active":"2024-02-13T03:21:13.691023+0000","last_became_peered":"2024-02-13T03:21:13.691023+0000","last_unstale":"2024-02-13T03:22:34.985537+0000","last_undegraded":"2024-02-13T03:22:34.985537+0000","last_fullsized":"2024-02-13T03:22:34.985537+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992030+0000","last_change":"2024-02-13T03:21:09.666829+0000","last_active":"2024-02-13T03:22:34.992030+0000","last_peered":"2024-02-13T03:22:34.992030+0000","last_clean":"2024-02-13T03:22:34.992030+0000","last_became_active":"2024-02-13T03:21:09.666424+0000","last_became_peered":"2024-02-13T03:21:09.666424+0000","last_unstale":"2024-02-13T03:22:34.992030+0000","last_undegraded":"2024-02-13T03:22:34.992030+0000","last_fullsized":"2024-02-13T03:22:34.992030+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'17","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.997217+0000","last_change":"2024-02-13T03:21:11.681384+0000","last_active":"2024-02-13T03:22:34.997217+0000","last_peered":"2024-02-13T03:22:34.997217+0000","last_clean":"2024-02-13T03:22:34.997217+0000","last_became_active":"2024-02-13T03:21:11.681073+0000","last_became_peered":"2024-02-13T03:21:11.681073+0000","last_unstale":"2024-02-13T03:22:34.997217+0000","last_undegraded":"2024-02-13T03:22:34.997217+0000","last_fullsized":"2024-02-13T03:22:34.997217+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987374+0000","last_change":"2024-02-13T03:21:15.719544+0000","last_active":"2024-02-13T03:22:34.987374+0000","last_peered":"2024-02-13T03:22:34.987374+0000","last_clean":"2024-02-13T03:22:34.987374+0000","last_became_active":"2024-02-13T03:21:15.719320+0000","last_became_peered":"2024-02-13T03:21:15.719320+0000","last_unstale":"2024-02-13T03:22:34.987374+0000","last_undegraded":"2024-02-13T03:22:34.987374+0000","last_fullsized":"2024-02-13T03:22:34.987374+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.995789+0000","last_change":"2024-02-13T03:21:13.692644+0000","last_active":"2024-02-13T03:22:34.995789+0000","last_peered":"2024-02-13T03:22:34.995789+0000","last_clean":"2024-02-13T03:22:34.995789+0000","last_became_active":"2024-02-13T03:21:13.692203+0000","last_became_peered":"2024-02-13T03:21:13.692203+0000","last_unstale":"2024-02-13T03:22:34.995789+0000","last_undegraded":"2024-02-13T03:22:34.995789+0000","last_fullsized":"2024-02-13T03:22:34.995789+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985430+0000","last_change":"2024-02-13T03:21:09.675350+0000","last_active":"2024-02-13T03:22:34.985430+0000","last_peered":"2024-02-13T03:22:34.985430+0000","last_clean":"2024-02-13T03:22:34.985430+0000","last_became_active":"2024-02-13T03:21:09.674981+0000","last_became_peered":"2024-02-13T03:21:09.674981+0000","last_unstale":"2024-02-13T03:22:34.985430+0000","last_undegraded":"2024-02-13T03:22:34.985430+0000","last_fullsized":"2024-02-13T03:22:34.985430+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"60'4","reported_seq":23,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992136+0000","last_change":"2024-02-13T03:21:11.675505+0000","last_active":"2024-02-13T03:22:34.992136+0000","last_peered":"2024-02-13T03:22:34.992136+0000","last_clean":"2024-02-13T03:22:34.992136+0000","last_became_active":"2024-02-13T03:21:11.675052+0000","last_became_peered":"2024-02-13T03:21:11.675052+0000","last_unstale":"2024-02-13T03:22:34.992136+0000","last_undegraded":"2024-02-13T03:22:34.992136+0000","last_fullsized":"2024-02-13T03:22:34.992136+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991222+0000","last_change":"2024-02-13T03:21:15.701933+0000","last_active":"2024-02-13T03:22:34.991222+0000","last_peered":"2024-02-13T03:22:34.991222+0000","last_clean":"2024-02-13T03:22:34.991222+0000","last_became_active":"2024-02-13T03:21:15.701606+0000","last_became_peered":"2024-02-13T03:21:15.701606+0000","last_unstale":"2024-02-13T03:22:34.991222+0000","last_undegraded":"2024-02-13T03:22:34.991222+0000","last_fullsized":"2024-02-13T03:22:34.991222+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987582+0000","last_change":"2024-02-13T03:21:13.698530+0000","last_active":"2024-02-13T03:22:34.987582+0000","last_peered":"2024-02-13T03:22:34.987582+0000","last_clean":"2024-02-13T03:22:34.987582+0000","last_became_active":"2024-02-13T03:21:13.698185+0000","last_became_peered":"2024-02-13T03:21:13.698185+0000","last_unstale":"2024-02-13T03:22:34.987582+0000","last_undegraded":"2024-02-13T03:22:34.987582+0000","last_fullsized":"2024-02-13T03:22:34.987582+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996883+0000","last_change":"2024-02-13T03:21:09.679180+0000","last_active":"2024-02-13T03:22:34.996883+0000","last_peered":"2024-02-13T03:22:34.996883+0000","last_clean":"2024-02-13T03:22:34.996883+0000","last_became_active":"2024-02-13T03:21:09.678859+0000","last_became_peered":"2024-02-13T03:21:09.678859+0000","last_unstale":"2024-02-13T03:22:34.996883+0000","last_undegraded":"2024-02-13T03:22:34.996883+0000","last_fullsized":"2024-02-13T03:22:34.996883+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"60'17","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992408+0000","last_change":"2024-02-13T03:21:12.018803+0000","last_active":"2024-02-13T03:22:34.992408+0000","last_peered":"2024-02-13T03:22:34.992408+0000","last_clean":"2024-02-13T03:22:34.992408+0000","last_became_active":"2024-02-13T03:21:12.017648+0000","last_became_peered":"2024-02-13T03:21:12.017648+0000","last_unstale":"2024-02-13T03:22:34.992408+0000","last_undegraded":"2024-02-13T03:22:34.992408+0000","last_fullsized":"2024-02-13T03:22:34.992408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"60'1","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.983488+0000","last_change":"2024-02-13T03:21:15.710918+0000","last_active":"2024-02-13T03:22:34.983488+0000","last_peered":"2024-02-13T03:22:34.983488+0000","last_clean":"2024-02-13T03:22:34.983488+0000","last_became_active":"2024-02-13T03:21:15.710532+0000","last_became_peered":"2024-02-13T03:21:15.710532+0000","last_unstale":"2024-02-13T03:22:34.983488+0000","last_undegraded":"2024-02-13T03:22:34.983488+0000","last_fullsized":"2024-02-13T03:22:34.983488+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992650+0000","last_change":"2024-02-13T03:21:13.694780+0000","last_active":"2024-02-13T03:22:34.992650+0000","last_peered":"2024-02-13T03:22:34.992650+0000","last_clean":"2024-02-13T03:22:34.992650+0000","last_became_active":"2024-02-13T03:21:13.694381+0000","last_became_peered":"2024-02-13T03:21:13.694381+0000","last_unstale":"2024-02-13T03:22:34.992650+0000","last_undegraded":"2024-02-13T03:22:34.992650+0000","last_fullsized":"2024-02-13T03:22:34.992650+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988175+0000","last_change":"2024-02-13T03:21:09.673670+0000","last_active":"2024-02-13T03:22:34.988175+0000","last_peered":"2024-02-13T03:22:34.988175+0000","last_clean":"2024-02-13T03:22:34.988175+0000","last_became_active":"2024-02-13T03:21:09.672121+0000","last_became_peered":"2024-02-13T03:21:09.672121+0000","last_unstale":"2024-02-13T03:22:34.988175+0000","last_undegraded":"2024-02-13T03:22:34.988175+0000","last_fullsized":"2024-02-13T03:22:34.988175+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"60'9","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:22:36.006223+0000","last_change":"2024-02-13T03:22:36.006223+0000","last_active":"2024-02-13T03:22:36.006223+0000","last_peered":"2024-02-13T03:22:36.006223+0000","last_clean":"2024-02-13T03:22:36.006223+0000","last_became_active":"2024-02-13T03:22:35.999802+0000","last_became_peered":"2024-02-13T03:22:35.999802+0000","last_unstale":"2024-02-13T03:22:36.006223+0000","last_undegraded":"2024-02-13T03:22:36.006223+0000","last_fullsized":"2024-02-13T03:22:36.006223+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991728+0000","last_change":"2024-02-13T03:21:15.712517+0000","last_active":"2024-02-13T03:22:34.991728+0000","last_peered":"2024-02-13T03:22:34.991728+0000","last_clean":"2024-02-13T03:22:34.991728+0000","last_became_active":"2024-02-13T03:21:15.712193+0000","last_became_peered":"2024-02-13T03:21:15.712193+0000","last_unstale":"2024-02-13T03:22:34.991728+0000","last_undegraded":"2024-02-13T03:22:34.991728+0000","last_fullsized":"2024-02-13T03:22:34.991728+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988444+0000","last_change":"2024-02-13T03:21:13.686338+0000","last_active":"2024-02-13T03:22:34.988444+0000","last_peered":"2024-02-13T03:22:34.988444+0000","last_clean":"2024-02-13T03:22:34.988444+0000","last_became_active":"2024-02-13T03:21:13.686016+0000","last_became_peered":"2024-02-13T03:21:13.686016+0000","last_unstale":"2024-02-13T03:22:34.988444+0000","last_undegraded":"2024-02-13T03:22:34.988444+0000","last_fullsized":"2024-02-13T03:22:34.988444+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"53'1","reported_seq":25,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991704+0000","last_change":"2024-02-13T03:21:09.677024+0000","last_active":"2024-02-13T03:22:34.991704+0000","last_peered":"2024-02-13T03:22:34.991704+0000","last_clean":"2024-02-13T03:22:34.991704+0000","last_became_active":"2024-02-13T03:21:09.676705+0000","last_became_peered":"2024-02-13T03:21:09.676705+0000","last_unstale":"2024-02-13T03:22:34.991704+0000","last_undegraded":"2024-02-13T03:22:34.991704+0000","last_fullsized":"2024-02-13T03:22:34.991704+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"60'11","reported_seq":36,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991429+0000","last_change":"2024-02-13T03:21:11.680974+0000","last_active":"2024-02-13T03:22:34.991429+0000","last_peered":"2024-02-13T03:22:34.991429+0000","last_clean":"2024-02-13T03:22:34.991429+0000","last_became_active":"2024-02-13T03:21:11.680688+0000","last_became_peered":"2024-02-13T03:21:11.680688+0000","last_unstale":"2024-02-13T03:22:34.991429+0000","last_undegraded":"2024-02-13T03:22:34.991429+0000","last_fullsized":"2024-02-13T03:22:34.991429+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985011+0000","last_change":"2024-02-13T03:21:15.712764+0000","last_active":"2024-02-13T03:22:34.985011+0000","last_peered":"2024-02-13T03:22:34.985011+0000","last_clean":"2024-02-13T03:22:34.985011+0000","last_became_active":"2024-02-13T03:21:15.712423+0000","last_became_peered":"2024-02-13T03:21:15.712423+0000","last_unstale":"2024-02-13T03:22:34.985011+0000","last_undegraded":"2024-02-13T03:22:34.985011+0000","last_fullsized":"2024-02-13T03:22:34.985011+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"60'8","reported_seq":81,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:42.898321+0000","last_change":"2024-02-13T03:21:13.700498+0000","last_active":"2024-02-13T03:25:42.898321+0000","last_peered":"2024-02-13T03:25:42.898321+0000","last_clean":"2024-02-13T03:25:42.898321+0000","last_became_active":"2024-02-13T03:21:13.700072+0000","last_became_peered":"2024-02-13T03:21:13.700072+0000","last_unstale":"2024-02-13T03:25:42.898321+0000","last_undegraded":"2024-02-13T03:25:42.898321+0000","last_fullsized":"2024-02-13T03:25:42.898321+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987655+0000","last_change":"2024-02-13T03:21:09.893368+0000","last_active":"2024-02-13T03:22:34.987655+0000","last_peered":"2024-02-13T03:22:34.987655+0000","last_clean":"2024-02-13T03:22:34.987655+0000","last_became_active":"2024-02-13T03:21:09.892998+0000","last_became_peered":"2024-02-13T03:21:09.892998+0000","last_unstale":"2024-02-13T03:22:34.987655+0000","last_undegraded":"2024-02-13T03:22:34.987655+0000","last_fullsized":"2024-02-13T03:22:34.987655+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"60'22","reported_seq":62,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992520+0000","last_change":"2024-02-13T03:21:12.018509+0000","last_active":"2024-02-13T03:22:34.992520+0000","last_peered":"2024-02-13T03:22:34.992520+0000","last_clean":"2024-02-13T03:22:34.992520+0000","last_became_active":"2024-02-13T03:21:12.017459+0000","last_became_peered":"2024-02-13T03:21:12.017459+0000","last_unstale":"2024-02-13T03:22:34.992520+0000","last_undegraded":"2024-02-13T03:22:34.992520+0000","last_fullsized":"2024-02-13T03:22:34.992520+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992009+0000","last_change":"2024-02-13T03:21:15.719839+0000","last_active":"2024-02-13T03:22:34.992009+0000","last_peered":"2024-02-13T03:22:34.992009+0000","last_clean":"2024-02-13T03:22:34.992009+0000","last_became_active":"2024-02-13T03:21:15.719549+0000","last_became_peered":"2024-02-13T03:21:15.719549+0000","last_unstale":"2024-02-13T03:22:34.992009+0000","last_undegraded":"2024-02-13T03:22:34.992009+0000","last_fullsized":"2024-02-13T03:22:34.992009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"60'8","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:42.895282+0000","last_change":"2024-02-13T03:21:13.690254+0000","last_active":"2024-02-13T03:25:42.895282+0000","last_peered":"2024-02-13T03:25:42.895282+0000","last_clean":"2024-02-13T03:25:42.895282+0000","last_became_active":"2024-02-13T03:21:13.689829+0000","last_became_peered":"2024-02-13T03:21:13.689829+0000","last_unstale":"2024-02-13T03:25:42.895282+0000","last_undegraded":"2024-02-13T03:25:42.895282+0000","last_fullsized":"2024-02-13T03:25:42.895282+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.984299+0000","last_change":"2024-02-13T03:21:09.667086+0000","last_active":"2024-02-13T03:22:34.984299+0000","last_peered":"2024-02-13T03:22:34.984299+0000","last_clean":"2024-02-13T03:22:34.984299+0000","last_became_active":"2024-02-13T03:21:09.666585+0000","last_became_peered":"2024-02-13T03:21:09.666585+0000","last_unstale":"2024-02-13T03:22:34.984299+0000","last_undegraded":"2024-02-13T03:22:34.984299+0000","last_fullsized":"2024-02-13T03:22:34.984299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"60'15","reported_seq":48,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996963+0000","last_change":"2024-02-13T03:21:12.025158+0000","last_active":"2024-02-13T03:22:34.996963+0000","last_peered":"2024-02-13T03:22:34.996963+0000","last_clean":"2024-02-13T03:22:34.996963+0000","last_became_active":"2024-02-13T03:21:12.024786+0000","last_became_peered":"2024-02-13T03:21:12.024786+0000","last_unstale":"2024-02-13T03:22:34.996963+0000","last_undegraded":"2024-02-13T03:22:34.996963+0000","last_fullsized":"2024-02-13T03:22:34.996963+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990062+0000","last_change":"2024-02-13T03:21:15.719431+0000","last_active":"2024-02-13T03:22:34.990062+0000","last_peered":"2024-02-13T03:22:34.990062+0000","last_clean":"2024-02-13T03:22:34.990062+0000","last_became_active":"2024-02-13T03:21:15.719060+0000","last_became_peered":"2024-02-13T03:21:15.719060+0000","last_unstale":"2024-02-13T03:22:34.990062+0000","last_undegraded":"2024-02-13T03:22:34.990062+0000","last_fullsized":"2024-02-13T03:22:34.990062+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992205+0000","last_change":"2024-02-13T03:21:13.692813+0000","last_active":"2024-02-13T03:22:34.992205+0000","last_peered":"2024-02-13T03:22:34.992205+0000","last_clean":"2024-02-13T03:22:34.992205+0000","last_became_active":"2024-02-13T03:21:13.692461+0000","last_became_peered":"2024-02-13T03:21:13.692461+0000","last_unstale":"2024-02-13T03:22:34.992205+0000","last_undegraded":"2024-02-13T03:22:34.992205+0000","last_fullsized":"2024-02-13T03:22:34.992205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987681+0000","last_change":"2024-02-13T03:21:09.673501+0000","last_active":"2024-02-13T03:22:34.987681+0000","last_peered":"2024-02-13T03:22:34.987681+0000","last_clean":"2024-02-13T03:22:34.987681+0000","last_became_active":"2024-02-13T03:21:09.673173+0000","last_became_peered":"2024-02-13T03:21:09.673173+0000","last_unstale":"2024-02-13T03:22:34.987681+0000","last_undegraded":"2024-02-13T03:22:34.987681+0000","last_fullsized":"2024-02-13T03:22:34.987681+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"60'21","reported_seq":63,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985684+0000","last_change":"2024-02-13T03:21:12.022273+0000","last_active":"2024-02-13T03:22:34.985684+0000","last_peered":"2024-02-13T03:22:34.985684+0000","last_clean":"2024-02-13T03:22:34.985684+0000","last_became_active":"2024-02-13T03:21:12.021613+0000","last_became_peered":"2024-02-13T03:21:12.021613+0000","last_unstale":"2024-02-13T03:22:34.985684+0000","last_undegraded":"2024-02-13T03:22:34.985684+0000","last_fullsized":"2024-02-13T03:22:34.985684+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988276+0000","last_change":"2024-02-13T03:21:15.717901+0000","last_active":"2024-02-13T03:22:34.988276+0000","last_peered":"2024-02-13T03:22:34.988276+0000","last_clean":"2024-02-13T03:22:34.988276+0000","last_became_active":"2024-02-13T03:21:15.717303+0000","last_became_peered":"2024-02-13T03:21:15.717303+0000","last_unstale":"2024-02-13T03:22:34.988276+0000","last_undegraded":"2024-02-13T03:22:34.988276+0000","last_fullsized":"2024-02-13T03:22:34.988276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996925+0000","last_change":"2024-02-13T03:21:13.694546+0000","last_active":"2024-02-13T03:22:34.996925+0000","last_peered":"2024-02-13T03:22:34.996925+0000","last_clean":"2024-02-13T03:22:34.996925+0000","last_became_active":"2024-02-13T03:21:13.694163+0000","last_became_peered":"2024-02-13T03:21:13.694163+0000","last_unstale":"2024-02-13T03:22:34.996925+0000","last_undegraded":"2024-02-13T03:22:34.996925+0000","last_fullsized":"2024-02-13T03:22:34.996925+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.995629+0000","last_change":"2024-02-13T03:21:09.677907+0000","last_active":"2024-02-13T03:22:34.995629+0000","last_peered":"2024-02-13T03:22:34.995629+0000","last_clean":"2024-02-13T03:22:34.995629+0000","last_became_active":"2024-02-13T03:21:09.677552+0000","last_became_peered":"2024-02-13T03:21:09.677552+0000","last_unstale":"2024-02-13T03:22:34.995629+0000","last_undegraded":"2024-02-13T03:22:34.995629+0000","last_fullsized":"2024-02-13T03:22:34.995629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"60'12","reported_seq":41,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992094+0000","last_change":"2024-02-13T03:21:12.019452+0000","last_active":"2024-02-13T03:22:34.992094+0000","last_peered":"2024-02-13T03:22:34.992094+0000","last_clean":"2024-02-13T03:22:34.992094+0000","last_became_active":"2024-02-13T03:21:12.019132+0000","last_became_peered":"2024-02-13T03:21:12.019132+0000","last_unstale":"2024-02-13T03:22:34.992094+0000","last_undegraded":"2024-02-13T03:22:34.992094+0000","last_fullsized":"2024-02-13T03:22:34.992094+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.995381+0000","last_change":"2024-02-13T03:21:15.721807+0000","last_active":"2024-02-13T03:22:34.995381+0000","last_peered":"2024-02-13T03:22:34.995381+0000","last_clean":"2024-02-13T03:22:34.995381+0000","last_became_active":"2024-02-13T03:21:15.721490+0000","last_became_peered":"2024-02-13T03:21:15.721490+0000","last_unstale":"2024-02-13T03:22:34.995381+0000","last_undegraded":"2024-02-13T03:22:34.995381+0000","last_fullsized":"2024-02-13T03:22:34.995381+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"53'2","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988761+0000","last_change":"2024-02-13T03:21:09.675843+0000","last_active":"2024-02-13T03:22:34.988761+0000","last_peered":"2024-02-13T03:22:34.988761+0000","last_clean":"2024-02-13T03:22:34.988761+0000","last_became_active":"2024-02-13T03:21:09.675404+0000","last_became_peered":"2024-02-13T03:21:09.675404+0000","last_unstale":"2024-02-13T03:22:34.988761+0000","last_undegraded":"2024-02-13T03:22:34.988761+0000","last_fullsized":"2024-02-13T03:22:34.988761+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"60'27","reported_seq":72,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.994717+0000","last_change":"2024-02-13T03:21:12.025133+0000","last_active":"2024-02-13T03:22:34.994717+0000","last_peered":"2024-02-13T03:22:34.994717+0000","last_clean":"2024-02-13T03:22:34.994717+0000","last_became_active":"2024-02-13T03:21:12.024766+0000","last_became_peered":"2024-02-13T03:21:12.024766+0000","last_unstale":"2024-02-13T03:22:34.994717+0000","last_undegraded":"2024-02-13T03:22:34.994717+0000","last_fullsized":"2024-02-13T03:22:34.994717+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"60'8","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:42.898188+0000","last_change":"2024-02-13T03:21:13.701953+0000","last_active":"2024-02-13T03:25:42.898188+0000","last_peered":"2024-02-13T03:25:42.898188+0000","last_clean":"2024-02-13T03:25:42.898188+0000","last_became_active":"2024-02-13T03:21:13.701595+0000","last_became_peered":"2024-02-13T03:21:13.701595+0000","last_unstale":"2024-02-13T03:25:42.898188+0000","last_undegraded":"2024-02-13T03:25:42.898188+0000","last_fullsized":"2024-02-13T03:25:42.898188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991754+0000","last_change":"2024-02-13T03:21:15.708758+0000","last_active":"2024-02-13T03:22:34.991754+0000","last_peered":"2024-02-13T03:22:34.991754+0000","last_clean":"2024-02-13T03:22:34.991754+0000","last_became_active":"2024-02-13T03:21:15.708444+0000","last_became_peered":"2024-02-13T03:21:15.708444+0000","last_unstale":"2024-02-13T03:22:34.991754+0000","last_undegraded":"2024-02-13T03:22:34.991754+0000","last_fullsized":"2024-02-13T03:22:34.991754+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988555+0000","last_change":"2024-02-13T03:21:09.675874+0000","last_active":"2024-02-13T03:22:34.988555+0000","last_peered":"2024-02-13T03:22:34.988555+0000","last_clean":"2024-02-13T03:22:34.988555+0000","last_became_active":"2024-02-13T03:21:09.674990+0000","last_became_peered":"2024-02-13T03:21:09.674990+0000","last_unstale":"2024-02-13T03:22:34.988555+0000","last_undegraded":"2024-02-13T03:22:34.988555+0000","last_fullsized":"2024-02-13T03:22:34.988555+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"60'24","reported_seq":70,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990905+0000","last_change":"2024-02-13T03:21:11.680985+0000","last_active":"2024-02-13T03:22:34.990905+0000","last_peered":"2024-02-13T03:22:34.990905+0000","last_clean":"2024-02-13T03:22:34.990905+0000","last_became_active":"2024-02-13T03:21:11.680602+0000","last_became_peered":"2024-02-13T03:21:11.680602+0000","last_unstale":"2024-02-13T03:22:34.990905+0000","last_undegraded":"2024-02-13T03:22:34.990905+0000","last_fullsized":"2024-02-13T03:22:34.990905+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988012+0000","last_change":"2024-02-13T03:21:13.683804+0000","last_active":"2024-02-13T03:22:34.988012+0000","last_peered":"2024-02-13T03:22:34.988012+0000","last_clean":"2024-02-13T03:22:34.988012+0000","last_became_active":"2024-02-13T03:21:13.683415+0000","last_became_peered":"2024-02-13T03:21:13.683415+0000","last_unstale":"2024-02-13T03:22:34.988012+0000","last_undegraded":"2024-02-13T03:22:34.988012+0000","last_fullsized":"2024-02-13T03:22:34.988012+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987259+0000","last_change":"2024-02-13T03:21:15.716870+0000","last_active":"2024-02-13T03:22:34.987259+0000","last_peered":"2024-02-13T03:22:34.987259+0000","last_clean":"2024-02-13T03:22:34.987259+0000","last_became_active":"2024-02-13T03:21:15.714813+0000","last_became_peered":"2024-02-13T03:21:15.714813+0000","last_unstale":"2024-02-13T03:22:34.987259+0000","last_undegraded":"2024-02-13T03:22:34.987259+0000","last_fullsized":"2024-02-13T03:22:34.987259+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.989897+0000","last_change":"2024-02-13T03:21:13.688574+0000","last_active":"2024-02-13T03:22:34.989897+0000","last_peered":"2024-02-13T03:22:34.989897+0000","last_clean":"2024-02-13T03:22:34.989897+0000","last_became_active":"2024-02-13T03:21:13.688224+0000","last_became_peered":"2024-02-13T03:21:13.688224+0000","last_unstale":"2024-02-13T03:22:34.989897+0000","last_undegraded":"2024-02-13T03:22:34.989897+0000","last_fullsized":"2024-02-13T03:22:34.989897+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988460+0000","last_change":"2024-02-13T03:21:09.679650+0000","last_active":"2024-02-13T03:22:34.988460+0000","last_peered":"2024-02-13T03:22:34.988460+0000","last_clean":"2024-02-13T03:22:34.988460+0000","last_became_active":"2024-02-13T03:21:09.679184+0000","last_became_peered":"2024-02-13T03:21:09.679184+0000","last_unstale":"2024-02-13T03:22:34.988460+0000","last_undegraded":"2024-02-13T03:22:34.988460+0000","last_fullsized":"2024-02-13T03:22:34.988460+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"60'25","reported_seq":63,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996196+0000","last_change":"2024-02-13T03:21:12.021923+0000","last_active":"2024-02-13T03:22:34.996196+0000","last_peered":"2024-02-13T03:22:34.996196+0000","last_clean":"2024-02-13T03:22:34.996196+0000","last_became_active":"2024-02-13T03:21:12.021003+0000","last_became_peered":"2024-02-13T03:21:12.021003+0000","last_unstale":"2024-02-13T03:22:34.996196+0000","last_undegraded":"2024-02-13T03:22:34.996196+0000","last_fullsized":"2024-02-13T03:22:34.996196+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.983033+0000","last_change":"2024-02-13T03:21:15.714837+0000","last_active":"2024-02-13T03:22:34.983033+0000","last_peered":"2024-02-13T03:22:34.983033+0000","last_clean":"2024-02-13T03:22:34.983033+0000","last_became_active":"2024-02-13T03:21:15.714514+0000","last_became_peered":"2024-02-13T03:21:15.714514+0000","last_unstale":"2024-02-13T03:22:34.983033+0000","last_undegraded":"2024-02-13T03:22:34.983033+0000","last_fullsized":"2024-02-13T03:22:34.983033+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988681+0000","last_change":"2024-02-13T03:21:13.685173+0000","last_active":"2024-02-13T03:22:34.988681+0000","last_peered":"2024-02-13T03:22:34.988681+0000","last_clean":"2024-02-13T03:22:34.988681+0000","last_became_active":"2024-02-13T03:21:13.684888+0000","last_became_peered":"2024-02-13T03:21:13.684888+0000","last_unstale":"2024-02-13T03:22:34.988681+0000","last_undegraded":"2024-02-13T03:22:34.988681+0000","last_fullsized":"2024-02-13T03:22:34.988681+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996481+0000","last_change":"2024-02-13T03:21:09.678441+0000","last_active":"2024-02-13T03:22:34.996481+0000","last_peered":"2024-02-13T03:22:34.996481+0000","last_clean":"2024-02-13T03:22:34.996481+0000","last_became_active":"2024-02-13T03:21:09.677981+0000","last_became_peered":"2024-02-13T03:21:09.677981+0000","last_unstale":"2024-02-13T03:22:34.996481+0000","last_undegraded":"2024-02-13T03:22:34.996481+0000","last_fullsized":"2024-02-13T03:22:34.996481+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"60'15","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:22:36.433302+0000","last_change":"2024-02-13T03:22:36.433302+0000","last_active":"2024-02-13T03:22:36.433302+0000","last_peered":"2024-02-13T03:22:36.433302+0000","last_clean":"2024-02-13T03:22:36.433302+0000","last_became_active":"2024-02-13T03:22:35.997847+0000","last_became_peered":"2024-02-13T03:22:35.997847+0000","last_unstale":"2024-02-13T03:22:36.433302+0000","last_undegraded":"2024-02-13T03:22:36.433302+0000","last_fullsized":"2024-02-13T03:22:36.433302+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991457+0000","last_change":"2024-02-13T03:21:15.716004+0000","last_active":"2024-02-13T03:22:34.991457+0000","last_peered":"2024-02-13T03:22:34.991457+0000","last_clean":"2024-02-13T03:22:34.991457+0000","last_became_active":"2024-02-13T03:21:15.715549+0000","last_became_peered":"2024-02-13T03:21:15.715549+0000","last_unstale":"2024-02-13T03:22:34.991457+0000","last_undegraded":"2024-02-13T03:22:34.991457+0000","last_fullsized":"2024-02-13T03:22:34.991457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.995154+0000","last_change":"2024-02-13T03:21:08.660483+0000","last_active":"2024-02-13T03:22:34.995154+0000","last_peered":"2024-02-13T03:22:34.995154+0000","last_clean":"2024-02-13T03:22:34.995154+0000","last_became_active":"2024-02-13T03:21:08.660084+0000","last_became_peered":"2024-02-13T03:21:08.660084+0000","last_unstale":"2024-02-13T03:22:34.995154+0000","last_undegraded":"2024-02-13T03:22:34.995154+0000","last_fullsized":"2024-02-13T03:22:34.995154+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:07.620086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:07.620086+0000","last_clean_scrub_stamp":"2024-02-13T03:21:07.620086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991160+0000","last_change":"2024-02-13T03:21:09.673377+0000","last_active":"2024-02-13T03:22:34.991160+0000","last_peered":"2024-02-13T03:22:34.991160+0000","last_clean":"2024-02-13T03:22:34.991160+0000","last_became_active":"2024-02-13T03:21:09.672635+0000","last_became_peered":"2024-02-13T03:21:09.672635+0000","last_unstale":"2024-02-13T03:22:34.991160+0000","last_undegraded":"2024-02-13T03:22:34.991160+0000","last_fullsized":"2024-02-13T03:22:34.991160+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"60'34","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:42.894205+0000","last_change":"2024-02-13T03:21:11.680288+0000","last_active":"2024-02-13T03:25:42.894205+0000","last_peered":"2024-02-13T03:25:42.894205+0000","last_clean":"2024-02-13T03:25:42.894205+0000","last_became_active":"2024-02-13T03:21:11.679879+0000","last_became_peered":"2024-02-13T03:21:11.679879+0000","last_unstale":"2024-02-13T03:25:42.894205+0000","last_undegraded":"2024-02-13T03:25:42.894205+0000","last_fullsized":"2024-02-13T03:25:42.894205+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985631+0000","last_change":"2024-02-13T03:21:13.690578+0000","last_active":"2024-02-13T03:22:34.985631+0000","last_peered":"2024-02-13T03:22:34.985631+0000","last_clean":"2024-02-13T03:22:34.985631+0000","last_became_active":"2024-02-13T03:21:13.690236+0000","last_became_peered":"2024-02-13T03:21:13.690236+0000","last_unstale":"2024-02-13T03:22:34.985631+0000","last_undegraded":"2024-02-13T03:22:34.985631+0000","last_fullsized":"2024-02-13T03:22:34.985631+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988543+0000","last_change":"2024-02-13T03:21:15.718127+0000","last_active":"2024-02-13T03:22:34.988543+0000","last_peered":"2024-02-13T03:22:34.988543+0000","last_clean":"2024-02-13T03:22:34.988543+0000","last_became_active":"2024-02-13T03:21:15.717847+0000","last_became_peered":"2024-02-13T03:21:15.717847+0000","last_unstale":"2024-02-13T03:22:34.988543+0000","last_undegraded":"2024-02-13T03:22:34.988543+0000","last_fullsized":"2024-02-13T03:22:34.988543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988180+0000","last_change":"2024-02-13T03:21:09.889381+0000","last_active":"2024-02-13T03:22:34.988180+0000","last_peered":"2024-02-13T03:22:34.988180+0000","last_clean":"2024-02-13T03:22:34.988180+0000","last_became_active":"2024-02-13T03:21:09.889030+0000","last_became_peered":"2024-02-13T03:21:09.889030+0000","last_unstale":"2024-02-13T03:22:34.988180+0000","last_undegraded":"2024-02-13T03:22:34.988180+0000","last_fullsized":"2024-02-13T03:22:34.988180+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988303+0000","last_change":"2024-02-13T03:21:08.660768+0000","last_active":"2024-02-13T03:22:34.988303+0000","last_peered":"2024-02-13T03:22:34.988303+0000","last_clean":"2024-02-13T03:22:34.988303+0000","last_became_active":"2024-02-13T03:21:08.660413+0000","last_became_peered":"2024-02-13T03:21:08.660413+0000","last_unstale":"2024-02-13T03:22:34.988303+0000","last_undegraded":"2024-02-13T03:22:34.988303+0000","last_fullsized":"2024-02-13T03:22:34.988303+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:07.620086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:07.620086+0000","last_clean_scrub_stamp":"2024-02-13T03:21:07.620086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"60'19","reported_seq":59,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992443+0000","last_change":"2024-02-13T03:21:11.676843+0000","last_active":"2024-02-13T03:22:34.992443+0000","last_peered":"2024-02-13T03:22:34.992443+0000","last_clean":"2024-02-13T03:22:34.992443+0000","last_became_active":"2024-02-13T03:21:11.676528+0000","last_became_peered":"2024-02-13T03:21:11.676528+0000","last_unstale":"2024-02-13T03:22:34.992443+0000","last_undegraded":"2024-02-13T03:22:34.992443+0000","last_fullsized":"2024-02-13T03:22:34.992443+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.989830+0000","last_change":"2024-02-13T03:21:13.696632+0000","last_active":"2024-02-13T03:22:34.989830+0000","last_peered":"2024-02-13T03:22:34.989830+0000","last_clean":"2024-02-13T03:22:34.989830+0000","last_became_active":"2024-02-13T03:21:13.696353+0000","last_became_peered":"2024-02-13T03:21:13.696353+0000","last_unstale":"2024-02-13T03:22:34.989830+0000","last_undegraded":"2024-02-13T03:22:34.989830+0000","last_fullsized":"2024-02-13T03:22:34.989830+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.997143+0000","last_change":"2024-02-13T03:21:15.722832+0000","last_active":"2024-02-13T03:22:34.997143+0000","last_peered":"2024-02-13T03:22:34.997143+0000","last_clean":"2024-02-13T03:22:34.997143+0000","last_became_active":"2024-02-13T03:21:15.722574+0000","last_became_peered":"2024-02-13T03:21:15.722574+0000","last_unstale":"2024-02-13T03:22:34.997143+0000","last_undegraded":"2024-02-13T03:22:34.997143+0000","last_fullsized":"2024-02-13T03:22:34.997143+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":23,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987424+0000","last_change":"2024-02-13T03:20:57.120214+0000","last_active":"2024-02-13T03:22:34.987424+0000","last_peered":"2024-02-13T03:22:34.987424+0000","last_clean":"2024-02-13T03:22:34.987424+0000","last_became_active":"2024-02-13T03:20:57.119905+0000","last_became_peered":"2024-02-13T03:20:57.119905+0000","last_unstale":"2024-02-13T03:22:34.987424+0000","last_undegraded":"2024-02-13T03:22:34.987424+0000","last_fullsized":"2024-02-13T03:22:34.987424+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:17:31.987382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:17:31.987382+0000","last_clean_scrub_stamp":"2024-02-13T03:17:31.987382+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991452+0000","last_change":"2024-02-13T03:21:09.671092+0000","last_active":"2024-02-13T03:22:34.991452+0000","last_peered":"2024-02-13T03:22:34.991452+0000","last_clean":"2024-02-13T03:22:34.991452+0000","last_became_active":"2024-02-13T03:21:09.670767+0000","last_became_peered":"2024-02-13T03:21:09.670767+0000","last_unstale":"2024-02-13T03:22:34.991452+0000","last_undegraded":"2024-02-13T03:22:34.991452+0000","last_fullsized":"2024-02-13T03:22:34.991452+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"60'14","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996742+0000","last_change":"2024-02-13T03:21:12.022092+0000","last_active":"2024-02-13T03:22:34.996742+0000","last_peered":"2024-02-13T03:22:34.996742+0000","last_clean":"2024-02-13T03:22:34.996742+0000","last_became_active":"2024-02-13T03:21:12.021274+0000","last_became_peered":"2024-02-13T03:21:12.021274+0000","last_unstale":"2024-02-13T03:22:34.996742+0000","last_undegraded":"2024-02-13T03:22:34.996742+0000","last_fullsized":"2024-02-13T03:22:34.996742+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988246+0000","last_change":"2024-02-13T03:21:13.702783+0000","last_active":"2024-02-13T03:22:34.988246+0000","last_peered":"2024-02-13T03:22:34.988246+0000","last_clean":"2024-02-13T03:22:34.988246+0000","last_became_active":"2024-02-13T03:21:13.702442+0000","last_became_peered":"2024-02-13T03:21:13.702442+0000","last_unstale":"2024-02-13T03:22:34.988246+0000","last_undegraded":"2024-02-13T03:22:34.988246+0000","last_fullsized":"2024-02-13T03:22:34.988246+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990323+0000","last_change":"2024-02-13T03:21:15.715398+0000","last_active":"2024-02-13T03:22:34.990323+0000","last_peered":"2024-02-13T03:22:34.990323+0000","last_clean":"2024-02-13T03:22:34.990323+0000","last_became_active":"2024-02-13T03:21:15.715080+0000","last_became_peered":"2024-02-13T03:21:15.715080+0000","last_unstale":"2024-02-13T03:22:34.990323+0000","last_undegraded":"2024-02-13T03:22:34.990323+0000","last_fullsized":"2024-02-13T03:22:34.990323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.983368+0000","last_change":"2024-02-13T03:21:09.671105+0000","last_active":"2024-02-13T03:22:34.983368+0000","last_peered":"2024-02-13T03:22:34.983368+0000","last_clean":"2024-02-13T03:22:34.983368+0000","last_became_active":"2024-02-13T03:21:09.669540+0000","last_became_peered":"2024-02-13T03:21:09.669540+0000","last_unstale":"2024-02-13T03:22:34.983368+0000","last_undegraded":"2024-02-13T03:22:34.983368+0000","last_fullsized":"2024-02-13T03:22:34.983368+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"60'5","reported_seq":297,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:49.480840+0000","last_change":"2024-02-13T03:21:08.659798+0000","last_active":"2024-02-13T03:25:49.480840+0000","last_peered":"2024-02-13T03:25:49.480840+0000","last_clean":"2024-02-13T03:25:49.480840+0000","last_became_active":"2024-02-13T03:21:08.659452+0000","last_became_peered":"2024-02-13T03:21:08.659452+0000","last_unstale":"2024-02-13T03:25:49.480840+0000","last_undegraded":"2024-02-13T03:25:49.480840+0000","last_fullsized":"2024-02-13T03:25:49.480840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:07.620086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:07.620086+0000","last_clean_scrub_stamp":"2024-02-13T03:21:07.620086+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":274,"num_read_kb":269,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"60'12","reported_seq":35,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985117+0000","last_change":"2024-02-13T03:21:11.677377+0000","last_active":"2024-02-13T03:22:34.985117+0000","last_peered":"2024-02-13T03:22:34.985117+0000","last_clean":"2024-02-13T03:22:34.985117+0000","last_became_active":"2024-02-13T03:21:11.677036+0000","last_became_peered":"2024-02-13T03:21:11.677036+0000","last_unstale":"2024-02-13T03:22:34.985117+0000","last_undegraded":"2024-02-13T03:22:34.985117+0000","last_fullsized":"2024-02-13T03:22:34.985117+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.997331+0000","last_change":"2024-02-13T03:21:13.699240+0000","last_active":"2024-02-13T03:22:34.997331+0000","last_peered":"2024-02-13T03:22:34.997331+0000","last_clean":"2024-02-13T03:22:34.997331+0000","last_became_active":"2024-02-13T03:21:13.698912+0000","last_became_peered":"2024-02-13T03:21:13.698912+0000","last_unstale":"2024-02-13T03:22:34.997331+0000","last_undegraded":"2024-02-13T03:22:34.997331+0000","last_fullsized":"2024-02-13T03:22:34.997331+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.987420+0000","last_change":"2024-02-13T03:21:15.709016+0000","last_active":"2024-02-13T03:22:34.987420+0000","last_peered":"2024-02-13T03:22:34.987420+0000","last_clean":"2024-02-13T03:22:34.987420+0000","last_became_active":"2024-02-13T03:21:15.708609+0000","last_became_peered":"2024-02-13T03:21:15.708609+0000","last_unstale":"2024-02-13T03:22:34.987420+0000","last_undegraded":"2024-02-13T03:22:34.987420+0000","last_fullsized":"2024-02-13T03:22:34.987420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991353+0000","last_change":"2024-02-13T03:21:09.890251+0000","last_active":"2024-02-13T03:22:34.991353+0000","last_peered":"2024-02-13T03:22:34.991353+0000","last_clean":"2024-02-13T03:22:34.991353+0000","last_became_active":"2024-02-13T03:21:09.889879+0000","last_became_peered":"2024-02-13T03:21:09.889879+0000","last_unstale":"2024-02-13T03:22:34.991353+0000","last_undegraded":"2024-02-13T03:22:34.991353+0000","last_fullsized":"2024-02-13T03:22:34.991353+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"60'25","reported_seq":68,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985306+0000","last_change":"2024-02-13T03:21:12.018603+0000","last_active":"2024-02-13T03:22:34.985306+0000","last_peered":"2024-02-13T03:22:34.985306+0000","last_clean":"2024-02-13T03:22:34.985306+0000","last_became_active":"2024-02-13T03:21:12.017722+0000","last_became_peered":"2024-02-13T03:21:12.017722+0000","last_unstale":"2024-02-13T03:22:34.985306+0000","last_undegraded":"2024-02-13T03:22:34.985306+0000","last_fullsized":"2024-02-13T03:22:34.985306+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996430+0000","last_change":"2024-02-13T03:21:13.695336+0000","last_active":"2024-02-13T03:22:34.996430+0000","last_peered":"2024-02-13T03:22:34.996430+0000","last_clean":"2024-02-13T03:22:34.996430+0000","last_became_active":"2024-02-13T03:21:13.694984+0000","last_became_peered":"2024-02-13T03:21:13.694984+0000","last_unstale":"2024-02-13T03:22:34.996430+0000","last_undegraded":"2024-02-13T03:22:34.996430+0000","last_fullsized":"2024-02-13T03:22:34.996430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.982980+0000","last_change":"2024-02-13T03:21:15.711017+0000","last_active":"2024-02-13T03:22:34.982980+0000","last_peered":"2024-02-13T03:22:34.982980+0000","last_clean":"2024-02-13T03:22:34.982980+0000","last_became_active":"2024-02-13T03:21:15.710708+0000","last_became_peered":"2024-02-13T03:21:15.710708+0000","last_unstale":"2024-02-13T03:22:34.982980+0000","last_undegraded":"2024-02-13T03:22:34.982980+0000","last_fullsized":"2024-02-13T03:22:34.982980+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996771+0000","last_change":"2024-02-13T03:21:09.892530+0000","last_active":"2024-02-13T03:22:34.996771+0000","last_peered":"2024-02-13T03:22:34.996771+0000","last_clean":"2024-02-13T03:22:34.996771+0000","last_became_active":"2024-02-13T03:21:09.891560+0000","last_became_peered":"2024-02-13T03:21:09.891560+0000","last_unstale":"2024-02-13T03:22:34.996771+0000","last_undegraded":"2024-02-13T03:22:34.996771+0000","last_fullsized":"2024-02-13T03:22:34.996771+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"60'16","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988563+0000","last_change":"2024-02-13T03:21:11.675808+0000","last_active":"2024-02-13T03:22:34.988563+0000","last_peered":"2024-02-13T03:22:34.988563+0000","last_clean":"2024-02-13T03:22:34.988563+0000","last_became_active":"2024-02-13T03:21:11.675329+0000","last_became_peered":"2024-02-13T03:21:11.675329+0000","last_unstale":"2024-02-13T03:22:34.988563+0000","last_undegraded":"2024-02-13T03:22:34.988563+0000","last_fullsized":"2024-02-13T03:22:34.988563+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"60'8","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:42.894554+0000","last_change":"2024-02-13T03:21:13.693739+0000","last_active":"2024-02-13T03:25:42.894554+0000","last_peered":"2024-02-13T03:25:42.894554+0000","last_clean":"2024-02-13T03:25:42.894554+0000","last_became_active":"2024-02-13T03:21:13.692363+0000","last_became_peered":"2024-02-13T03:21:13.692363+0000","last_unstale":"2024-02-13T03:25:42.894554+0000","last_undegraded":"2024-02-13T03:25:42.894554+0000","last_fullsized":"2024-02-13T03:25:42.894554+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.988055+0000","last_change":"2024-02-13T03:21:15.719024+0000","last_active":"2024-02-13T03:22:34.988055+0000","last_peered":"2024-02-13T03:22:34.988055+0000","last_clean":"2024-02-13T03:22:34.988055+0000","last_became_active":"2024-02-13T03:21:15.718682+0000","last_became_peered":"2024-02-13T03:21:15.718682+0000","last_unstale":"2024-02-13T03:22:34.988055+0000","last_undegraded":"2024-02-13T03:22:34.988055+0000","last_fullsized":"2024-02-13T03:22:34.988055+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.985373+0000","last_change":"2024-02-13T03:21:09.669986+0000","last_active":"2024-02-13T03:22:34.985373+0000","last_peered":"2024-02-13T03:22:34.985373+0000","last_clean":"2024-02-13T03:22:34.985373+0000","last_became_active":"2024-02-13T03:21:09.669635+0000","last_became_peered":"2024-02-13T03:21:09.669635+0000","last_unstale":"2024-02-13T03:22:34.985373+0000","last_undegraded":"2024-02-13T03:22:34.985373+0000","last_fullsized":"2024-02-13T03:22:34.985373+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"60'21","reported_seq":54,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990912+0000","last_change":"2024-02-13T03:21:11.678542+0000","last_active":"2024-02-13T03:22:34.990912+0000","last_peered":"2024-02-13T03:22:34.990912+0000","last_clean":"2024-02-13T03:22:34.990912+0000","last_became_active":"2024-02-13T03:21:11.678249+0000","last_became_peered":"2024-02-13T03:21:11.678249+0000","last_unstale":"2024-02-13T03:22:34.990912+0000","last_undegraded":"2024-02-13T03:22:34.990912+0000","last_fullsized":"2024-02-13T03:22:34.990912+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"60'8","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:42.894832+0000","last_change":"2024-02-13T03:21:13.686685+0000","last_active":"2024-02-13T03:25:42.894832+0000","last_peered":"2024-02-13T03:25:42.894832+0000","last_clean":"2024-02-13T03:25:42.894832+0000","last_became_active":"2024-02-13T03:21:13.686154+0000","last_became_peered":"2024-02-13T03:21:13.686154+0000","last_unstale":"2024-02-13T03:25:42.894832+0000","last_undegraded":"2024-02-13T03:25:42.894832+0000","last_fullsized":"2024-02-13T03:25:42.894832+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990935+0000","last_change":"2024-02-13T03:21:15.712051+0000","last_active":"2024-02-13T03:22:34.990935+0000","last_peered":"2024-02-13T03:22:34.990935+0000","last_clean":"2024-02-13T03:22:34.990935+0000","last_became_active":"2024-02-13T03:21:15.711786+0000","last_became_peered":"2024-02-13T03:21:15.711786+0000","last_unstale":"2024-02-13T03:22:34.990935+0000","last_undegraded":"2024-02-13T03:22:34.990935+0000","last_fullsized":"2024-02-13T03:22:34.990935+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991874+0000","last_change":"2024-02-13T03:21:09.668411+0000","last_active":"2024-02-13T03:22:34.991874+0000","last_peered":"2024-02-13T03:22:34.991874+0000","last_clean":"2024-02-13T03:22:34.991874+0000","last_became_active":"2024-02-13T03:21:09.668081+0000","last_became_peered":"2024-02-13T03:21:09.668081+0000","last_unstale":"2024-02-13T03:22:34.991874+0000","last_undegraded":"2024-02-13T03:22:34.991874+0000","last_fullsized":"2024-02-13T03:22:34.991874+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"60'30","reported_seq":79,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992317+0000","last_change":"2024-02-13T03:21:12.020181+0000","last_active":"2024-02-13T03:22:34.992317+0000","last_peered":"2024-02-13T03:22:34.992317+0000","last_clean":"2024-02-13T03:22:34.992317+0000","last_became_active":"2024-02-13T03:21:12.019872+0000","last_became_peered":"2024-02-13T03:21:12.019872+0000","last_unstale":"2024-02-13T03:22:34.992317+0000","last_undegraded":"2024-02-13T03:22:34.992317+0000","last_fullsized":"2024-02-13T03:22:34.992317+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990723+0000","last_change":"2024-02-13T03:21:13.694344+0000","last_active":"2024-02-13T03:22:34.990723+0000","last_peered":"2024-02-13T03:22:34.990723+0000","last_clean":"2024-02-13T03:22:34.990723+0000","last_became_active":"2024-02-13T03:21:13.693831+0000","last_became_peered":"2024-02-13T03:21:13.693831+0000","last_unstale":"2024-02-13T03:22:34.990723+0000","last_undegraded":"2024-02-13T03:22:34.990723+0000","last_fullsized":"2024-02-13T03:22:34.990723+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996504+0000","last_change":"2024-02-13T03:21:15.720324+0000","last_active":"2024-02-13T03:22:34.996504+0000","last_peered":"2024-02-13T03:22:34.996504+0000","last_clean":"2024-02-13T03:22:34.996504+0000","last_became_active":"2024-02-13T03:21:15.719862+0000","last_became_peered":"2024-02-13T03:21:15.719862+0000","last_unstale":"2024-02-13T03:22:34.996504+0000","last_undegraded":"2024-02-13T03:22:34.996504+0000","last_fullsized":"2024-02-13T03:22:34.996504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"60'8","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:42.898719+0000","last_change":"2024-02-13T03:21:13.700639+0000","last_active":"2024-02-13T03:25:42.898719+0000","last_peered":"2024-02-13T03:25:42.898719+0000","last_clean":"2024-02-13T03:25:42.898719+0000","last_became_active":"2024-02-13T03:21:13.700316+0000","last_became_peered":"2024-02-13T03:21:13.700316+0000","last_unstale":"2024-02-13T03:25:42.898719+0000","last_undegraded":"2024-02-13T03:25:42.898719+0000","last_fullsized":"2024-02-13T03:25:42.898719+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992220+0000","last_change":"2024-02-13T03:21:09.668351+0000","last_active":"2024-02-13T03:22:34.992220+0000","last_peered":"2024-02-13T03:22:34.992220+0000","last_clean":"2024-02-13T03:22:34.992220+0000","last_became_active":"2024-02-13T03:21:09.668010+0000","last_became_peered":"2024-02-13T03:21:09.668010+0000","last_unstale":"2024-02-13T03:22:34.992220+0000","last_undegraded":"2024-02-13T03:22:34.992220+0000","last_fullsized":"2024-02-13T03:22:34.992220+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"60'15","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992387+0000","last_change":"2024-02-13T03:21:11.680636+0000","last_active":"2024-02-13T03:22:34.992387+0000","last_peered":"2024-02-13T03:22:34.992387+0000","last_clean":"2024-02-13T03:22:34.992387+0000","last_became_active":"2024-02-13T03:21:11.680328+0000","last_became_peered":"2024-02-13T03:21:11.680328+0000","last_unstale":"2024-02-13T03:22:34.992387+0000","last_undegraded":"2024-02-13T03:22:34.992387+0000","last_fullsized":"2024-02-13T03:22:34.992387+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991564+0000","last_change":"2024-02-13T03:21:15.713297+0000","last_active":"2024-02-13T03:22:34.991564+0000","last_peered":"2024-02-13T03:22:34.991564+0000","last_clean":"2024-02-13T03:22:34.991564+0000","last_became_active":"2024-02-13T03:21:15.712900+0000","last_became_peered":"2024-02-13T03:21:15.712900+0000","last_unstale":"2024-02-13T03:22:34.991564+0000","last_undegraded":"2024-02-13T03:22:34.991564+0000","last_fullsized":"2024-02-13T03:22:34.991564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.997273+0000","last_change":"2024-02-13T03:21:13.693915+0000","last_active":"2024-02-13T03:22:34.997273+0000","last_peered":"2024-02-13T03:22:34.997273+0000","last_clean":"2024-02-13T03:22:34.997273+0000","last_became_active":"2024-02-13T03:21:13.693569+0000","last_became_peered":"2024-02-13T03:21:13.693569+0000","last_unstale":"2024-02-13T03:22:34.997273+0000","last_undegraded":"2024-02-13T03:22:34.997273+0000","last_fullsized":"2024-02-13T03:22:34.997273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.990666+0000","last_change":"2024-02-13T03:21:09.892621+0000","last_active":"2024-02-13T03:22:34.990666+0000","last_peered":"2024-02-13T03:22:34.990666+0000","last_clean":"2024-02-13T03:22:34.990666+0000","last_became_active":"2024-02-13T03:21:09.892095+0000","last_became_peered":"2024-02-13T03:21:09.892095+0000","last_unstale":"2024-02-13T03:22:34.990666+0000","last_undegraded":"2024-02-13T03:22:34.990666+0000","last_fullsized":"2024-02-13T03:22:34.990666+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"60'17","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991802+0000","last_change":"2024-02-13T03:21:11.680077+0000","last_active":"2024-02-13T03:22:34.991802+0000","last_peered":"2024-02-13T03:22:34.991802+0000","last_clean":"2024-02-13T03:22:34.991802+0000","last_became_active":"2024-02-13T03:21:11.679776+0000","last_became_peered":"2024-02-13T03:21:11.679776+0000","last_unstale":"2024-02-13T03:22:34.991802+0000","last_undegraded":"2024-02-13T03:22:34.991802+0000","last_fullsized":"2024-02-13T03:22:34.991802+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.989851+0000","last_change":"2024-02-13T03:21:15.712029+0000","last_active":"2024-02-13T03:22:34.989851+0000","last_peered":"2024-02-13T03:22:34.989851+0000","last_clean":"2024-02-13T03:22:34.989851+0000","last_became_active":"2024-02-13T03:21:15.711652+0000","last_became_peered":"2024-02-13T03:21:15.711652+0000","last_unstale":"2024-02-13T03:22:34.989851+0000","last_undegraded":"2024-02-13T03:22:34.989851+0000","last_fullsized":"2024-02-13T03:22:34.989851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991588+0000","last_change":"2024-02-13T03:21:13.686808+0000","last_active":"2024-02-13T03:22:34.991588+0000","last_peered":"2024-02-13T03:22:34.991588+0000","last_clean":"2024-02-13T03:22:34.991588+0000","last_became_active":"2024-02-13T03:21:13.685760+0000","last_became_peered":"2024-02-13T03:21:13.685760+0000","last_unstale":"2024-02-13T03:22:34.991588+0000","last_undegraded":"2024-02-13T03:22:34.991588+0000","last_fullsized":"2024-02-13T03:22:34.991588+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.996919+0000","last_change":"2024-02-13T03:21:09.674794+0000","last_active":"2024-02-13T03:22:34.996919+0000","last_peered":"2024-02-13T03:22:34.996919+0000","last_clean":"2024-02-13T03:22:34.996919+0000","last_became_active":"2024-02-13T03:21:09.674260+0000","last_became_peered":"2024-02-13T03:21:09.674260+0000","last_unstale":"2024-02-13T03:22:34.996919+0000","last_undegraded":"2024-02-13T03:22:34.996919+0000","last_fullsized":"2024-02-13T03:22:34.996919+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"60'17","reported_seq":45,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991621+0000","last_change":"2024-02-13T03:21:11.682859+0000","last_active":"2024-02-13T03:22:34.991621+0000","last_peered":"2024-02-13T03:22:34.991621+0000","last_clean":"2024-02-13T03:22:34.991621+0000","last_became_active":"2024-02-13T03:21:11.682576+0000","last_became_peered":"2024-02-13T03:21:11.682576+0000","last_unstale":"2024-02-13T03:22:34.991621+0000","last_undegraded":"2024-02-13T03:22:34.991621+0000","last_fullsized":"2024-02-13T03:22:34.991621+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991991+0000","last_change":"2024-02-13T03:21:15.714493+0000","last_active":"2024-02-13T03:22:34.991991+0000","last_peered":"2024-02-13T03:22:34.991991+0000","last_clean":"2024-02-13T03:22:34.991991+0000","last_became_active":"2024-02-13T03:21:15.714169+0000","last_became_peered":"2024-02-13T03:21:15.714169+0000","last_unstale":"2024-02-13T03:22:34.991991+0000","last_undegraded":"2024-02-13T03:22:34.991991+0000","last_fullsized":"2024-02-13T03:22:34.991991+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:14.676095+0000","last_clean_scrub_stamp":"2024-02-13T03:21:14.676095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"60'8","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-13T03:25:42.894612+0000","last_change":"2024-02-13T03:21:13.696276+0000","last_active":"2024-02-13T03:25:42.894612+0000","last_peered":"2024-02-13T03:25:42.894612+0000","last_clean":"2024-02-13T03:25:42.894612+0000","last_became_active":"2024-02-13T03:21:13.695849+0000","last_became_peered":"2024-02-13T03:21:13.695849+0000","last_unstale":"2024-02-13T03:25:42.894612+0000","last_undegraded":"2024-02-13T03:25:42.894612+0000","last_fullsized":"2024-02-13T03:25:42.894612+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:12.657930+0000","last_clean_scrub_stamp":"2024-02-13T03:21:12.657930+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.992534+0000","last_change":"2024-02-13T03:21:09.673847+0000","last_active":"2024-02-13T03:22:34.992534+0000","last_peered":"2024-02-13T03:22:34.992534+0000","last_clean":"2024-02-13T03:22:34.992534+0000","last_became_active":"2024-02-13T03:21:09.673525+0000","last_became_peered":"2024-02-13T03:21:09.673525+0000","last_unstale":"2024-02-13T03:22:34.992534+0000","last_undegraded":"2024-02-13T03:22:34.992534+0000","last_fullsized":"2024-02-13T03:22:34.992534+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:08.631612+0000","last_clean_scrub_stamp":"2024-02-13T03:21:08.631612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"60'10","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-13T03:22:34.991519+0000","last_change":"2024-02-13T03:21:11.679116+0000","last_active":"2024-02-13T03:22:34.991519+0000","last_peered":"2024-02-13T03:22:34.991519+0000","last_clean":"2024-02-13T03:22:34.991519+0000","last_became_active":"2024-02-13T03:21:11.678788+0000","last_became_peered":"2024-02-13T03:21:11.678788+0000","last_unstale":"2024-02-13T03:22:34.991519+0000","last_undegraded":"2024-02-13T03:22:34.991519+0000","last_fullsized":"2024-02-13T03:22:34.991519+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T03:21:10.652090+0000","last_clean_scrub_stamp":"2024-02-13T03:21:10.652090+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":584,"ondisk_log_size":584,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":274,"num_read_kb":269,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430269,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297724,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":643072,"data_stored":241758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91700000000000004,"15min":0.90100000000000002},"min":{"1min":0.61799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3029999999999999,"5min":1.347,"15min":1.347},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.878,"15min":0.86499999999999999},"min":{"1min":0.69399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.234,"5min":1.363,"15min":1.363},"last":1.0429999999999999}]},{"osd":1,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.94299999999999995,"15min":0.94799999999999995},"min":{"1min":0.625,"5min":0.505,"15min":0.505},"max":{"1min":1.1859999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93500000000000005,"15min":0.94099999999999995},"min":{"1min":0.65500000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.288,"5min":1.381,"15min":1.381},"last":1.119}]},{"osd":2,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.96099999999999997,"15min":0.93500000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.272,"5min":1.452,"15min":1.452},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.95399999999999996,"15min":0.93000000000000005},"min":{"1min":0.746,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3959999999999999,"5min":1.528,"15min":1.528},"last":1.0900000000000001}]},{"osd":3,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91400000000000003,"15min":0.90800000000000003},"min":{"1min":0.60299999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.0700000000000001,"5min":1.389,"15min":1.389},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.052,"5min":1.0129999999999999,"15min":1.0109999999999999},"min":{"1min":0.79600000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3100000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.96799999999999997}]},{"osd":4,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.96799999999999997,"15min":0.98599999999999999},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2130000000000001,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.232},{"interface":"front","average":{"1min":1.014,"5min":0.97699999999999998,"15min":0.96099999999999997},"min":{"1min":0.79300000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.252,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.157}]},{"osd":5,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0369999999999999,"15min":1.0569999999999999},"min":{"1min":0.72999999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3340000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.111,"15min":1.0840000000000001},"min":{"1min":0.75600000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.3959999999999999,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.1899999999999999}]},{"osd":6,"last update":"Tue Feb 13 03:24:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1419999999999999,"15min":1.1200000000000001},"min":{"1min":0.77300000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.399,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.272},{"interface":"front","average":{"1min":1.137,"5min":1.095,"15min":1.05},"min":{"1min":0.73299999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.431,"5min":1.534,"15min":1.534},"last":1.2070000000000001}]}]},{"osd":6,"up_from":42,"seq":180388626500,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297704,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447448,"statfs":{"total":95995035648,"available":95690186752,"internally_reserved":0,"allocated":622592,"data_stored":241561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.198,"5min":1.262,"15min":1.262},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.88800000000000001,"15min":0.86399999999999999},"min":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.91200000000000003}]},{"osd":1,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.89300000000000002,"15min":0.93000000000000005},"min":{"1min":0.55500000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.3460000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.147},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90000000000000002,"15min":0.89800000000000002},"min":{"1min":0.60399999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.133,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.89100000000000001}]},{"osd":2,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.93999999999999995,"15min":0.91500000000000004},"min":{"1min":0.752,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.165,"5min":1.292,"15min":1.292},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94399999999999995,"15min":0.94599999999999995},"min":{"1min":0.65700000000000003,"5min":0.627,"15min":0.627},"max":{"1min":1.2010000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.0169999999999999}]},{"osd":3,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93300000000000005,"15min":0.91100000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2589999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0309999999999999,"15min":1.0069999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3029999999999999,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.97099999999999997}]},{"osd":4,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.91800000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2729999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.998,"5min":0.97799999999999998,"15min":0.91300000000000003},"min":{"1min":0.53100000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.3129999999999999,"5min":1.607,"15min":1.607},"last":0.83599999999999997}]},{"osd":5,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.99099999999999999,"15min":0.997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.244,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0700000000000001,"15min":1.056},"min":{"1min":0.76100000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.355,"5min":1.883,"15min":1.883},"last":0.95199999999999996}]},{"osd":7,"last update":"Tue Feb 13 03:24:59 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.101,"15min":1.0600000000000001},"min":{"1min":0.621,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3979999999999999,"5min":1.575,"15min":1.575},"last":1.21},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1639999999999999,"15min":1.1359999999999999},"min":{"1min":0.73599999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.48,"5min":1.931,"15min":1.931},"last":1.1259999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297732,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447420,"statfs":{"total":95995035648,"available":95690158080,"internally_reserved":0,"allocated":651264,"data_stored":240761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.71699999999999997,"15min":0.59099999999999997},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.376},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.745,"15min":0.63},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.41399999999999998},"max":{"1min":1.157,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.76600000000000001}]},{"osd":2,"last update":"Tue Feb 13 03:25:43 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75800000000000001,"15min":0.69699999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1180000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.78400000000000003,"15min":0.71999999999999997},"min":{"1min":0.57499999999999996,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.2929999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.81999999999999995}]},{"osd":3,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.83599999999999997,"15min":0.81200000000000006},"min":{"1min":0.45700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.319,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.625},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.84099999999999997,"15min":0.81200000000000006},"min":{"1min":0.63500000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.0249999999999999}]},{"osd":4,"last update":"Tue Feb 13 03:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.0089999999999999,"15min":0.97099999999999997},"min":{"1min":0.61199999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2230000000000001,"5min":1.587,"15min":1.587},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.159,"5min":1.079,"15min":1.034},"min":{"1min":0.79600000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.1539999999999999}]},{"osd":5,"last update":"Tue Feb 13 03:25:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0389999999999999,"15min":1.002},"min":{"1min":0.85799999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5169999999999999,"5min":1.581,"15min":1.581},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.131,"15min":1.0860000000000001},"min":{"1min":0.88100000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.4970000000000001,"5min":1.675,"15min":1.675},"last":1.175}]},{"osd":6,"last update":"Tue Feb 13 03:25:22 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1779999999999999,"15min":1.1699999999999999},"min":{"1min":0.90000000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.417,"5min":1.734,"15min":1.734},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.1850000000000001,"15min":1.157},"min":{"1min":1.0109999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":1.228}]},{"osd":7,"last update":"Tue Feb 13 03:25:05 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.25,"15min":1.2609999999999999},"min":{"1min":0.95499999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.621,"5min":1.742,"15min":1.742},"last":1.28},{"interface":"front","average":{"1min":1.319,"5min":1.262,"15min":1.268},"min":{"1min":1.0720000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.3520000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297724,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":643072,"data_stored":241110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Feb 13 03:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73799999999999999,"15min":1.075},"min":{"1min":0.44900000000000001,"5min":0.35399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0940000000000001,"5min":1.4690000000000001,"15min":20.219999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76100000000000001,"15min":0.60599999999999998},"min":{"1min":0.55600000000000005,"5min":0.34699999999999998,"15min":0.32000000000000001},"max":{"1min":1.0700000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.68999999999999995}]},{"osd":2,"last update":"Tue Feb 13 03:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.67900000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.28399999999999997},"max":{"1min":1.0329999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.77900000000000003,"15min":0.69599999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.39000000000000001},"max":{"1min":1.167,"5min":1.371,"15min":1.3899999999999999},"last":0.71999999999999997}]},{"osd":3,"last update":"Tue Feb 13 03:25:23 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.83799999999999997,"15min":0.80700000000000005},"min":{"1min":0.64300000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.125,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88,"15min":0.82199999999999995},"min":{"1min":0.498,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.181,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.84099999999999997}]},{"osd":4,"last update":"Tue Feb 13 03:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89200000000000002,"15min":0.873},"min":{"1min":0.67700000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.26,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.745},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1080000000000001,"15min":1.056},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.403,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.95699999999999996}]},{"osd":5,"last update":"Tue Feb 13 03:25:40 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0549999999999999,"15min":1.0449999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3500000000000001,"5min":1.617,"15min":1.617},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.087,"15min":1.125},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.321,"5min":1.591,"15min":1.7649999999999999},"last":1.071}]},{"osd":6,"last update":"Tue Feb 13 03:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.133,"15min":1.131},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4510000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.169},{"interface":"front","average":{"1min":1.135,"5min":1.1599999999999999,"15min":1.1930000000000001},"min":{"1min":0.67000000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5549999999999999,"5min":1.718,"15min":1.718},"last":1.2450000000000001}]},{"osd":7,"last update":"Tue Feb 13 03:25:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.044,"15min":1.024},"min":{"1min":0.82199999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2310000000000001,"15min":1.2110000000000001},"min":{"1min":0.78400000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.4399999999999999,"5min":1.661,"15min":1.661},"last":1.268}]}]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297704,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447448,"statfs":{"total":95995035648,"available":95690186752,"internally_reserved":0,"allocated":622592,"data_stored":240928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:25:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72399999999999998,"15min":0.625},"min":{"1min":0.61299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.016,"5min":1.111,"15min":1.111},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80300000000000005,"15min":0.69099999999999995},"min":{"1min":0.58699999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.486,"5min":1.794,"15min":1.794},"last":1.012}]},{"osd":1,"last update":"Tue Feb 13 03:25:41 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.755,"15min":0.67400000000000004},"min":{"1min":0.56200000000000006,"5min":0.48499999999999999,"15min":0.376},"max":{"1min":1.2989999999999999,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.874,"15min":0.754},"min":{"1min":0.64400000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2070000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.77400000000000002}]},{"osd":3,"last update":"Tue Feb 13 03:25:27 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.89600000000000002,"15min":0.82999999999999996},"min":{"1min":0.60399999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.444,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.879,"15min":0.80100000000000005},"min":{"1min":0.70099999999999996,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.256,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":0.64500000000000002}]},{"osd":4,"last update":"Tue Feb 13 03:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.097,"15min":0.999},"min":{"1min":0.79400000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.8660000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.292},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0640000000000001,"15min":0.97399999999999998},"min":{"1min":0.77700000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.548,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.94899999999999995}]},{"osd":5,"last update":"Tue Feb 13 03:25:41 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1000000000000001,"15min":1.022},"min":{"1min":0.86299999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4770000000000001,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.21,"15min":1.1439999999999999},"min":{"1min":0.873,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.446,"5min":1.865,"15min":1.865},"last":1.1719999999999999}]},{"osd":6,"last update":"Tue Feb 13 03:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.113,"15min":1.0629999999999999},"min":{"1min":0.877,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.53,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.234},{"interface":"front","average":{"1min":1.296,"5min":1.2649999999999999,"15min":1.2310000000000001},"min":{"1min":1.0429999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5920000000000001,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.052}]},{"osd":7,"last update":"Tue Feb 13 03:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.145,"15min":1.113},"min":{"1min":0.95299999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.637,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2749999999999999,"15min":1.2470000000000001},"min":{"1min":0.72499999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.367}]}]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297732,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447420,"statfs":{"total":95995035648,"available":95690158080,"internally_reserved":0,"allocated":651264,"data_stored":241752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:25:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.73399999999999999,"15min":0.67800000000000005},"min":{"1min":0.42599999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96899999999999997,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.746,"15min":0.72799999999999998},"min":{"1min":0.54800000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1259999999999999,"5min":1.208,"15min":1.208},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Feb 13 03:25:30 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.751,"15min":0.79300000000000004},"min":{"1min":0.52000000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.2949999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78600000000000003,"15min":0.76600000000000001},"min":{"1min":0.58599999999999997,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1719999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.81999999999999995}]},{"osd":2,"last update":"Tue Feb 13 03:25:30 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85099999999999998,"15min":0.83899999999999997},"min":{"1min":0.61599999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2529999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.48899999999999999,"5min":0.38,"15min":0.38},"max":{"1min":1.042,"5min":1.214,"15min":1.214},"last":0.57399999999999995}]},{"osd":4,"last update":"Tue Feb 13 03:24:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.92700000000000005,"15min":0.89700000000000002},"min":{"1min":0.66400000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.97599999999999998,"15min":0.92200000000000004},"min":{"1min":0.748,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.234,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Tue Feb 13 03:25:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0760000000000001,"15min":1.038},"min":{"1min":0.82499999999999996,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5029999999999999,"5min":2.468,"15min":2.468},"last":1.254},{"interface":"front","average":{"1min":1.119,"5min":1.0469999999999999,"15min":1.024},"min":{"1min":0.71999999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.878}]},{"osd":6,"last update":"Tue Feb 13 03:25:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.0720000000000001,"15min":1.0469999999999999},"min":{"1min":0.84799999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.173,"15min":1.234},"min":{"1min":0.79100000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5169999999999999,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":1.3140000000000001}]},{"osd":7,"last update":"Tue Feb 13 03:25:05 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.093,"15min":1.0720000000000001},"min":{"1min":0.85799999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.145,"5min":1.133,"15min":1.1359999999999999},"min":{"1min":0.81899999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.379,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.91800000000000004}]}]},{"osd":4,"up_from":30,"seq":128849018966,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297724,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":643072,"data_stored":241220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.83599999999999997,"15min":0.79400000000000004},"min":{"1min":0.63400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.077,"5min":1.357,"15min":1.357},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.92900000000000005,"15min":0.83499999999999996},"min":{"1min":0.64100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.67300000000000004}]},{"osd":1,"last update":"Tue Feb 13 03:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.86599999999999999,"15min":0.81000000000000005},"min":{"1min":0.65300000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.1399999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.89100000000000001,"15min":0.86299999999999999},"min":{"1min":0.68799999999999994,"5min":0.372,"15min":0.372},"max":{"1min":1.246,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.91200000000000003}]},{"osd":2,"last update":"Tue Feb 13 03:25:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.89300000000000002,"15min":0.86499999999999999},"min":{"1min":0.59599999999999997,"5min":0.314,"15min":0.314},"max":{"1min":1.1699999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.016,"5min":0.97799999999999998,"15min":0.90600000000000003},"min":{"1min":0.73699999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.22,"5min":1.577,"15min":1.577},"last":0.78100000000000003}]},{"osd":3,"last update":"Tue Feb 13 03:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":0.99299999999999999,"15min":0.94699999999999995},"min":{"1min":0.77200000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.393,"5min":1.681,"15min":1.681},"last":1.048},{"interface":"front","average":{"1min":1.046,"5min":1.0209999999999999,"15min":0.93999999999999995},"min":{"1min":0.73099999999999998,"5min":0.622,"15min":0.622},"max":{"1min":1.268,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.073}]},{"osd":5,"last update":"Tue Feb 13 03:25:39 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.92900000000000005,"15min":0.80200000000000005},"min":{"1min":0.68000000000000005,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2190000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.91600000000000004,"15min":0.77000000000000002},"min":{"1min":0.69599999999999995,"5min":0.47899999999999998,"15min":0.46000000000000002},"max":{"1min":1.206,"5min":1.252,"15min":1.252},"last":0.96599999999999997}]},{"osd":6,"last update":"Tue Feb 13 03:25:24 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.95699999999999996,"15min":0.90500000000000003},"min":{"1min":0.81699999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.23,"5min":1.504,"15min":1.504},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.038,"15min":0.97199999999999998},"min":{"1min":0.76000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.94799999999999995}]},{"osd":7,"last update":"Tue Feb 13 03:25:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0569999999999999,"15min":1.054},"min":{"1min":0.89100000000000001,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3320000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.083,"15min":1.0680000000000001},"min":{"1min":0.92400000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.329,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.1080000000000001}]}]},{"osd":5,"up_from":36,"seq":154618822734,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297704,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447448,"statfs":{"total":95995035648,"available":95690186752,"internally_reserved":0,"allocated":622592,"data_stored":240463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 03:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89100000000000001,"15min":0.81699999999999995},"min":{"1min":0.67600000000000005,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2050000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.0329999999999999,"5min":0.92200000000000004,"15min":0.91500000000000004},"min":{"1min":0.64700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.3009999999999999},"last":0.92400000000000004}]},{"osd":1,"last update":"Tue Feb 13 03:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.90400000000000003,"15min":0.86599999999999999},"min":{"1min":0.67900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.749},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.93100000000000005,"15min":0.90600000000000003},"min":{"1min":0.64100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1990000000000001,"5min":1.347,"15min":1.6439999999999999},"last":1.111}]},{"osd":2,"last update":"Tue Feb 13 03:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.89900000000000002,"15min":0.86599999999999999},"min":{"1min":0.63400000000000001,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.385},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.95399999999999996,"15min":0.89800000000000002},"min":{"1min":0.80800000000000005,"5min":0.55200000000000005,"15min":0.52500000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.7529999999999999},"last":0.81000000000000005}]},{"osd":3,"last update":"Tue Feb 13 03:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.92400000000000004,"15min":0.88200000000000001},"min":{"1min":0.75900000000000001,"5min":0.59999999999999998,"15min":0.57099999999999995},"max":{"1min":1.194,"5min":1.3220000000000001,"15min":1.351},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.98199999999999998,"15min":0.93000000000000005},"min":{"1min":0.76800000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.294,"5min":3.0779999999999998,"15min":3.0779999999999998},"last":0.89500000000000002}]},{"osd":4,"last update":"Tue Feb 13 03:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91000000000000003,"15min":0.82999999999999996},"min":{"1min":0.68799999999999994,"5min":0.52200000000000002,"15min":0.47799999999999998},"max":{"1min":1.2090000000000001,"5min":1.302,"15min":1.8009999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.002,"5min":0.94799999999999995,"15min":0.85799999999999998},"min":{"1min":0.73199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2410000000000001,"5min":1.3109999999999999,"15min":1.468},"last":1.0289999999999999}]},{"osd":6,"last update":"Tue Feb 13 03:25:27 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.96599999999999997,"15min":0.89600000000000002},"min":{"1min":0.92500000000000004,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.405,"5min":1.52,"15min":1.52},"last":1.298},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.999,"15min":0.94299999999999995},"min":{"1min":0.68500000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.97399999999999998}]},{"osd":7,"last update":"Tue Feb 13 03:25:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.0569999999999999,"15min":1.012},"min":{"1min":0.90700000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3779999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.0600000000000001,"15min":0.98899999999999999},"min":{"1min":0.89800000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5940000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.1739999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-13T03:25:52.326 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-13T03:25:52.327 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-13T03:25:52.327 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-13T03:25:52.327 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 -- ceph health --format=json 2024-02-13T03:25:53.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:52 smithi161 bash[19769]: audit 2024-02-13T03:25:51.652558+0000 mgr.y (mgr.24473) 183 : audit [DBG] from='client.14766 v1:172.21.15.161:0/3054618815' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T03:25:53.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:52 smithi161 bash[27879]: audit 2024-02-13T03:25:51.652558+0000 mgr.y (mgr.24473) 183 : audit [DBG] from='client.14766 v1:172.21.15.161:0/3054618815' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T03:25:53.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:52 smithi183 bash[20824]: audit 2024-02-13T03:25:51.652558+0000 mgr.y (mgr.24473) 183 : audit [DBG] from='client.14766 v1:172.21.15.161:0/3054618815' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T03:25:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:53 smithi161 bash[19769]: cluster 2024-02-13T03:25:52.666104+0000 mgr.y (mgr.24473) 184 : cluster [DBG] pgmap v134: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:54.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:53 smithi161 bash[27879]: cluster 2024-02-13T03:25:52.666104+0000 mgr.y (mgr.24473) 184 : cluster [DBG] pgmap v134: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:54.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:53 smithi183 bash[20824]: cluster 2024-02-13T03:25:52.666104+0000 mgr.y (mgr.24473) 184 : cluster [DBG] pgmap v134: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:56.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:55 smithi161 bash[19769]: cluster 2024-02-13T03:25:54.666829+0000 mgr.y (mgr.24473) 185 : cluster [DBG] pgmap v135: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:56.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:55 smithi161 bash[27879]: cluster 2024-02-13T03:25:54.666829+0000 mgr.y (mgr.24473) 185 : cluster [DBG] pgmap v135: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:56.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:55 smithi183 bash[20824]: cluster 2024-02-13T03:25:54.666829+0000 mgr.y (mgr.24473) 185 : cluster [DBG] pgmap v135: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:25:56.237 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:25:55 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:25:57.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:56 smithi161 bash[19769]: audit 2024-02-13T03:25:55.746747+0000 mgr.y (mgr.24473) 186 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:57.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:56 smithi161 bash[27879]: audit 2024-02-13T03:25:55.746747+0000 mgr.y (mgr.24473) 186 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:57.211 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T03:25:57.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:56 smithi183 bash[20824]: audit 2024-02-13T03:25:55.746747+0000 mgr.y (mgr.24473) 186 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:25:58.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:57 smithi161 bash[27879]: cluster 2024-02-13T03:25:56.668213+0000 mgr.y (mgr.24473) 187 : cluster [DBG] pgmap v136: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:58.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:57 smithi161 bash[19769]: cluster 2024-02-13T03:25:56.668213+0000 mgr.y (mgr.24473) 187 : cluster [DBG] pgmap v136: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:58.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:57 smithi183 bash[20824]: cluster 2024-02-13T03:25:56.668213+0000 mgr.y (mgr.24473) 187 : cluster [DBG] pgmap v136: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:25:59.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:25:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:25:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:26:00.036 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:25:59 smithi161 bash[19769]: cluster 2024-02-13T03:25:58.668962+0000 mgr.y (mgr.24473) 188 : cluster [DBG] pgmap v137: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:00.036 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:25:59 smithi161 bash[27879]: cluster 2024-02-13T03:25:58.668962+0000 mgr.y (mgr.24473) 188 : cluster [DBG] pgmap v137: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:00.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.160+0000 7f5f96b70700 1 -- 172.21.15.161:0/3731786474 >> v1:172.21.15.161:6789/0 conn(0x7f5f90103330 legacy=0x7f5f901057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:26:00.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.160+0000 7f5f96b70700 1 -- 172.21.15.161:0/3731786474 shutdown_connections 2024-02-13T03:26:00.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.160+0000 7f5f96b70700 1 -- 172.21.15.161:0/3731786474 >> 172.21.15.161:0/3731786474 conn(0x7f5f900f2620 msgr2=0x7f5f900f4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:26:00.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.160+0000 7f5f96b70700 1 -- 172.21.15.161:0/3731786474 shutdown_connections 2024-02-13T03:26:00.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.160+0000 7f5f96b70700 1 -- 172.21.15.161:0/3731786474 wait complete. 2024-02-13T03:26:00.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.160+0000 7f5f96b70700 1 Processor -- start 2024-02-13T03:26:00.166 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.160+0000 7f5f96b70700 1 -- start start 2024-02-13T03:26:00.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f96b70700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5f901174b0 con 0x7f5f90103330 2024-02-13T03:26:00.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f9490c700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f5f900fca50 0x7f5f90060eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:34208/0 (socket says 172.21.15.161:34208) 2024-02-13T03:26:00.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f96b70700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5f90118690 con 0x7f5f900ff7f0 2024-02-13T03:26:00.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f9490c700 1 -- 172.21.15.161:0/10725423 learned_addr learned my addr 172.21.15.161:0/10725423 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:26:00.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f96b70700 1 -- 172.21.15.161:0/10725423 --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5f90119890 con 0x7f5f900fca50 2024-02-13T03:26:00.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526021216 0 0) 0x7f5f90119890 con 0x7f5f900fca50 2024-02-13T03:26:00.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f78003630 con 0x7f5f900fca50 2024-02-13T03:26:00.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885732183 0 0) 0x7f5f901174b0 con 0x7f5f90103330 2024-02-13T03:26:00.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f90119890 con 0x7f5f90103330 2024-02-13T03:26:00.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222901933 0 0) 0x7f5f90118690 con 0x7f5f900ff7f0 2024-02-13T03:26:00.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f901174b0 con 0x7f5f900ff7f0 2024-02-13T03:26:00.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 884975885 0 0) 0x7f5f78003630 con 0x7f5f900fca50 2024-02-13T03:26:00.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5f90118690 con 0x7f5f900fca50 2024-02-13T03:26:00.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5f8c002950 con 0x7f5f900fca50 2024-02-13T03:26:00.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 218841892 0 0) 0x7f5f90119890 con 0x7f5f90103330 2024-02-13T03:26:00.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5f78003630 con 0x7f5f90103330 2024-02-13T03:26:00.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5f88002240 con 0x7f5f90103330 2024-02-13T03:26:00.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2327942648 0 0) 0x7f5f901174b0 con 0x7f5f900ff7f0 2024-02-13T03:26:00.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5f90119890 con 0x7f5f900ff7f0 2024-02-13T03:26:00.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2212747574 0 0) 0x7f5f90118690 con 0x7f5f900fca50 2024-02-13T03:26:00.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 >> v1:172.21.15.183:6789/0 conn(0x7f5f900ff7f0 legacy=0x7f5f900615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:26:00.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 >> v1:172.21.15.161:6789/0 conn(0x7f5f90103330 legacy=0x7f5f90115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:26:00.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9011aa90 con 0x7f5f900fca50 2024-02-13T03:26:00.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f96b70700 1 -- 172.21.15.161:0/10725423 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f901176a0 con 0x7f5f900fca50 2024-02-13T03:26:00.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f5f8c0017d0 con 0x7f5f900fca50 2024-02-13T03:26:00.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f96b70700 1 -- 172.21.15.161:0/10725423 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f90117b90 con 0x7f5f900fca50 2024-02-13T03:26:00.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5f8c005a70 con 0x7f5f900fca50 2024-02-13T03:26:00.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.164+0000 7f5f96b70700 1 -- 172.21.15.161:0/10725423 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f900604a0 con 0x7f5f900fca50 2024-02-13T03:26:00.171 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.168+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f5f8c001330 con 0x7f5f900fca50 2024-02-13T03:26:00.172 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.168+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(63..63 src has 1..63) v4 ==== 6143+0+0 (unknown 3137093693 0 0) 0x7f5f8c078530 con 0x7f5f900fca50 2024-02-13T03:26:00.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.172+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f5f8c04c100 con 0x7f5f900fca50 2024-02-13T03:26:00.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:25:59 smithi183 bash[20824]: cluster 2024-02-13T03:25:58.668962+0000 mgr.y (mgr.24473) 188 : cluster [DBG] pgmap v137: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:00.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.396+0000 7f5f96b70700 1 -- 172.21.15.161:0/10725423 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5f900f81e0 con 0x7f5f900fca50 2024-02-13T03:26:00.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.396+0000 7f5f85ffb700 1 -- 172.21.15.161:0/10725423 <== mon.2 v1:172.21.15.161:6790/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (unknown 3730786198 0 3397129104) 0x7f5f8c04a510 con 0x7f5f900fca50 2024-02-13T03:26:00.402 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-02-13T03:26:00.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.400+0000 7f5f96b70700 1 -- 172.21.15.161:0/10725423 >> v1:172.21.15.161:6800/4114337139 conn(0x7f5f780622c0 legacy=0x7f5f78064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:26:00.405 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.400+0000 7f5f96b70700 1 -- 172.21.15.161:0/10725423 >> v1:172.21.15.161:6790/0 conn(0x7f5f900fca50 legacy=0x7f5f90060eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:26:00.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.400+0000 7f5f96b70700 1 -- 172.21.15.161:0/10725423 shutdown_connections 2024-02-13T03:26:00.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.400+0000 7f5f96b70700 1 -- 172.21.15.161:0/10725423 >> 172.21.15.161:0/10725423 conn(0x7f5f900f2620 msgr2=0x7f5f901067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:26:00.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.400+0000 7f5f96b70700 1 -- 172.21.15.161:0/10725423 shutdown_connections 2024-02-13T03:26:00.406 INFO:teuthology.orchestra.run.smithi161.stderr:2024-02-13T03:26:00.400+0000 7f5f96b70700 1 -- 172.21.15.161:0/10725423 wait complete. 2024-02-13T03:26:01.043 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:00 smithi161 bash[27879]: audit 2024-02-13T03:26:00.399727+0000 mon.c (mon.2) 71 : audit [DBG] from='client.? v1:172.21.15.161:0/10725423' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-13T03:26:01.044 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:00 smithi161 bash[19769]: audit 2024-02-13T03:26:00.399727+0000 mon.c (mon.2) 71 : audit [DBG] from='client.? v1:172.21.15.161:0/10725423' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-13T03:26:01.045 INFO:teuthology.orchestra.run.smithi161.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-13T03:26:01.045 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-13T03:26:01.045 INFO:tasks.cephadm:Setup complete, yielding 2024-02-13T03:26:01.046 INFO:teuthology.run_tasks:Running task workunit... 2024-02-13T03:26:01.059 INFO:tasks.workunit:Pulling workunits from ref 84e714279aa684636ac645daf0a1a85c2094cfb3 2024-02-13T03:26:01.060 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-13T03:26:01.060 DEBUG:teuthology.orchestra.run.smithi161:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-13T03:26:01.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T03:26:01.067 INFO:teuthology.orchestra.run.smithi161.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-02-13T03:26:01.068 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-02-13T03:26:01.117 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-02-13T03:26:01.117 DEBUG:teuthology.orchestra.run.smithi161:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-02-13T03:26:01.165 INFO:tasks.workunit:timeout=3h 2024-02-13T03:26:01.165 INFO:tasks.workunit:cleanup=True 2024-02-13T03:26:01.166 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ljflores/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 84e714279aa684636ac645daf0a1a85c2094cfb3 2024-02-13T03:26:01.214 INFO:tasks.workunit.client.0.smithi161.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-13T03:26:01.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:00 smithi183 bash[20824]: audit 2024-02-13T03:26:00.399727+0000 mon.c (mon.2) 71 : audit [DBG] from='client.? v1:172.21.15.161:0/10725423' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-13T03:26:01.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:26:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:26:01] "GET /metrics HTTP/1.1" 200 172184 "" "Prometheus/2.33.4" 2024-02-13T03:26:02.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:01 smithi161 bash[27879]: cluster 2024-02-13T03:26:00.670413+0000 mgr.y (mgr.24473) 189 : cluster [DBG] pgmap v138: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:02.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:01 smithi161 bash[19769]: cluster 2024-02-13T03:26:00.670413+0000 mgr.y (mgr.24473) 189 : cluster [DBG] pgmap v138: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:02.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:01 smithi183 bash[20824]: cluster 2024-02-13T03:26:00.670413+0000 mgr.y (mgr.24473) 189 : cluster [DBG] pgmap v138: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:03 smithi161 bash[27879]: cluster 2024-02-13T03:26:02.671121+0000 mgr.y (mgr.24473) 190 : cluster [DBG] pgmap v139: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:03 smithi161 bash[19769]: cluster 2024-02-13T03:26:02.671121+0000 mgr.y (mgr.24473) 190 : cluster [DBG] pgmap v139: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:03.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:03 smithi183 bash[20824]: cluster 2024-02-13T03:26:02.671121+0000 mgr.y (mgr.24473) 190 : cluster [DBG] pgmap v139: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:04.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:04 smithi161 bash[19769]: audit 2024-02-13T03:26:03.527036+0000 mon.c (mon.2) 72 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:26:04.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:04 smithi161 bash[19769]: audit 2024-02-13T03:26:03.845642+0000 mon.c (mon.2) 73 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:26:04.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:04 smithi161 bash[19769]: audit 2024-02-13T03:26:03.846816+0000 mon.c (mon.2) 74 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:26:04.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:04 smithi161 bash[19769]: audit 2024-02-13T03:26:03.854339+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:26:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:04 smithi161 bash[27879]: audit 2024-02-13T03:26:03.527036+0000 mon.c (mon.2) 72 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:26:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:04 smithi161 bash[27879]: audit 2024-02-13T03:26:03.845642+0000 mon.c (mon.2) 73 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:26:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:04 smithi161 bash[27879]: audit 2024-02-13T03:26:03.846816+0000 mon.c (mon.2) 74 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:26:04.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:04 smithi161 bash[27879]: audit 2024-02-13T03:26:03.854339+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:26:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:04 smithi183 bash[20824]: audit 2024-02-13T03:26:03.527036+0000 mon.c (mon.2) 72 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:26:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:04 smithi183 bash[20824]: audit 2024-02-13T03:26:03.845642+0000 mon.c (mon.2) 73 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:26:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:04 smithi183 bash[20824]: audit 2024-02-13T03:26:03.846816+0000 mon.c (mon.2) 74 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:26:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:04 smithi183 bash[20824]: audit 2024-02-13T03:26:03.854339+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:26:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:05 smithi161 bash[27879]: cluster 2024-02-13T03:26:04.671770+0000 mgr.y (mgr.24473) 191 : cluster [DBG] pgmap v140: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:05 smithi161 bash[19769]: cluster 2024-02-13T03:26:04.671770+0000 mgr.y (mgr.24473) 191 : cluster [DBG] pgmap v140: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:05 smithi183 bash[20824]: cluster 2024-02-13T03:26:04.671770+0000 mgr.y (mgr.24473) 191 : cluster [DBG] pgmap v140: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:06.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:26:05 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:26:06.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:06 smithi161 bash[27879]: audit 2024-02-13T03:26:05.751680+0000 mgr.y (mgr.24473) 192 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:06.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:06 smithi161 bash[19769]: audit 2024-02-13T03:26:05.751680+0000 mgr.y (mgr.24473) 192 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:06.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:06 smithi183 bash[20824]: audit 2024-02-13T03:26:05.751680+0000 mgr.y (mgr.24473) 192 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:07 smithi161 bash[27879]: cluster 2024-02-13T03:26:06.673164+0000 mgr.y (mgr.24473) 193 : cluster [DBG] pgmap v141: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:07 smithi161 bash[19769]: cluster 2024-02-13T03:26:06.673164+0000 mgr.y (mgr.24473) 193 : cluster [DBG] pgmap v141: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:07.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:07 smithi183 bash[20824]: cluster 2024-02-13T03:26:06.673164+0000 mgr.y (mgr.24473) 193 : cluster [DBG] pgmap v141: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:09.731 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:26:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:26:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:26:09.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:09 smithi183 bash[20824]: cluster 2024-02-13T03:26:08.673570+0000 mgr.y (mgr.24473) 194 : cluster [DBG] pgmap v142: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:10.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:09 smithi161 bash[27879]: cluster 2024-02-13T03:26:08.673570+0000 mgr.y (mgr.24473) 194 : cluster [DBG] pgmap v142: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:10.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:09 smithi161 bash[19769]: cluster 2024-02-13T03:26:08.673570+0000 mgr.y (mgr.24473) 194 : cluster [DBG] pgmap v142: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:11.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:26:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:26:11] "GET /metrics HTTP/1.1" 200 172190 "" "Prometheus/2.33.4" 2024-02-13T03:26:11.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:11 smithi183 bash[20824]: cluster 2024-02-13T03:26:10.674972+0000 mgr.y (mgr.24473) 195 : cluster [DBG] pgmap v143: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:11 smithi161 bash[27879]: cluster 2024-02-13T03:26:10.674972+0000 mgr.y (mgr.24473) 195 : cluster [DBG] pgmap v143: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:12.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:11 smithi161 bash[19769]: cluster 2024-02-13T03:26:10.674972+0000 mgr.y (mgr.24473) 195 : cluster [DBG] pgmap v143: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:13 smithi161 bash[27879]: cluster 2024-02-13T03:26:12.675633+0000 mgr.y (mgr.24473) 196 : cluster [DBG] pgmap v144: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:13.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:13 smithi161 bash[19769]: cluster 2024-02-13T03:26:12.675633+0000 mgr.y (mgr.24473) 196 : cluster [DBG] pgmap v144: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:13 smithi183 bash[20824]: cluster 2024-02-13T03:26:12.675633+0000 mgr.y (mgr.24473) 196 : cluster [DBG] pgmap v144: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:15.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:15 smithi183 bash[20824]: cluster 2024-02-13T03:26:14.676302+0000 mgr.y (mgr.24473) 197 : cluster [DBG] pgmap v145: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:15.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:26:15 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:26:16.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:15 smithi161 bash[27879]: cluster 2024-02-13T03:26:14.676302+0000 mgr.y (mgr.24473) 197 : cluster [DBG] pgmap v145: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:16.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:15 smithi161 bash[19769]: cluster 2024-02-13T03:26:14.676302+0000 mgr.y (mgr.24473) 197 : cluster [DBG] pgmap v145: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:17.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:16 smithi161 bash[27879]: audit 2024-02-13T03:26:15.759072+0000 mgr.y (mgr.24473) 198 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:17.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:16 smithi161 bash[19769]: audit 2024-02-13T03:26:15.759072+0000 mgr.y (mgr.24473) 198 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:17.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:16 smithi183 bash[20824]: audit 2024-02-13T03:26:15.759072+0000 mgr.y (mgr.24473) 198 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:18.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:17 smithi161 bash[27879]: cluster 2024-02-13T03:26:16.678049+0000 mgr.y (mgr.24473) 199 : cluster [DBG] pgmap v146: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:18.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:17 smithi161 bash[19769]: cluster 2024-02-13T03:26:16.678049+0000 mgr.y (mgr.24473) 199 : cluster [DBG] pgmap v146: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:18.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:17 smithi183 bash[20824]: cluster 2024-02-13T03:26:16.678049+0000 mgr.y (mgr.24473) 199 : cluster [DBG] pgmap v146: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:19.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:26:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:26:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:26:20.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:19 smithi161 bash[27879]: cluster 2024-02-13T03:26:18.678530+0000 mgr.y (mgr.24473) 200 : cluster [DBG] pgmap v147: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:20.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:19 smithi161 bash[19769]: cluster 2024-02-13T03:26:18.678530+0000 mgr.y (mgr.24473) 200 : cluster [DBG] pgmap v147: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:20.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:19 smithi183 bash[20824]: cluster 2024-02-13T03:26:18.678530+0000 mgr.y (mgr.24473) 200 : cluster [DBG] pgmap v147: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:21.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:26:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:26:21] "GET /metrics HTTP/1.1" 200 172195 "" "Prometheus/2.33.4" 2024-02-13T03:26:22.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:21 smithi161 bash[27879]: cluster 2024-02-13T03:26:20.679374+0000 mgr.y (mgr.24473) 201 : cluster [DBG] pgmap v148: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:22.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:21 smithi161 bash[19769]: cluster 2024-02-13T03:26:20.679374+0000 mgr.y (mgr.24473) 201 : cluster [DBG] pgmap v148: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:22.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:21 smithi183 bash[20824]: cluster 2024-02-13T03:26:20.679374+0000 mgr.y (mgr.24473) 201 : cluster [DBG] pgmap v148: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:23 smithi161 bash[27879]: cluster 2024-02-13T03:26:22.679806+0000 mgr.y (mgr.24473) 202 : cluster [DBG] pgmap v149: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:23.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:23 smithi161 bash[19769]: cluster 2024-02-13T03:26:22.679806+0000 mgr.y (mgr.24473) 202 : cluster [DBG] pgmap v149: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:23 smithi183 bash[20824]: cluster 2024-02-13T03:26:22.679806+0000 mgr.y (mgr.24473) 202 : cluster [DBG] pgmap v149: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:26.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:25 smithi161 bash[27879]: cluster 2024-02-13T03:26:24.680218+0000 mgr.y (mgr.24473) 203 : cluster [DBG] pgmap v150: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:26.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:25 smithi161 bash[19769]: cluster 2024-02-13T03:26:24.680218+0000 mgr.y (mgr.24473) 203 : cluster [DBG] pgmap v150: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:26.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:25 smithi183 bash[20824]: cluster 2024-02-13T03:26:24.680218+0000 mgr.y (mgr.24473) 203 : cluster [DBG] pgmap v150: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:26.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:26:25 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:26:27.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:26 smithi161 bash[27879]: audit 2024-02-13T03:26:25.770210+0000 mgr.y (mgr.24473) 204 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:27.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:26 smithi161 bash[19769]: audit 2024-02-13T03:26:25.770210+0000 mgr.y (mgr.24473) 204 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:27.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:26 smithi183 bash[20824]: audit 2024-02-13T03:26:25.770210+0000 mgr.y (mgr.24473) 204 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:28.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:27 smithi161 bash[27879]: cluster 2024-02-13T03:26:26.680994+0000 mgr.y (mgr.24473) 205 : cluster [DBG] pgmap v151: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:28.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:27 smithi161 bash[19769]: cluster 2024-02-13T03:26:26.680994+0000 mgr.y (mgr.24473) 205 : cluster [DBG] pgmap v151: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:28.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:27 smithi183 bash[20824]: cluster 2024-02-13T03:26:26.680994+0000 mgr.y (mgr.24473) 205 : cluster [DBG] pgmap v151: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:29.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:26:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:26:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:26:30.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:29 smithi161 bash[27879]: cluster 2024-02-13T03:26:28.681420+0000 mgr.y (mgr.24473) 206 : cluster [DBG] pgmap v152: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T03:26:30.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:29 smithi161 bash[19769]: cluster 2024-02-13T03:26:28.681420+0000 mgr.y (mgr.24473) 206 : cluster [DBG] pgmap v152: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T03:26:30.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:29 smithi183 bash[20824]: cluster 2024-02-13T03:26:28.681420+0000 mgr.y (mgr.24473) 206 : cluster [DBG] pgmap v152: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T03:26:31.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:26:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:26:31] "GET /metrics HTTP/1.1" 200 172195 "" "Prometheus/2.33.4" 2024-02-13T03:26:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:31 smithi161 bash[27879]: cluster 2024-02-13T03:26:30.682774+0000 mgr.y (mgr.24473) 207 : cluster [DBG] pgmap v153: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:32.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:31 smithi161 bash[19769]: cluster 2024-02-13T03:26:30.682774+0000 mgr.y (mgr.24473) 207 : cluster [DBG] pgmap v153: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:32.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:31 smithi183 bash[20824]: cluster 2024-02-13T03:26:30.682774+0000 mgr.y (mgr.24473) 207 : cluster [DBG] pgmap v153: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:33 smithi161 bash[27879]: cluster 2024-02-13T03:26:32.683515+0000 mgr.y (mgr.24473) 208 : cluster [DBG] pgmap v154: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T03:26:33.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:33 smithi161 bash[19769]: cluster 2024-02-13T03:26:32.683515+0000 mgr.y (mgr.24473) 208 : cluster [DBG] pgmap v154: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T03:26:33.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:33 smithi183 bash[20824]: cluster 2024-02-13T03:26:32.683515+0000 mgr.y (mgr.24473) 208 : cluster [DBG] pgmap v154: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T03:26:36.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:35 smithi161 bash[27879]: cluster 2024-02-13T03:26:34.684252+0000 mgr.y (mgr.24473) 209 : cluster [DBG] pgmap v155: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T03:26:36.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:35 smithi161 bash[19769]: cluster 2024-02-13T03:26:34.684252+0000 mgr.y (mgr.24473) 209 : cluster [DBG] pgmap v155: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T03:26:36.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:35 smithi183 bash[20824]: cluster 2024-02-13T03:26:34.684252+0000 mgr.y (mgr.24473) 209 : cluster [DBG] pgmap v155: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T03:26:36.237 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:26:35 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:26:37.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:36 smithi161 bash[27879]: audit 2024-02-13T03:26:35.780337+0000 mgr.y (mgr.24473) 210 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:37.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:36 smithi161 bash[19769]: audit 2024-02-13T03:26:35.780337+0000 mgr.y (mgr.24473) 210 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:36 smithi183 bash[20824]: audit 2024-02-13T03:26:35.780337+0000 mgr.y (mgr.24473) 210 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:38.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:37 smithi161 bash[27879]: cluster 2024-02-13T03:26:36.685773+0000 mgr.y (mgr.24473) 211 : cluster [DBG] pgmap v156: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:38.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:37 smithi161 bash[19769]: cluster 2024-02-13T03:26:36.685773+0000 mgr.y (mgr.24473) 211 : cluster [DBG] pgmap v156: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:37 smithi183 bash[20824]: cluster 2024-02-13T03:26:36.685773+0000 mgr.y (mgr.24473) 211 : cluster [DBG] pgmap v156: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:39.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:26:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:26:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:26:40.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:39 smithi161 bash[27879]: cluster 2024-02-13T03:26:38.686281+0000 mgr.y (mgr.24473) 212 : cluster [DBG] pgmap v157: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:40.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:39 smithi161 bash[19769]: cluster 2024-02-13T03:26:38.686281+0000 mgr.y (mgr.24473) 212 : cluster [DBG] pgmap v157: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:40.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:39 smithi183 bash[20824]: cluster 2024-02-13T03:26:38.686281+0000 mgr.y (mgr.24473) 212 : cluster [DBG] pgmap v157: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:26:41.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:26:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:26:41] "GET /metrics HTTP/1.1" 200 172190 "" "Prometheus/2.33.4" 2024-02-13T03:26:42.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:41 smithi161 bash[19769]: cluster 2024-02-13T03:26:40.687483+0000 mgr.y (mgr.24473) 213 : cluster [DBG] pgmap v158: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:42.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:41 smithi161 bash[27879]: cluster 2024-02-13T03:26:40.687483+0000 mgr.y (mgr.24473) 213 : cluster [DBG] pgmap v158: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:42.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:41 smithi183 bash[20824]: cluster 2024-02-13T03:26:40.687483+0000 mgr.y (mgr.24473) 213 : cluster [DBG] pgmap v158: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:43 smithi161 bash[27879]: cluster 2024-02-13T03:26:42.688057+0000 mgr.y (mgr.24473) 214 : cluster [DBG] pgmap v159: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T03:26:43.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:43 smithi161 bash[19769]: cluster 2024-02-13T03:26:42.688057+0000 mgr.y (mgr.24473) 214 : cluster [DBG] pgmap v159: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T03:26:43.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:43 smithi183 bash[20824]: cluster 2024-02-13T03:26:42.688057+0000 mgr.y (mgr.24473) 214 : cluster [DBG] pgmap v159: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T03:26:45.023 INFO:tasks.workunit.client.0.smithi161.stderr:Note: checking out '84e714279aa684636ac645daf0a1a85c2094cfb3'. 2024-02-13T03:26:45.023 INFO:tasks.workunit.client.0.smithi161.stderr: 2024-02-13T03:26:45.023 INFO:tasks.workunit.client.0.smithi161.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-13T03:26:45.023 INFO:tasks.workunit.client.0.smithi161.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-13T03:26:45.023 INFO:tasks.workunit.client.0.smithi161.stderr:state without impacting any branches by performing another checkout. 2024-02-13T03:26:45.023 INFO:tasks.workunit.client.0.smithi161.stderr: 2024-02-13T03:26:45.023 INFO:tasks.workunit.client.0.smithi161.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-13T03:26:45.023 INFO:tasks.workunit.client.0.smithi161.stderr:do so (now or later) by using -b with the checkout command again. Example: 2024-02-13T03:26:45.023 INFO:tasks.workunit.client.0.smithi161.stderr: 2024-02-13T03:26:45.024 INFO:tasks.workunit.client.0.smithi161.stderr: git checkout -b 2024-02-13T03:26:45.024 INFO:tasks.workunit.client.0.smithi161.stderr: 2024-02-13T03:26:45.024 INFO:tasks.workunit.client.0.smithi161.stderr:HEAD is now at 84e714279aa qa/suites/orch: whitelist warnings that are expected in test environments 2024-02-13T03:26:45.028 DEBUG:teuthology.orchestra.run.smithi161:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-13T03:26:45.088 INFO:tasks.workunit.client.0.smithi161.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-13T03:26:45.091 INFO:tasks.workunit.client.0.smithi161.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-13T03:26:45.091 INFO:tasks.workunit.client.0.smithi161.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-13T03:26:45.826 INFO:tasks.workunit.client.0.smithi161.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-13T03:26:45.950 INFO:tasks.workunit.client.0.smithi161.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-13T03:26:46.006 INFO:tasks.workunit.client.0.smithi161.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-13T03:26:46.008 INFO:tasks.workunit.client.0.smithi161.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-13T03:26:46.008 INFO:tasks.workunit.client.0.smithi161.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-13T03:26:46.069 INFO:tasks.workunit.client.0.smithi161.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-13T03:26:46.074 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-02-13T03:26:46.074 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-13T03:26:46.121 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-02-13T03:26:46.122 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-02-13T03:26:46.123 DEBUG:teuthology.orchestra.run.smithi161:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=84e714279aa684636ac645daf0a1a85c2094cfb3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-02-13T03:26:46.176 INFO:tasks.workunit.client.0.smithi161.stderr:+ parallel=1 2024-02-13T03:26:46.176 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' '' = --serial ']' 2024-02-13T03:26:46.176 INFO:tasks.workunit.client.0.smithi161.stderr:+ color= 2024-02-13T03:26:46.177 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' -t 1 ']' 2024-02-13T03:26:46.177 INFO:tasks.workunit.client.0.smithi161.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-02-13T03:26:46.177 INFO:tasks.workunit.client.0.smithi161.stderr:+ declare -A pids 2024-02-13T03:26:46.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:45 smithi161 bash[27879]: cluster 2024-02-13T03:26:44.688667+0000 mgr.y (mgr.24473) 215 : cluster [DBG] pgmap v160: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T03:26:46.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:45 smithi161 bash[19769]: cluster 2024-02-13T03:26:44.688667+0000 mgr.y (mgr.24473) 215 : cluster [DBG] pgmap v160: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T03:26:46.178 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.178 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.178 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_aio 2024-02-13T03:26:46.178 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_aio' 2024-02-13T03:26:46.178 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_aio 2024-02-13T03:26:46.179 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.182 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_aio 2024-02-13T03:26:46.182 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61714 2024-02-13T03:26:46.182 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_aio on pid 61714' 2024-02-13T03:26:46.182 INFO:tasks.workunit.client.0.smithi161.stdout:test api_aio on pid 61714 2024-02-13T03:26:46.183 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61714 2024-02-13T03:26:46.183 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.183 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.183 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-02-13T03:26:46.183 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_aio_pp 2024-02-13T03:26:46.183 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_aio_pp' 2024-02-13T03:26:46.184 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_aio_pp 2024-02-13T03:26:46.184 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.185 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_aio 2024-02-13T03:26:46.185 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_aio.log 2024-02-13T03:26:46.185 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_aio: /' 2024-02-13T03:26:46.188 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_aio_pp 2024-02-13T03:26:46.188 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61722 2024-02-13T03:26:46.188 INFO:tasks.workunit.client.0.smithi161.stdout:test api_aio_pp on pid 61722 2024-02-13T03:26:46.189 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_aio_pp on pid 61722' 2024-02-13T03:26:46.189 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61722 2024-02-13T03:26:46.189 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-02-13T03:26:46.189 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.189 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.189 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_io 2024-02-13T03:26:46.189 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_io' 2024-02-13T03:26:46.190 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_io 2024-02-13T03:26:46.190 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_aio_pp 2024-02-13T03:26:46.190 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.190 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-02-13T03:26:46.191 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_aio_pp: /' 2024-02-13T03:26:46.193 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_io 2024-02-13T03:26:46.193 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61730 2024-02-13T03:26:46.193 INFO:tasks.workunit.client.0.smithi161.stdout:test api_io on pid 61730 2024-02-13T03:26:46.194 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_io on pid 61730' 2024-02-13T03:26:46.194 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61730 2024-02-13T03:26:46.194 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.194 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.194 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-02-13T03:26:46.194 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_io_pp 2024-02-13T03:26:46.194 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_io_pp' 2024-02-13T03:26:46.195 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_io_pp 2024-02-13T03:26:46.195 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.195 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_io 2024-02-13T03:26:46.195 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_io.log 2024-02-13T03:26:46.195 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_io: /' 2024-02-13T03:26:46.197 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_io_pp 2024-02-13T03:26:46.197 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61738 2024-02-13T03:26:46.198 INFO:tasks.workunit.client.0.smithi161.stdout:test api_io_pp on pid 61738 2024-02-13T03:26:46.198 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_io_pp on pid 61738' 2024-02-13T03:26:46.198 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61738 2024-02-13T03:26:46.198 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.199 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.199 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-02-13T03:26:46.199 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_asio 2024-02-13T03:26:46.199 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_asio' 2024-02-13T03:26:46.200 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.200 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_asio 2024-02-13T03:26:46.200 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_io_pp 2024-02-13T03:26:46.200 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-02-13T03:26:46.201 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_io_pp: /' 2024-02-13T03:26:46.202 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_asio 2024-02-13T03:26:46.202 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61746 2024-02-13T03:26:46.202 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_asio on pid 61746' 2024-02-13T03:26:46.203 INFO:tasks.workunit.client.0.smithi161.stdout:test api_asio on pid 61746 2024-02-13T03:26:46.204 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-02-13T03:26:46.204 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61746 2024-02-13T03:26:46.204 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.204 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.204 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_list 2024-02-13T03:26:46.204 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_list' 2024-02-13T03:26:46.204 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_asio 2024-02-13T03:26:46.204 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_list 2024-02-13T03:26:46.204 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_asio.log 2024-02-13T03:26:46.205 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_asio: /' 2024-02-13T03:26:46.205 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.206 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_list 2024-02-13T03:26:46.206 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61754 2024-02-13T03:26:46.206 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_list on pid 61754' 2024-02-13T03:26:46.207 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61754 2024-02-13T03:26:46.207 INFO:tasks.workunit.client.0.smithi161.stdout:test api_list on pid 61754 2024-02-13T03:26:46.207 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.207 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.207 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-02-13T03:26:46.208 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_lock 2024-02-13T03:26:46.208 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_lock' 2024-02-13T03:26:46.208 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.208 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_list 2024-02-13T03:26:46.209 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_lock 2024-02-13T03:26:46.209 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_list.log 2024-02-13T03:26:46.209 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_list: /' 2024-02-13T03:26:46.211 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_lock 2024-02-13T03:26:46.211 INFO:tasks.workunit.client.0.smithi161.stdout:test api_lock on pid 61762 2024-02-13T03:26:46.211 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61762 2024-02-13T03:26:46.212 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_lock on pid 61762' 2024-02-13T03:26:46.212 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61762 2024-02-13T03:26:46.212 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.212 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.212 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-02-13T03:26:46.212 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_lock_pp 2024-02-13T03:26:46.212 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_lock_pp' 2024-02-13T03:26:46.214 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_lock: /' 2024-02-13T03:26:46.214 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_lock 2024-02-13T03:26:46.214 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_lock.log 2024-02-13T03:26:46.220 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.220 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_lock_pp 2024-02-13T03:26:46.223 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_lock_pp 2024-02-13T03:26:46.223 INFO:tasks.workunit.client.0.smithi161.stdout:test api_lock_pp on pid 61778 2024-02-13T03:26:46.223 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61778 2024-02-13T03:26:46.223 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_lock_pp on pid 61778' 2024-02-13T03:26:46.223 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61778 2024-02-13T03:26:46.224 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.224 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.224 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-02-13T03:26:46.230 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_misc 2024-02-13T03:26:46.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_misc' 2024-02-13T03:26:46.231 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_lock_pp: /' 2024-02-13T03:26:46.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-02-13T03:26:46.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_lock_pp 2024-02-13T03:26:46.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:45 smithi183 bash[20824]: cluster 2024-02-13T03:26:44.688667+0000 mgr.y (mgr.24473) 215 : cluster [DBG] pgmap v160: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T03:26:46.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:26:45 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:26:46.238 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_misc 2024-02-13T03:26:46.238 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_misc 2024-02-13T03:26:46.239 INFO:tasks.workunit.client.0.smithi161.stdout:test api_misc on pid 61839 2024-02-13T03:26:46.239 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61839 2024-02-13T03:26:46.239 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_misc on pid 61839' 2024-02-13T03:26:46.239 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61839 2024-02-13T03:26:46.239 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.239 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.243 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-02-13T03:26:46.247 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_misc_pp 2024-02-13T03:26:46.248 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_misc_pp' 2024-02-13T03:26:46.248 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_misc 2024-02-13T03:26:46.248 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_misc: /' 2024-02-13T03:26:46.251 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_misc.log 2024-02-13T03:26:46.255 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_misc_pp 2024-02-13T03:26:46.255 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.257 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_misc_pp 2024-02-13T03:26:46.258 INFO:tasks.workunit.client.0.smithi161.stdout:test api_misc_pp on pid 61886 2024-02-13T03:26:46.258 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61886 2024-02-13T03:26:46.258 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_misc_pp on pid 61886' 2024-02-13T03:26:46.258 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61886 2024-02-13T03:26:46.258 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.258 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.258 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-02-13T03:26:46.259 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_tier_pp 2024-02-13T03:26:46.259 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_tier_pp' 2024-02-13T03:26:46.259 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.259 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_tier_pp 2024-02-13T03:26:46.261 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_misc_pp: /' 2024-02-13T03:26:46.261 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_tier_pp 2024-02-13T03:26:46.261 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61898 2024-02-13T03:26:46.261 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_tier_pp on pid 61898' 2024-02-13T03:26:46.262 INFO:tasks.workunit.client.0.smithi161.stdout:test api_tier_pp on pid 61898 2024-02-13T03:26:46.262 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61898 2024-02-13T03:26:46.262 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.262 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.262 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-02-13T03:26:46.263 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_misc_pp 2024-02-13T03:26:46.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-02-13T03:26:46.270 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_pool 2024-02-13T03:26:46.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_pool' 2024-02-13T03:26:46.271 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_tier_pp 2024-02-13T03:26:46.272 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_pool 2024-02-13T03:26:46.272 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.274 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_pool 2024-02-13T03:26:46.274 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61916 2024-02-13T03:26:46.274 INFO:tasks.workunit.client.0.smithi161.stdout:test api_pool on pid 61916 2024-02-13T03:26:46.275 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_pool on pid 61916' 2024-02-13T03:26:46.275 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61916 2024-02-13T03:26:46.275 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.275 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.276 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_snapshots 2024-02-13T03:26:46.276 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_snapshots' 2024-02-13T03:26:46.276 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-02-13T03:26:46.276 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.276 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-02-13T03:26:46.278 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_pool: /' 2024-02-13T03:26:46.279 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_tier_pp: /' 2024-02-13T03:26:46.279 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_snapshots 2024-02-13T03:26:46.280 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_snapshots 2024-02-13T03:26:46.280 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61928 2024-02-13T03:26:46.280 INFO:tasks.workunit.client.0.smithi161.stdout:test api_snapshots on pid 61928 2024-02-13T03:26:46.281 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_snapshots on pid 61928' 2024-02-13T03:26:46.281 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61928 2024-02-13T03:26:46.281 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.281 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.281 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_pool 2024-02-13T03:26:46.283 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_pool.log 2024-02-13T03:26:46.285 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-02-13T03:26:46.285 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_snapshots_pp 2024-02-13T03:26:46.285 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_snapshots_pp' 2024-02-13T03:26:46.292 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-02-13T03:26:46.295 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_snapshots 2024-02-13T03:26:46.299 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_snapshots_pp 2024-02-13T03:26:46.299 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_snapshots: /' 2024-02-13T03:26:46.302 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.307 INFO:tasks.workunit.client.0.smithi161.stdout:test api_snapshots_pp on pid 61964 2024-02-13T03:26:46.307 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_snapshots_pp 2024-02-13T03:26:46.307 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61964 2024-02-13T03:26:46.307 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_snapshots_pp on pid 61964' 2024-02-13T03:26:46.307 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61964 2024-02-13T03:26:46.307 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.308 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.315 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-02-13T03:26:46.315 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_stat 2024-02-13T03:26:46.315 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_stat' 2024-02-13T03:26:46.315 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.319 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_stat 2024-02-13T03:26:46.320 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_snapshots_pp 2024-02-13T03:26:46.320 INFO:tasks.workunit.client.0.smithi161.stdout:test api_stat on pid 61996 2024-02-13T03:26:46.321 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_stat 2024-02-13T03:26:46.321 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61996 2024-02-13T03:26:46.321 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_stat on pid 61996' 2024-02-13T03:26:46.321 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=61996 2024-02-13T03:26:46.321 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.322 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.322 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_stat_pp 2024-02-13T03:26:46.322 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_stat_pp' 2024-02-13T03:26:46.322 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-02-13T03:26:46.322 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-02-13T03:26:46.322 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_stat_pp 2024-02-13T03:26:46.322 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-02-13T03:26:46.322 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_stat: /' 2024-02-13T03:26:46.322 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_stat.log 2024-02-13T03:26:46.322 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_stat 2024-02-13T03:26:46.322 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.325 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_stat_pp 2024-02-13T03:26:46.325 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62020 2024-02-13T03:26:46.326 INFO:tasks.workunit.client.0.smithi161.stdout:test api_stat_pp on pid 62020 2024-02-13T03:26:46.326 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_stat_pp on pid 62020' 2024-02-13T03:26:46.326 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=62020 2024-02-13T03:26:46.326 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.326 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.331 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-02-13T03:26:46.331 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_watch_notify 2024-02-13T03:26:46.331 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_watch_notify' 2024-02-13T03:26:46.332 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-02-13T03:26:46.332 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_stat_pp: /' 2024-02-13T03:26:46.333 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_stat_pp 2024-02-13T03:26:46.338 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_watch_notify 2024-02-13T03:26:46.338 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.347 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_watch_notify 2024-02-13T03:26:46.347 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62051 2024-02-13T03:26:46.347 INFO:tasks.workunit.client.0.smithi161.stdout:test api_watch_notify on pid 62051 2024-02-13T03:26:46.348 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_watch_notify on pid 62051' 2024-02-13T03:26:46.348 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=62051 2024-02-13T03:26:46.348 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.348 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.349 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-02-13T03:26:46.350 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_watch_notify_pp 2024-02-13T03:26:46.350 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_watch_notify_pp' 2024-02-13T03:26:46.350 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_watch_notify: /' 2024-02-13T03:26:46.350 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.351 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_watch_notify_pp 2024-02-13T03:26:46.352 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-02-13T03:26:46.352 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_watch_notify 2024-02-13T03:26:46.354 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_watch_notify_pp 2024-02-13T03:26:46.354 INFO:tasks.workunit.client.0.smithi161.stdout:test api_watch_notify_pp on pid 62081 2024-02-13T03:26:46.355 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62081 2024-02-13T03:26:46.355 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_watch_notify_pp on pid 62081' 2024-02-13T03:26:46.355 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=62081 2024-02-13T03:26:46.355 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.355 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.355 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-02-13T03:26:46.357 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-02-13T03:26:46.357 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-02-13T03:26:46.357 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-02-13T03:26:46.359 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_cmd 2024-02-13T03:26:46.359 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_cmd' 2024-02-13T03:26:46.365 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.367 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_cmd 2024-02-13T03:26:46.368 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_cmd 2024-02-13T03:26:46.369 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62118 2024-02-13T03:26:46.369 INFO:tasks.workunit.client.0.smithi161.stdout:test api_cmd on pid 62118 2024-02-13T03:26:46.370 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_cmd on pid 62118' 2024-02-13T03:26:46.370 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=62118 2024-02-13T03:26:46.370 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.370 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.370 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_cmd_pp 2024-02-13T03:26:46.370 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_cmd_pp' 2024-02-13T03:26:46.371 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-02-13T03:26:46.371 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.371 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_cmd_pp 2024-02-13T03:26:46.372 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_cmd_pp 2024-02-13T03:26:46.372 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62123 2024-02-13T03:26:46.372 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_cmd_pp on pid 62123' 2024-02-13T03:26:46.373 INFO:tasks.workunit.client.0.smithi161.stdout:test api_cmd_pp on pid 62123 2024-02-13T03:26:46.374 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=62123 2024-02-13T03:26:46.374 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.374 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.374 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-02-13T03:26:46.374 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_cmd 2024-02-13T03:26:46.374 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_service 2024-02-13T03:26:46.374 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_cmd.log 2024-02-13T03:26:46.375 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_service' 2024-02-13T03:26:46.375 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_cmd: /' 2024-02-13T03:26:46.375 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.375 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_service 2024-02-13T03:26:46.375 INFO:tasks.workunit.client.0.smithi161.stdout:test api_service on pid 62138 2024-02-13T03:26:46.376 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_service 2024-02-13T03:26:46.376 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62138 2024-02-13T03:26:46.376 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_service on pid 62138' 2024-02-13T03:26:46.376 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=62138 2024-02-13T03:26:46.376 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.376 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.376 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_service_pp 2024-02-13T03:26:46.377 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_service_pp' 2024-02-13T03:26:46.377 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-02-13T03:26:46.377 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-02-13T03:26:46.377 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_cmd_pp 2024-02-13T03:26:46.379 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-02-13T03:26:46.379 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_service_pp 2024-02-13T03:26:46.379 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_service 2024-02-13T03:26:46.383 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.387 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_service: /' 2024-02-13T03:26:46.387 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_service.log 2024-02-13T03:26:46.389 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_service_pp 2024-02-13T03:26:46.390 INFO:tasks.workunit.client.0.smithi161.stdout:test api_service_pp on pid 62158 2024-02-13T03:26:46.390 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62158 2024-02-13T03:26:46.390 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_service_pp on pid 62158' 2024-02-13T03:26:46.390 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=62158 2024-02-13T03:26:46.391 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.391 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.391 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-02-13T03:26:46.391 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_c_write_operations 2024-02-13T03:26:46.391 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_c_write_operations' 2024-02-13T03:26:46.392 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_c_write_operations 2024-02-13T03:26:46.392 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.392 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_service_pp 2024-02-13T03:26:46.392 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_service_pp: /' 2024-02-13T03:26:46.395 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-02-13T03:26:46.405 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_c_write_operations 2024-02-13T03:26:46.405 INFO:tasks.workunit.client.0.smithi161.stdout:test api_c_write_operations on pid 62179 2024-02-13T03:26:46.405 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62179 2024-02-13T03:26:46.405 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_c_write_operations on pid 62179' 2024-02-13T03:26:46.405 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=62179 2024-02-13T03:26:46.406 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.406 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.406 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s api_c_read_operations 2024-02-13T03:26:46.407 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' api_c_read_operations' 2024-02-13T03:26:46.407 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-02-13T03:26:46.408 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_c_write_operations 2024-02-13T03:26:46.408 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-02-13T03:26:46.408 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-02-13T03:26:46.409 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.411 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo api_c_read_operations 2024-02-13T03:26:46.413 INFO:tasks.workunit.client.0.smithi161.stdout:test api_c_read_operations on pid 62207 2024-02-13T03:26:46.413 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=api_c_read_operations 2024-02-13T03:26:46.413 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62207 2024-02-13T03:26:46.413 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test api_c_read_operations on pid 62207' 2024-02-13T03:26:46.413 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=62207 2024-02-13T03:26:46.413 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.413 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.419 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-02-13T03:26:46.421 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_api_c_read_operations 2024-02-13T03:26:46.421 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-02-13T03:26:46.422 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-02-13T03:26:46.423 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s list_parallel 2024-02-13T03:26:46.423 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' list_parallel' 2024-02-13T03:26:46.428 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.428 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo list_parallel 2024-02-13T03:26:46.429 INFO:tasks.workunit.client.0.smithi161.stdout:test list_parallel on pid 62244 2024-02-13T03:26:46.430 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=list_parallel 2024-02-13T03:26:46.430 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62244 2024-02-13T03:26:46.430 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test list_parallel on pid 62244' 2024-02-13T03:26:46.430 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=62244 2024-02-13T03:26:46.430 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.430 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.430 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-02-13T03:26:46.435 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_list_parallel 2024-02-13T03:26:46.436 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ list_parallel: /' 2024-02-13T03:26:46.439 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s open_pools_parallel 2024-02-13T03:26:46.439 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' open_pools_parallel' 2024-02-13T03:26:46.440 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.440 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo open_pools_parallel 2024-02-13T03:26:46.442 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=open_pools_parallel 2024-02-13T03:26:46.442 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62273 2024-02-13T03:26:46.442 INFO:tasks.workunit.client.0.smithi161.stdout:test open_pools_parallel on pid 62273 2024-02-13T03:26:46.443 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test open_pools_parallel on pid 62273' 2024-02-13T03:26:46.443 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=62273 2024-02-13T03:26:46.443 INFO:tasks.workunit.client.0.smithi161.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-13T03:26:46.443 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.443 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-02-13T03:26:46.444 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_list_parallel.log 2024-02-13T03:26:46.444 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_open_pools_parallel 2024-02-13T03:26:46.445 INFO:tasks.workunit.client.0.smithi161.stderr:++ printf %25s delete_pools_parallel 2024-02-13T03:26:46.445 INFO:tasks.workunit.client.0.smithi161.stderr:+ r=' delete_pools_parallel' 2024-02-13T03:26:46.445 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-02-13T03:26:46.445 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-02-13T03:26:46.454 INFO:tasks.workunit.client.0.smithi161.stderr:++ echo delete_pools_parallel 2024-02-13T03:26:46.463 INFO:tasks.workunit.client.0.smithi161.stderr:++ awk '{print $1}' 2024-02-13T03:26:46.465 INFO:tasks.workunit.client.0.smithi161.stderr:+ ff=delete_pools_parallel 2024-02-13T03:26:46.465 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62308 2024-02-13T03:26:46.465 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo 'test delete_pools_parallel on pid 62308' 2024-02-13T03:26:46.465 INFO:tasks.workunit.client.0.smithi161.stdout:test delete_pools_parallel on pid 62308 2024-02-13T03:26:46.466 INFO:tasks.workunit.client.0.smithi161.stderr:+ pids[$f]=62308 2024-02-13T03:26:46.466 INFO:tasks.workunit.client.0.smithi161.stderr:+ ret=0 2024-02-13T03:26:46.466 INFO:tasks.workunit.client.0.smithi161.stderr:+ '[' 1 -eq 1 ']' 2024-02-13T03:26:46.466 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:26:46.466 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61928 2024-02-13T03:26:46.466 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61928 2024-02-13T03:26:46.467 INFO:tasks.workunit.client.0.smithi161.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-02-13T03:26:46.473 INFO:tasks.workunit.client.0.smithi161.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-02-13T03:26:46.475 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph_test_rados_delete_pools_parallel 2024-02-13T03:26:46.475 INFO:tasks.workunit.client.0.smithi161.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-02-13T03:26:47.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:47 smithi161 bash[27879]: audit 2024-02-13T03:26:46.420367+0000 mon.b (mon.1) 26 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:47 smithi161 bash[27879]: audit 2024-02-13T03:26:46.442446+0000 mon.c (mon.2) 75 : audit [DBG] from='client.? v1:172.21.15.161:0/541183560' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-13T03:26:47.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:47 smithi161 bash[19769]: audit 2024-02-13T03:26:46.420367+0000 mon.b (mon.1) 26 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:47.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:47 smithi161 bash[19769]: audit 2024-02-13T03:26:46.442446+0000 mon.c (mon.2) 75 : audit [DBG] from='client.? v1:172.21.15.161:0/541183560' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-13T03:26:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:47 smithi183 bash[20824]: audit 2024-02-13T03:26:46.420367+0000 mon.b (mon.1) 26 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:47 smithi183 bash[20824]: audit 2024-02-13T03:26:46.442446+0000 mon.c (mon.2) 75 : audit [DBG] from='client.? v1:172.21.15.161:0/541183560' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-13T03:26:49.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:26:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:26:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:26:51.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:26:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:26:51] "GET /metrics HTTP/1.1" 200 177028 "" "Prometheus/2.33.4" 2024-02-13T03:26:52.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:52 smithi161 bash[27879]: audit 2024-02-13T03:26:45.789715+0000 mgr.y (mgr.24473) 216 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:52 smithi161 bash[27879]: audit 2024-02-13T03:26:47.143081+0000 mon.a (mon.0) 694 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:52 smithi161 bash[19769]: audit 2024-02-13T03:26:45.789715+0000 mgr.y (mgr.24473) 216 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:52 smithi161 bash[19769]: audit 2024-02-13T03:26:47.143081+0000 mon.a (mon.0) 694 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:52.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:52 smithi183 bash[20824]: audit 2024-02-13T03:26:45.789715+0000 mgr.y (mgr.24473) 216 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:52.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:52 smithi183 bash[20824]: audit 2024-02-13T03:26:47.143081+0000 mon.a (mon.0) 694 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:56.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:26:55 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:26:56.736 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [----------] Global test environment set-up. 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [----------] 12 tests from AsioRados 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (3 ms) 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (0 ms) 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (1 ms) 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (1 ms) 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (1 ms) 2024-02-13T03:26:56.737 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (0 ms) 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (0 ms) 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (0 ms) 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (1 ms) 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (1 ms) 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (1 ms) 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [----------] 12 tests from AsioRados (11 ms total) 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: 2024-02-13T03:26:56.738 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [----------] Global test environment tear-down 2024-02-13T03:26:56.739 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (10510 ms total) 2024-02-13T03:26:56.739 INFO:tasks.workunit.client.0.smithi161.stdout: api_asio: [ PASSED ] 12 tests. 2024-02-13T03:26:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[27879]: cluster 2024-02-13T03:26:46.689877+0000 mgr.y (mgr.24473) 217 : cluster [DBG] pgmap v161: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[27879]: audit 2024-02-13T03:26:48.036599+0000 mon.b (mon.1) 27 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[27879]: cluster 2024-02-13T03:26:48.690534+0000 mgr.y (mgr.24473) 218 : cluster [DBG] pgmap v163: 196 pgs: 64 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-02-13T03:26:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[27879]: cluster 2024-02-13T03:26:50.691903+0000 mgr.y (mgr.24473) 219 : cluster [DBG] pgmap v164: 196 pgs: 44 creating+peering, 132 active+clean, 20 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:26:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[27879]: audit 2024-02-13T03:26:51.697272+0000 mon.c (mon.2) 76 : audit [DBG] from='client.? v1:172.21.15.161:0/2936944751' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-13T03:26:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[27879]: cluster 2024-02-13T03:26:52.298626+0000 mon.a (mon.0) 695 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-13T03:26:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[27879]: audit 2024-02-13T03:26:52.311427+0000 mon.a (mon.0) 696 : audit [INF] from='client.? v1:172.21.15.161:0/259716468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi161-61726-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[27879]: audit 2024-02-13T03:26:52.311971+0000 mon.a (mon.0) 697 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:56.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[19769]: cluster 2024-02-13T03:26:46.689877+0000 mgr.y (mgr.24473) 217 : cluster [DBG] pgmap v161: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:56.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[19769]: audit 2024-02-13T03:26:48.036599+0000 mon.b (mon.1) 27 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:56.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[19769]: cluster 2024-02-13T03:26:48.690534+0000 mgr.y (mgr.24473) 218 : cluster [DBG] pgmap v163: 196 pgs: 64 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-02-13T03:26:56.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[19769]: cluster 2024-02-13T03:26:50.691903+0000 mgr.y (mgr.24473) 219 : cluster [DBG] pgmap v164: 196 pgs: 44 creating+peering, 132 active+clean, 20 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:26:56.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[19769]: audit 2024-02-13T03:26:51.697272+0000 mon.c (mon.2) 76 : audit [DBG] from='client.? v1:172.21.15.161:0/2936944751' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-13T03:26:56.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[19769]: cluster 2024-02-13T03:26:52.298626+0000 mon.a (mon.0) 695 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-13T03:26:56.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[19769]: audit 2024-02-13T03:26:52.311427+0000 mon.a (mon.0) 696 : audit [INF] from='client.? v1:172.21.15.161:0/259716468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi161-61726-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:56.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:56 smithi161 bash[19769]: audit 2024-02-13T03:26:52.311971+0000 mon.a (mon.0) 697 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:56 smithi183 bash[20824]: cluster 2024-02-13T03:26:46.689877+0000 mgr.y (mgr.24473) 217 : cluster [DBG] pgmap v161: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:56 smithi183 bash[20824]: audit 2024-02-13T03:26:48.036599+0000 mon.b (mon.1) 27 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:56 smithi183 bash[20824]: cluster 2024-02-13T03:26:48.690534+0000 mgr.y (mgr.24473) 218 : cluster [DBG] pgmap v163: 196 pgs: 64 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-02-13T03:26:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:56 smithi183 bash[20824]: cluster 2024-02-13T03:26:50.691903+0000 mgr.y (mgr.24473) 219 : cluster [DBG] pgmap v164: 196 pgs: 44 creating+peering, 132 active+clean, 20 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:26:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:56 smithi183 bash[20824]: audit 2024-02-13T03:26:51.697272+0000 mon.c (mon.2) 76 : audit [DBG] from='client.? v1:172.21.15.161:0/2936944751' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-13T03:26:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:56 smithi183 bash[20824]: cluster 2024-02-13T03:26:52.298626+0000 mon.a (mon.0) 695 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-13T03:26:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:56 smithi183 bash[20824]: audit 2024-02-13T03:26:52.311427+0000 mon.a (mon.0) 696 : audit [INF] from='client.? v1:172.21.15.161:0/259716468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi161-61726-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:56 smithi183 bash[20824]: audit 2024-02-13T03:26:52.311971+0000 mon.a (mon.0) 697 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:57.815 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-02-13T03:26:57.815 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [----------] Global test environment set-up. 2024-02-13T03:26:57.815 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [----------] 4 tests from LibRadosService 2024-02-13T03:26:57.815 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-02-13T03:26:57.816 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5034 ms) 2024-02-13T03:26:57.816 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-02-13T03:26:57.816 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [ OK ] LibRadosService.RegisterLate (42 ms) 2024-02-13T03:26:57.816 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-02-13T03:26:57.816 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: cluster: 2024-02-13T03:26:57.816 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: id: b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:26:57.816 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: health: HEALTH_OK 2024-02-13T03:26:57.816 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.816 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: services: 2024-02-13T03:26:57.816 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 11m) 2024-02-13T03:26:57.816 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: mgr: y(active, since 5m), standbys: x 2024-02-13T03:26:57.816 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: osd: 8 osds: 8 up (since 5m), 8 in (since 6m) 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: data: 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: objects: 224 objects, 5.7 KiB 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: pgs: 132 active+clean 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: io: 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: client: 767 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: cluster: 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: id: b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:26:57.817 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: health: HEALTH_OK 2024-02-13T03:26:57.818 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.818 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: services: 2024-02-13T03:26:57.818 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 11m) 2024-02-13T03:26:57.818 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: mgr: y(active, since 5m), standbys: x 2024-02-13T03:26:57.818 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: osd: 8 osds: 8 up (since 5m), 8 in (since 6m) 2024-02-13T03:26:57.818 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-02-13T03:26:57.818 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-13T03:26:57.818 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.818 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: data: 2024-02-13T03:26:57.818 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-13T03:26:57.818 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: objects: 224 objects, 5.7 KiB 2024-02-13T03:26:57.818 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:26:57.818 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: pgs: 132 active+clean 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: io: 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: cluster: 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: id: b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: health: HEALTH_OK 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: services: 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 11m) 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: mgr: y(active, since 5m), standbys: x 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: osd: 8 osds: 8 up (since 6m), 8 in (since 6m) 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-13T03:26:57.819 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.820 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: data: 2024-02-13T03:26:57.820 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-13T03:26:57.820 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: objects: 224 objects, 5.7 KiB 2024-02-13T03:26:57.820 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:26:57.820 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: pgs: 132 active+clean 2024-02-13T03:26:57.820 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.820 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: io: 2024-02-13T03:26:57.820 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-02-13T03:26:57.820 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.820 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.820 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: cluster: 2024-02-13T03:26:57.820 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: id: b9b3ef98-ca1d-11ee-95b9-87774f69a715 2024-02-13T03:26:57.820 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: health: HEALTH_OK 2024-02-13T03:26:57.821 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:26:57.821 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: services: 2024-02-13T03:26:57.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:52.324721+0000 mon.c (mon.2) 77 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi161-61759-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:52.325145+0000 mon.c (mon.2) 78 : audit [INF] from='client.? v1:172.21.15.161:0/1175961112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi161-61765-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:52.325511+0000 mon.c (mon.2) 79 : audit [INF] from='client.? v1:172.21.15.161:0/392796170' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi161-61798-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:52.325954+0000 mon.c (mon.2) 80 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:52.326382+0000 mon.c (mon.2) 81 : audit [INF] from='client.? v1:172.21.15.161:0/3042168812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi161-61906-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:52.326941+0000 mon.c (mon.2) 82 : audit [INF] from='client.? v1:172.21.15.161:0/1502259945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi161-61936-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:52.327420+0000 mon.c (mon.2) 83 : audit [INF] from='client.? v1:172.21.15.161:0/1913315211' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi161-61735-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:52.327705+0000 mon.c (mon.2) 84 : audit [INF] from='client.? v1:172.21.15.161:0/606117760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi161-61743-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:52.328612+0000 mon.b (mon.1) 28 : audit [INF] from='client.? v1:172.21.15.161:0/3651106834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi161-62225-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:52.328987+0000 mon.b (mon.1) 29 : audit [INF] from='client.? v1:172.21.15.161:0/56578333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi161-61719-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:52.329329+0000 mon.b (mon.1) 30 : audit [INF] from='client.? v1:172.21.15.161:0/2078631688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: cluster 2024-02-13T03:26:52.693015+0000 mgr.y (mgr.24473) 220 : cluster [DBG] pgmap v166: 580 pgs: 44 creating+peering, 132 active+clean, 404 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:53.721507+0000 mon.c (mon.2) 85 : audit [DBG] from='client.? v1:172.21.15.161:0/811745849' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: cluster 2024-02-13T03:26:54.694050+0000 mgr.y (mgr.24473) 221 : cluster [DBG] pgmap v167: 580 pgs: 44 creating+peering, 132 active+clean, 404 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:55.747385+0000 mon.c (mon.2) 86 : audit [DBG] from='client.? v1:172.21.15.161:0/352888529' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:55.801000+0000 mgr.y (mgr.24473) 222 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: cluster 2024-02-13T03:26:56.599803+0000 mon.a (mon.0) 698 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.604352+0000 mon.b (mon.1) 31 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.605975+0000 mon.a (mon.0) 699 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi161-61759-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.606220+0000 mon.a (mon.0) 700 : audit [INF] from='client.24838 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi161-62225-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.606476+0000 mon.a (mon.0) 701 : audit [INF] from='client.24728 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi161-61765-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.606621+0000 mon.a (mon.0) 702 : audit [INF] from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi161-61719-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.606774+0000 mon.a (mon.0) 703 : audit [INF] from='client.24734 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi161-61798-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.607159+0000 mon.a (mon.0) 704 : audit [INF] from='client.24826 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.607290+0000 mon.a (mon.0) 705 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.607478+0000 mon.a (mon.0) 706 : audit [INF] from='client.24752 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi161-61906-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.607657+0000 mon.a (mon.0) 707 : audit [INF] from='client.24758 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi161-61936-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.607833+0000 mon.a (mon.0) 708 : audit [INF] from='client.24710 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi161-61735-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.615544+0000 mon.a (mon.0) 709 : audit [INF] from='client.? v1:172.21.15.161:0/259716468' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi161-61726-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: cluster 2024-02-13T03:26:56.617905+0000 mon.a (mon.0) 710 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-13T03:26:57.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.619863+0000 mon.a (mon.0) 711 : audit [INF] from='client.24716 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi161-61743-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.622156+0000 mon.c (mon.2) 87 : audit [INF] from='client.? v1:172.21.15.161:0/1333079462' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi161-62012-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.622755+0000 mon.c (mon.2) 88 : audit [INF] from='client.? v1:172.21.15.161:0/1708383590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi161-62063-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.623035+0000 mon.c (mon.2) 89 : audit [INF] from='client.? v1:172.21.15.161:0/2516704546' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.623940+0000 mon.c (mon.2) 90 : audit [INF] from='client.? v1:172.21.15.161:0/3998698116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.624141+0000 mon.c (mon.2) 91 : audit [INF] from='client.? v1:172.21.15.161:0/3457363402' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.624307+0000 mon.c (mon.2) 92 : audit [INF] from='client.? v1:172.21.15.161:0/1554898749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi161-61993-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.624459+0000 mon.c (mon.2) 93 : audit [INF] from='client.? v1:172.21.15.161:0/3284003100' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi161-62041-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.624571+0000 mon.a (mon.0) 712 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:26:57.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.626766+0000 mon.a (mon.0) 713 : audit [INF] from='client.24779 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi161-62012-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.648905+0000 mon.a (mon.0) 714 : audit [INF] from='client.24788 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi161-62063-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:56.649181+0000 mon.a (mon.0) 715 : audit [INF] from='client.24764 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: cluster 2024-02-13T03:26:56.695082+0000 mgr.y (mgr.24473) 223 : cluster [DBG] pgmap v169: 900 pgs: 311 creating+peering, 132 active+clean, 457 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:57.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:57.077008+0000 mon.a (mon.0) 716 : audit [INF] from='client.24854 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:57.077760+0000 mon.a (mon.0) 717 : audit [INF] from='client.24866 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:57.078060+0000 mon.a (mon.0) 718 : audit [INF] from='client.24773 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi161-61993-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:57.078331+0000 mon.a (mon.0) 719 : audit [INF] from='client.24782 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi161-62041-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:57.628853+0000 mon.a (mon.0) 720 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi161-61759-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:57.629169+0000 mon.a (mon.0) 721 : audit [INF] from='client.24838 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi161-62225-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:57.629340+0000 mon.a (mon.0) 722 : audit [INF] from='client.24728 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi161-61765-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[19769]: audit 2024-02-13T03:26:57.629506+0000 mon.a (mon.0) 723 : audit [INF] from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi161-61719-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.934 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:52.324721+0000 mon.c (mon.2) 77 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi161-61759-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.934 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:52.325145+0000 mon.c (mon.2) 78 : audit [INF] from='client.? v1:172.21.15.161:0/1175961112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi161-61765-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.934 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:52.325511+0000 mon.c (mon.2) 79 : audit [INF] from='client.? v1:172.21.15.161:0/392796170' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi161-61798-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.934 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:52.325954+0000 mon.c (mon.2) 80 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.934 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:52.326382+0000 mon.c (mon.2) 81 : audit [INF] from='client.? v1:172.21.15.161:0/3042168812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi161-61906-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:52.326941+0000 mon.c (mon.2) 82 : audit [INF] from='client.? v1:172.21.15.161:0/1502259945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi161-61936-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:52.327420+0000 mon.c (mon.2) 83 : audit [INF] from='client.? v1:172.21.15.161:0/1913315211' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi161-61735-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:52.327705+0000 mon.c (mon.2) 84 : audit [INF] from='client.? v1:172.21.15.161:0/606117760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi161-61743-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:52.328612+0000 mon.b (mon.1) 28 : audit [INF] from='client.? v1:172.21.15.161:0/3651106834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi161-62225-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:52.328987+0000 mon.b (mon.1) 29 : audit [INF] from='client.? v1:172.21.15.161:0/56578333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi161-61719-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:52.329329+0000 mon.b (mon.1) 30 : audit [INF] from='client.? v1:172.21.15.161:0/2078631688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: cluster 2024-02-13T03:26:52.693015+0000 mgr.y (mgr.24473) 220 : cluster [DBG] pgmap v166: 580 pgs: 44 creating+peering, 132 active+clean, 404 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:53.721507+0000 mon.c (mon.2) 85 : audit [DBG] from='client.? v1:172.21.15.161:0/811745849' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: cluster 2024-02-13T03:26:54.694050+0000 mgr.y (mgr.24473) 221 : cluster [DBG] pgmap v167: 580 pgs: 44 creating+peering, 132 active+clean, 404 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:55.747385+0000 mon.c (mon.2) 86 : audit [DBG] from='client.? v1:172.21.15.161:0/352888529' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:55.801000+0000 mgr.y (mgr.24473) 222 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: cluster 2024-02-13T03:26:56.599803+0000 mon.a (mon.0) 698 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.604352+0000 mon.b (mon.1) 31 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.605975+0000 mon.a (mon.0) 699 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi161-61759-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.935 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.606220+0000 mon.a (mon.0) 700 : audit [INF] from='client.24838 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi161-62225-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.606476+0000 mon.a (mon.0) 701 : audit [INF] from='client.24728 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi161-61765-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.606621+0000 mon.a (mon.0) 702 : audit [INF] from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi161-61719-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.606774+0000 mon.a (mon.0) 703 : audit [INF] from='client.24734 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi161-61798-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.607159+0000 mon.a (mon.0) 704 : audit [INF] from='client.24826 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.607290+0000 mon.a (mon.0) 705 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.607478+0000 mon.a (mon.0) 706 : audit [INF] from='client.24752 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi161-61906-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.607657+0000 mon.a (mon.0) 707 : audit [INF] from='client.24758 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi161-61936-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.607833+0000 mon.a (mon.0) 708 : audit [INF] from='client.24710 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi161-61735-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.615544+0000 mon.a (mon.0) 709 : audit [INF] from='client.? v1:172.21.15.161:0/259716468' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi161-61726-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: cluster 2024-02-13T03:26:56.617905+0000 mon.a (mon.0) 710 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-13T03:26:57.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.619863+0000 mon.a (mon.0) 711 : audit [INF] from='client.24716 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi161-61743-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.622156+0000 mon.c (mon.2) 87 : audit [INF] from='client.? v1:172.21.15.161:0/1333079462' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi161-62012-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.936 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.622755+0000 mon.c (mon.2) 88 : audit [INF] from='client.? v1:172.21.15.161:0/1708383590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi161-62063-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.623035+0000 mon.c (mon.2) 89 : audit [INF] from='client.? v1:172.21.15.161:0/2516704546' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.623940+0000 mon.c (mon.2) 90 : audit [INF] from='client.? v1:172.21.15.161:0/3998698116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.624141+0000 mon.c (mon.2) 91 : audit [INF] from='client.? v1:172.21.15.161:0/3457363402' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.624307+0000 mon.c (mon.2) 92 : audit [INF] from='client.? v1:172.21.15.161:0/1554898749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi161-61993-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.624459+0000 mon.c (mon.2) 93 : audit [INF] from='client.? v1:172.21.15.161:0/3284003100' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi161-62041-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.624571+0000 mon.a (mon.0) 712 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.626766+0000 mon.a (mon.0) 713 : audit [INF] from='client.24779 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi161-62012-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.648905+0000 mon.a (mon.0) 714 : audit [INF] from='client.24788 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi161-62063-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:56.649181+0000 mon.a (mon.0) 715 : audit [INF] from='client.24764 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: cluster 2024-02-13T03:26:56.695082+0000 mgr.y (mgr.24473) 223 : cluster [DBG] pgmap v169: 900 pgs: 311 creating+peering, 132 active+clean, 457 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:57.077008+0000 mon.a (mon.0) 716 : audit [INF] from='client.24854 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:57.077760+0000 mon.a (mon.0) 717 : audit [INF] from='client.24866 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:57.078060+0000 mon.a (mon.0) 718 : audit [INF] from='client.24773 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi161-61993-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:57.078331+0000 mon.a (mon.0) 719 : audit [INF] from='client.24782 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi161-62041-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:57.628853+0000 mon.a (mon.0) 720 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi161-61759-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:57.629169+0000 mon.a (mon.0) 721 : audit [INF] from='client.24838 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi161-62225-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:57.629340+0000 mon.a (mon.0) 722 : audit [INF] from='client.24728 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi161-61765-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:57 smithi161 bash[27879]: audit 2024-02-13T03:26:57.629506+0000 mon.a (mon.0) 723 : audit [INF] from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi161-61719-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:52.324721+0000 mon.c (mon.2) 77 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi161-61759-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:52.325145+0000 mon.c (mon.2) 78 : audit [INF] from='client.? v1:172.21.15.161:0/1175961112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi161-61765-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:52.325511+0000 mon.c (mon.2) 79 : audit [INF] from='client.? v1:172.21.15.161:0/392796170' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi161-61798-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:52.325954+0000 mon.c (mon.2) 80 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:52.326382+0000 mon.c (mon.2) 81 : audit [INF] from='client.? v1:172.21.15.161:0/3042168812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi161-61906-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:52.326941+0000 mon.c (mon.2) 82 : audit [INF] from='client.? v1:172.21.15.161:0/1502259945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi161-61936-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:52.327420+0000 mon.c (mon.2) 83 : audit [INF] from='client.? v1:172.21.15.161:0/1913315211' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi161-61735-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:52.327705+0000 mon.c (mon.2) 84 : audit [INF] from='client.? v1:172.21.15.161:0/606117760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi161-61743-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:52.328612+0000 mon.b (mon.1) 28 : audit [INF] from='client.? v1:172.21.15.161:0/3651106834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi161-62225-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:52.328987+0000 mon.b (mon.1) 29 : audit [INF] from='client.? v1:172.21.15.161:0/56578333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi161-61719-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:52.329329+0000 mon.b (mon.1) 30 : audit [INF] from='client.? v1:172.21.15.161:0/2078631688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: cluster 2024-02-13T03:26:52.693015+0000 mgr.y (mgr.24473) 220 : cluster [DBG] pgmap v166: 580 pgs: 44 creating+peering, 132 active+clean, 404 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:57.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:53.721507+0000 mon.c (mon.2) 85 : audit [DBG] from='client.? v1:172.21.15.161:0/811745849' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-13T03:26:57.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: cluster 2024-02-13T03:26:54.694050+0000 mgr.y (mgr.24473) 221 : cluster [DBG] pgmap v167: 580 pgs: 44 creating+peering, 132 active+clean, 404 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-13T03:26:57.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:55.747385+0000 mon.c (mon.2) 86 : audit [DBG] from='client.? v1:172.21.15.161:0/352888529' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-13T03:26:57.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:55.801000+0000 mgr.y (mgr.24473) 222 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:26:57.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: cluster 2024-02-13T03:26:56.599803+0000 mon.a (mon.0) 698 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-13T03:26:57.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.604352+0000 mon.b (mon.1) 31 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:26:57.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.605975+0000 mon.a (mon.0) 699 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi161-61759-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.606220+0000 mon.a (mon.0) 700 : audit [INF] from='client.24838 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi161-62225-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.606476+0000 mon.a (mon.0) 701 : audit [INF] from='client.24728 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi161-61765-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.606621+0000 mon.a (mon.0) 702 : audit [INF] from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi161-61719-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.606774+0000 mon.a (mon.0) 703 : audit [INF] from='client.24734 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi161-61798-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.607159+0000 mon.a (mon.0) 704 : audit [INF] from='client.24826 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.607290+0000 mon.a (mon.0) 705 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.607478+0000 mon.a (mon.0) 706 : audit [INF] from='client.24752 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi161-61906-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.607657+0000 mon.a (mon.0) 707 : audit [INF] from='client.24758 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi161-61936-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.607833+0000 mon.a (mon.0) 708 : audit [INF] from='client.24710 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi161-61735-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.615544+0000 mon.a (mon.0) 709 : audit [INF] from='client.? v1:172.21.15.161:0/259716468' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi161-61726-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: cluster 2024-02-13T03:26:56.617905+0000 mon.a (mon.0) 710 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.619863+0000 mon.a (mon.0) 711 : audit [INF] from='client.24716 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi161-61743-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.622156+0000 mon.c (mon.2) 87 : audit [INF] from='client.? v1:172.21.15.161:0/1333079462' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi161-62012-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.622755+0000 mon.c (mon.2) 88 : audit [INF] from='client.? v1:172.21.15.161:0/1708383590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi161-62063-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.623035+0000 mon.c (mon.2) 89 : audit [INF] from='client.? v1:172.21.15.161:0/2516704546' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.623940+0000 mon.c (mon.2) 90 : audit [INF] from='client.? v1:172.21.15.161:0/3998698116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.624141+0000 mon.c (mon.2) 91 : audit [INF] from='client.? v1:172.21.15.161:0/3457363402' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.624307+0000 mon.c (mon.2) 92 : audit [INF] from='client.? v1:172.21.15.161:0/1554898749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi161-61993-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.624459+0000 mon.c (mon.2) 93 : audit [INF] from='client.? v1:172.21.15.161:0/3284003100' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi161-62041-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.624571+0000 mon.a (mon.0) 712 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.626766+0000 mon.a (mon.0) 713 : audit [INF] from='client.24779 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi161-62012-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.648905+0000 mon.a (mon.0) 714 : audit [INF] from='client.24788 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi161-62063-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:56.649181+0000 mon.a (mon.0) 715 : audit [INF] from='client.24764 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: cluster 2024-02-13T03:26:56.695082+0000 mgr.y (mgr.24473) 223 : cluster [DBG] pgmap v169: 900 pgs: 311 creating+peering, 132 active+clean, 457 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:57.077008+0000 mon.a (mon.0) 716 : audit [INF] from='client.24854 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:57.077760+0000 mon.a (mon.0) 717 : audit [INF] from='client.24866 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62124-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:57.078060+0000 mon.a (mon.0) 718 : audit [INF] from='client.24773 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi161-61993-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:57.078331+0000 mon.a (mon.0) 719 : audit [INF] from='client.24782 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi161-62041-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:57.628853+0000 mon.a (mon.0) 720 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi161-61759-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:57.629169+0000 mon.a (mon.0) 721 : audit [INF] from='client.24838 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi161-62225-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.990 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:57.629340+0000 mon.a (mon.0) 722 : audit [INF] from='client.24728 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi161-61765-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:57.991 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:57 smithi183 bash[20824]: audit 2024-02-13T03:26:57.629506+0000 mon.a (mon.0) 723 : audit [INF] from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi161-61719-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:58.261 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: api_list: [==========] Running 11 tests from 3 test suites. 2024-02-13T03:26:58.261 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [----------] Global test environment set-up. 2024-02-13T03:26:58.261 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [----------] 7 tests from LibRadosList 2024-02-13T03:26:58.261 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-02-13T03:26:58.261 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ OK ] LibRadosList.ListObjects (311 ms) 2024-02-13T03:26:58.261 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-02-13T03:26:58.261 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (36 ms) 2024-02-13T03:26:58.261 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-02-13T03:26:58.261 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: myset foo1,foo2,foo3 2024-02-13T03:26:58.261 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo1 2024-02-13T03:26:58.261 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo2 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo3 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: myset foo1,foo4,foo5 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo4 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo5 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo1 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: myset foo6,foo7 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo7 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo6 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: ns1:foo4 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: ns1:foo5 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: ns2:foo7 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: ns2:foo6 2024-02-13T03:26:58.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: ns1:foo1 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: :foo1 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: :foo2 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: :foo3 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (89 ms) 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 1 0 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 10 0 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 13 0 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 7 0 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 14 0 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 0 0 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 15 0 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 11 0 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 5 0 2024-02-13T03:26:58.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 8 0 2024-02-13T03:26:58.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 6 0 2024-02-13T03:26:58.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 3 0 2024-02-13T03:26:58.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 4 0 2024-02-13T03:26:58.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 12 0 2024-02-13T03:26:58.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 9 0 2024-02-13T03:26:58.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 2 0 2024-02-13T03:26:58.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-13T03:26:58.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (70 ms) 2024-02-13T03:26:58.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-02-13T03:26:58.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: x cursor=MIN 2024-02-13T03:26:58.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=1 cursor=13:02547ec2:::1:head 2024-02-13T03:26:58.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=10 cursor=13:52ea6a34:::10:head 2024-02-13T03:26:58.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=13 cursor=13:566253c9:::13:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=7 cursor=13:5c6b0b28:::7:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=14 cursor=13:62a1935d:::14:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=0 cursor=13:6cac518f:::0:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=15 cursor=13:863748b0:::15:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=11 cursor=13:89d3ae78:::11:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=5 cursor=13:b29083e3:::5:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=8 cursor=13:bd63b0f1:::8:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=6 cursor=13:c4fdafeb:::6:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=3 cursor=13:cfc208b3:::3:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=4 cursor=13:d83876eb:::4:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=12 cursor=13:de5d7c5f:::12:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=9 cursor=13:e960b815:::9:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > oid=2 cursor=13:f905c69b:::2:head 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: FIRST> seek to MIN oid=1 2024-02-13T03:26:58.265 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=1 cursor=13:02547ec2:::1:head 2024-02-13T03:26:58.266 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:02547ec2:::1:head 2024-02-13T03:26:58.266 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:02547ec2:::1:head -> 1 2024-02-13T03:26:58.266 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=10 cursor=13:52ea6a34:::10:head 2024-02-13T03:26:58.266 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:52ea6a34:::10:head 2024-02-13T03:26:58.266 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:52ea6a34:::10:head -> 10 2024-02-13T03:26:58.266 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=13 cursor=13:566253c9:::13:head 2024-02-13T03:26:58.266 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:566253c9:::13:head 2024-02-13T03:26:58.266 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:566253c9:::13:head -> 13 2024-02-13T03:26:58.266 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=7 cursor=13:5c6b0b28:::7:head 2024-02-13T03:26:58.266 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:5c6b0b28:::7:head 2024-02-13T03:26:58.266 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:5c6b0b28:::7:head -> 7 2024-02-13T03:26:58.314 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=14 cursor=13:62a1935d:::14:head 2024-02-13T03:26:58.314 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:62a1935d:::14:head 2024-02-13T03:26:58.314 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:62a1935d:::14:head -> 14 2024-02-13T03:26:58.314 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=0 cursor=13:6cac518f:::0:head 2024-02-13T03:26:58.314 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:6cac518f:::0:head 2024-02-13T03:26:58.314 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:6cac518f:::0:head -> 0 2024-02-13T03:26:58.314 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=15 cursor=13:863748b0:::15:head 2024-02-13T03:26:58.314 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:863748b0:::15:head 2024-02-13T03:26:58.314 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:863748b0:::15:head -> 15 2024-02-13T03:26:58.314 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=11 cursor=13:89d3ae78:::11:head 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:89d3ae78:::11:head 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:89d3ae78:::11:head -> 11 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=5 cursor=13:b29083e3:::5:head 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:b29083e3:::5:head 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:b29083e3:::5:head -> 5 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=8 cursor=13:bd63b0f1:::8:head 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:bd63b0f1:::8:head 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:bd63b0f1:::8:head -> 8 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=6 cursor=13:c4fdafeb:::6:head 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:c4fdafeb:::6:head 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:c4fdafeb:::6:head -> 6 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=3 cursor=13:cfc208b3:::3:head 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:cfc208b3:::3:head 2024-02-13T03:26:58.315 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:cfc208b3:::3:head -> 3 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=4 cursor=13:d83876eb:::4:head 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:d83876eb:::4:head 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:d83876eb:::4:head -> 4 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=12 cursor=13:de5d7c5f:::12:head 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:de5d7c5f:::12:head 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:de5d7c5f:::12:head -> 12 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=9 cursor=13:e960b815:::9:head 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:e960b815:::9:head 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:e960b815:::9:head -> 9 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : oid=2 cursor=13:f905c69b:::2:head 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:f905c69b:::2:head 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:f905c69b:::2:head -> 2 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:d83876eb:::4:head 2024-02-13T03:26:58.316 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:d83876eb:::4:head expected=13:d83876eb:::4:head 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:d83876eb:::4:head -> 4 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=4 expected=4 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:cfc208b3:::3:head 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:cfc208b3:::3:head expected=13:cfc208b3:::3:head 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:cfc208b3:::3:head -> 3 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=3 expected=3 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:f905c69b:::2:head 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:f905c69b:::2:head expected=13:f905c69b:::2:head 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:f905c69b:::2:head -> 2 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=2 expected=2 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:e960b815:::9:head 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:e960b815:::9:head expected=13:e960b815:::9:head 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:e960b815:::9:head -> 9 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=9 expected=9 2024-02-13T03:26:58.317 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:de5d7c5f:::12:head 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:de5d7c5f:::12:head expected=13:de5d7c5f:::12:head 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:de5d7c5f:::12:head -> 12 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=12 expected=12 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:c4fdafeb:::6:head 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:c4fdafeb:::6:head expected=13:c4fdafeb:::6:head 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:c4fdafeb:::6:head -> 6 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=6 expected=6 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:bd63b0f1:::8:head 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:bd63b0f1:::8:head expected=13:bd63b0f1:::8:head 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:bd63b0f1:::8:head -> 8 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=8 expected=8 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:b29083e3:::5:head 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:b29083e3:::5:head expected=13:b29083e3:::5:head 2024-02-13T03:26:58.318 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:b29083e3:::5:head -> 5 2024-02-13T03:26:58.467 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=5 expecte list_parallel: process_1_[62276]: starting. 2024-02-13T03:26:58.468 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_1_[62276]: creating pool ceph_test_rados_list_parallel.smithi161-62255 2024-02-13T03:26:58.468 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_1_[62276]: created object 0... 2024-02-13T03:26:58.468 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_1_[62276]: created object 25... 2024-02-13T03:26:58.468 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_1_[62276]: created object 49... 2024-02-13T03:26:58.468 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_1_[62276]: finishing. 2024-02-13T03:26:58.468 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_1_[62276]: shutting down. 2024-02-13T03:26:58.468 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_2_[62277]: starting. 2024-02-13T03:26:58.468 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_2_[62277]: listing objects. 2024-02-13T03:26:58.468 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_2_[62277]: listed object 0... 2024-02-13T03:26:58.468 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_2_[62277]: listed object 25... 2024-02-13T03:26:58.468 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_2_[62277]: saw 50 objects 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_2_[62277]: shutting down. 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_3_[62984]: starting. 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_3_[62984]: creating pool ceph_test_rados_list_parallel.smithi161-62255 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_3_[62984]: created object 0... 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_3_[62984]: created object 25... 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_3_[62984]: created object 49... 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_3_[62984]: finishing. 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_3_[62984]: shutting down. 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_4_[62985]: starting. 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_4_[62985]: listing objects. 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_4_[62985]: listed object 0... 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_4_[62985]: listed object 25... 2024-02-13T03:26:58.469 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_4_[62985]: saw 45 objects 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_4_[62985]: shutting down. 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_5_[62986]: starting. 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_5_[62986]: removed 25 objects... 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_5_[62986]: removed half of the objects 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_5_[62986]: removed 50 objects... 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_5_[62986]: removed 50 objects 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_5_[62986]: shutting down. 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_6_[63033]: starting. 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_6_[63033]: creating pool ceph_test_rados_list_parallel.smithi161-62255 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_6_[63033]: created object 0... 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_6_[63033]: created object 25... 2024-02-13T03:26:58.470 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_6_[63033]: created object 49... 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_6_[63033]: finishing. 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_6_[63033]: shutting down. 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_7_[63034]: starting. 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_7_[63034]: listing objects. 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_7_[63034]: listed object 0... 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_7_[63034]: listed object 25... 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_7_[63034]: listed object 50... 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_7_[63034]: saw 55 objects 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_7_[63034]: shutting down. 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_8_[63035]: starting. 2024-02-13T03:26:58.471 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_8_[63035]: added 25 objects... 2024-02-13T03:26:58.472 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_8_[63035]: added half of the objects 2024-02-13T03:26:58.472 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_8_[63035]: added 50 objects... 2024-02-13T03:26:58.472 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_8_[63035]: added 50 objects 2024-02-13T03:26:58.472 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_8_[63035]: shutting down. 2024-02-13T03:26:58.472 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.472 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.663 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.663 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_9_[63082]: starting. 2024-02-13T03:26:58.663 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_9_[63082]: creating pool ceph_test_rados_list_parallel.smithi161-62255 2024-02-13T03:26:58.663 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_9_[63082]: created object 0... 2024-02-13T03:26:58.663 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_9_[63082]: created object 25... 2024-02-13T03:26:58.663 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_9_[63082]: created object 49... 2024-02-13T03:26:58.663 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_9_[63082]: finishing. 2024-02-13T03:26:58.663 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_9_[63082]: shutting down. 2024-02-13T03:26:58.663 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.663 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_10_[63083]: starting. 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_10_[63083]: listing objects. 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_10_[63083]: listed object 0... 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_10_[63083]: listed object 25... 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_10_[63083]: listed object 50... 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_10_[63083]: listed object 75... 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_10_[63083]: saw 97 objects 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_10_[63083]: shutting down. 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_13_[63086]: starting. 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_13_[63086]: removed 25 objects... 2024-02-13T03:26:58.664 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_13_[63086]: removed half of the objects 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_13_[63086]: removed 50 objects... 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_13_[63086]: removed 50 objects 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_13_[63086]: shutting down. 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_11_[63084]: starting. 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_11_[63084]: added 25 objects... 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_11_[63084]: added half of the objects 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_11_[63084]: added 50 objects... 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_11_[63084]: added 50 objects 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_11_[63084]: shutting down. 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.665 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_12_[63085]: starting. 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_12_[63085]: added 25 objects... 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_12_[63085]: added half of the objects 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_12_[63085]: added 50 objects... 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_12_[63085]: added 50 objects 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_12_[63085]: shutting down. 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_14_[63162]: starting. 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_14_[63162]: creating pool ceph_test_rados_list_parallel.smithi161-62255 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_14_[63162]: created object 0... 2024-02-13T03:26:58.666 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_14_[63162]: created object 25... 2024-02-13T03:26:58.667 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_14_[63162]: created object 49... 2024-02-13T03:26:58.667 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_14_[63162]: finishing. 2024-02-13T03:26:58.667 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_14_[63162]: shutting down. 2024-02-13T03:26:58.667 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.667 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.667 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.667 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:26:58.667 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_15_[63163]: starting. 2024-02-13T03:26:58.667 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_15_[63163]: listing objects. 2024-02-13T03:26:58.667 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_15_[63163]: listed object 0... 2024-02-13T03:26:59.486 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:26:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:26:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.629699+0000 mon.a (mon.0) 724 : audit [INF] from='client.24734 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi161-61798-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.629862+0000 mon.a (mon.0) 725 : audit [INF] from='client.24826 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.630021+0000 mon.a (mon.0) 726 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.630190+0000 mon.a (mon.0) 727 : audit [INF] from='client.24752 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi161-61906-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.630312+0000 mon.a (mon.0) 728 : audit [INF] from='client.24758 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi161-61936-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.630430+0000 mon.a (mon.0) 729 : audit [INF] from='client.24710 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi161-61735-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.630551+0000 mon.a (mon.0) 730 : audit [INF] from='client.24716 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi161-61743-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.630676+0000 mon.a (mon.0) 731 : audit [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:26:59.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.630808+0000 mon.a (mon.0) 732 : audit [INF] from='client.24779 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi161-62012-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.630918+0000 mon.a (mon.0) 733 : audit [INF] from='client.24788 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi161-62063-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.631015+0000 mon.a (mon.0) 734 : audit [INF] from='client.24764 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.631125+0000 mon.a (mon.0) 735 : audit [INF] from='client.24854 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.631327+0000 mon.a (mon.0) 736 : audit [INF] from='client.24866 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.631443+0000 mon.a (mon.0) 737 : audit [INF] from='client.24773 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi161-61993-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.631593+0000 mon.a (mon.0) 738 : audit [INF] from='client.24782 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi161-62041-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: cluster 2024-02-13T03:26:57.631722+0000 mon.a (mon.0) 739 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-13T03:26:59.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.643581+0000 mon.b (mon.1) 32 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi161-62096-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:59.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.673442+0000 mon.a (mon.0) 740 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi161-62096-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:59.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:26:59 smithi183 bash[20824]: audit 2024-02-13T03:26:57.803890+0000 mon.c (mon.2) 94 : audit [DBG] from='client.? v1:172.21.15.161:0/2351312691' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-13T03:26:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.629699+0000 mon.a (mon.0) 724 : audit [INF] from='client.24734 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi161-61798-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.629862+0000 mon.a (mon.0) 725 : audit [INF] from='client.24826 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.630021+0000 mon.a (mon.0) 726 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.630190+0000 mon.a (mon.0) 727 : audit [INF] from='client.24752 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi161-61906-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.630312+0000 mon.a (mon.0) 728 : audit [INF] from='client.24758 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi161-61936-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.630430+0000 mon.a (mon.0) 729 : audit [INF] from='client.24710 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi161-61735-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.630551+0000 mon.a (mon.0) 730 : audit [INF] from='client.24716 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi161-61743-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.630676+0000 mon.a (mon.0) 731 : audit [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:26:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.630808+0000 mon.a (mon.0) 732 : audit [INF] from='client.24779 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi161-62012-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.630918+0000 mon.a (mon.0) 733 : audit [INF] from='client.24788 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi161-62063-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.631015+0000 mon.a (mon.0) 734 : audit [INF] from='client.24764 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.631125+0000 mon.a (mon.0) 735 : audit [INF] from='client.24854 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.631327+0000 mon.a (mon.0) 736 : audit [INF] from='client.24866 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.631443+0000 mon.a (mon.0) 737 : audit [INF] from='client.24773 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi161-61993-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.631593+0000 mon.a (mon.0) 738 : audit [INF] from='client.24782 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi161-62041-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: cluster 2024-02-13T03:26:57.631722+0000 mon.a (mon.0) 739 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-13T03:26:59.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.643581+0000 mon.b (mon.1) 32 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi161-62096-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:59.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.673442+0000 mon.a (mon.0) 740 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi161-62096-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:59.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[27879]: audit 2024-02-13T03:26:57.803890+0000 mon.c (mon.2) 94 : audit [DBG] from='client.? v1:172.21.15.161:0/2351312691' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-13T03:26:59.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.629699+0000 mon.a (mon.0) 724 : audit [INF] from='client.24734 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi161-61798-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.629862+0000 mon.a (mon.0) 725 : audit [INF] from='client.24826 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.630021+0000 mon.a (mon.0) 726 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.630190+0000 mon.a (mon.0) 727 : audit [INF] from='client.24752 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi161-61906-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.630312+0000 mon.a (mon.0) 728 : audit [INF] from='client.24758 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi161-61936-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.630430+0000 mon.a (mon.0) 729 : audit [INF] from='client.24710 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi161-61735-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.630551+0000 mon.a (mon.0) 730 : audit [INF] from='client.24716 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi161-61743-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.630676+0000 mon.a (mon.0) 731 : audit [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:26:59.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.630808+0000 mon.a (mon.0) 732 : audit [INF] from='client.24779 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi161-62012-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.630918+0000 mon.a (mon.0) 733 : audit [INF] from='client.24788 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi161-62063-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.631015+0000 mon.a (mon.0) 734 : audit [INF] from='client.24764 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.631125+0000 mon.a (mon.0) 735 : audit [INF] from='client.24854 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.631327+0000 mon.a (mon.0) 736 : audit [INF] from='client.24866 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.631443+0000 mon.a (mon.0) 737 : audit [INF] from='client.24773 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi161-61993-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.631593+0000 mon.a (mon.0) 738 : audit [INF] from='client.24782 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi161-62041-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:26:59.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: cluster 2024-02-13T03:26:57.631722+0000 mon.a (mon.0) 739 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-13T03:26:59.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.643581+0000 mon.b (mon.1) 32 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi161-62096-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:59.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.673442+0000 mon.a (mon.0) 740 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi161-62096-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:26:59.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:26:59 smithi161 bash[19769]: audit 2024-02-13T03:26:57.803890+0000 mon.c (mon.2) 94 : audit [DBG] from='client.? v1:172.21.15.161:0/2351312691' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-13T03:27:00.172 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_15_[63163]: listed object 25... 2024-02-13T03:27:00.172 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_15_[63163]: listed object 50... 2024-02-13T03:27:00.172 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_15_[63163]: listed object 75... 2024-02-13T03:27:00.172 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_15_[63163]: listed object 100... 2024-02-13T03:27:00.172 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_15_[63163]: listed object 125... 2024-02-13T03:27:00.172 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_15_[63163]: saw 150 objects 2024-02-13T03:27:00.172 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_15_[63163]: shutting down. 2024-02-13T03:27:00.172 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:27:00.172 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_16_[63164]: starting. 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_16_[63164]: added 25 objects... 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_16_[63164]: added half of the objects 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_16_[63164]: added 50 objects... 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_16_[63164]: added 50 objects 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: process_16_[63164]: shutting down. 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******************************* 2024-02-13T03:27:00.173 INFO:tasks.workunit.client.0.smithi161.stdout: list_parallel: ******* SUCCESS ********** 2024-02-13T03:27:00.177 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-02-13T03:27:00.177 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [==========] Running 16 tests from 1 test suite. 2024-02-13T03:27:00.177 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-02-13T03:27:00.177 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest 2024-02-13T03:27:00.177 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-02-13T03:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-02-13T03:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-02-13T03:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (272 ms) 2024-02-13T03:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-02-13T03:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (10 ms) 2024-02-13T03:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-02-13T03:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (12 ms) 2024-02-13T03:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-02-13T03:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (40 ms) 2024-02-13T03:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-02-13T03:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (7 ms) 2024-02-13T03:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-02-13T03:27:00.178 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (9 ms) 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (5 ms) 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (5 ms) 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (5 ms) 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (5 ms) 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (14 ms) 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-02-13T03:27:00.179 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (10 ms) 2024-02-13T03:27:00.180 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-02-13T03:27:00.180 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (10 ms) 2024-02-13T03:27:00.180 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-02-13T03:27:00.180 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (8 ms) 2024-02-13T03:27:00.180 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest (418 ms total) 2024-02-13T03:27:00.180 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: 2024-02-13T03:27:00.180 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-02-13T03:27:00.180 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [==========] 16 tests from 1 test suite ran. (13740 ms total) 2024-02-13T03:27:00.180 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_read_operations: [ PASSED ] 16 tests. 2024-02-13T03:27:00.183 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-02-13T03:27:00.183 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-02-13T03:27:00.183 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-02-13T03:27:00.183 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-02-13T03:27:00.183 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-02-13T03:27:00.183 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (37 ms) 2024-02-13T03:27:00.183 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-02-13T03:27:00.183 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (48 ms) 2024-02-13T03:27:00.183 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-02-13T03:27:00.183 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (13706 ms) 2024-02-13T03:27:00.183 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (13792 ms total) 2024-02-13T03:27:00.183 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: 2024-02-13T03:27:00.183 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-02-13T03:27:00.184 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (13792 ms total) 2024-02-13T03:27:00.184 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-02-13T03:27:00.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: Running main() from gmock_main.cc 2024-02-13T03:27:00.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-02-13T03:27:00.262 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [----------] Global test environment set-up. 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (60 ms) 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (44 ms) 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (13690 ms) 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.629699+0000 mon.a [INF] from='client.24734 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi161-61798-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.629862+0000 mon.a [INF] from='client.24826 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.630021+0000 mon.a [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.630190+0000 mon.a [INF] from='client.24752 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi161-61906-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.630312+0000 mon.a [INF] from='client.24758 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi161-61936-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.263 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.630430+0000 mon.a [INF] from='client.24710 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi161-61735-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.630551+0000 mon.a [INF] from='client.24716 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi161-61743-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.630676+0000 mon.a [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:00.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.630808+0000 mon.a [INF] from='client.24779 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi161-62012-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.630918+0000 mon.a [INF] from='client.24788 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi161-62063-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.631015+0000 mon.a [INF] from='client.24764 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.264 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.631125+0000 mon.a [INF] from='client.24854 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.688 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.631327+0000 mon.a [INF] from='client.24866 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62124-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.688 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.631443+0000 mon.a [INF] from='client.24773 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi161-61993-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.688 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.631593+0000 mon.a [INF] from='client.24782 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi161-62041-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.688 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.643581+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi161-62096-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:00.689 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:57.673442+0000 mon.a [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi161-62096-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:00.689 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:59.160957+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:00.689 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:59.161241+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/953175453' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:00.689 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:59.174057+0000 mon.a [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:00.689 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:59.175201+0000 mon.a [INF] from='client.25013 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:00.689 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:59.189409+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:00.689 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:59.190274+0000 mon.a [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:00.689 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:59.192169+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:00.689 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:59.192656+0000 mon.a [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:00.689 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:59.193365+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi161-62012-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:00.689 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:26:59.193979+0000 mon.a [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi161-62012-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:26:58.434399+0000 mon.c (mon.2) 95 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: cluster 2024-02-13T03:26:58.695851+0000 mgr.y (mgr.24473) 224 : cluster [DBG] pgmap v171: 900 pgs: 279 creating+peering, 132 active+clean, 489 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 802 B/s rd, 0 op/s 2024-02-13T03:27:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:26:59.160957+0000 mon.c (mon.2) 96 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:26:59.161241+0000 mon.c (mon.2) 97 : audit [INF] from='client.? v1:172.21.15.161:0/953175453' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: cluster 2024-02-13T03:26:59.161420+0000 mon.a (mon.0) 741 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-13T03:27:00.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:26:59.174057+0000 mon.a (mon.0) 742 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:00.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:26:59.175201+0000 mon.a (mon.0) 743 : audit [INF] from='client.25013 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:00.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:26:59.189409+0000 mon.c (mon.2) 98 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:00.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:26:59.190274+0000 mon.a (mon.0) 744 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:00.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:26:59.192169+0000 mon.c (mon.2) 99 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:00.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:26:59.192656+0000 mon.a (mon.0) 745 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:00.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:26:59.193365+0000 mon.c (mon.2) 100 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi161-62012-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:00.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:26:59.193979+0000 mon.a (mon.0) 746 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi161-62012-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:00.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:26:59.435796+0000 mon.c (mon.2) 101 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:00.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:27:00.165254+0000 mon.a (mon.0) 747 : audit [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi161-62096-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]': finished 2024-02-13T03:27:00.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:27:00.165937+0000 mon.a (mon.0) 748 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:27:00.166161+0000 mon.a (mon.0) 749 : audit [INF] from='client.25013 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:00.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:27:00.166633+0000 mon.a (mon.0) 750 : audit [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi161-62012-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:00.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: cluster 2024-02-13T03:27:00.167240+0000 mon.a (mon.0) 751 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-13T03:27:00.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:00 smithi183 bash[20824]: audit 2024-02-13T03:27:00.174259+0000 mon.c (mon.2) 102 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi161-62012-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:26:58.434399+0000 mon.c (mon.2) 95 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: cluster 2024-02-13T03:26:58.695851+0000 mgr.y (mgr.24473) 224 : cluster [DBG] pgmap v171: 900 pgs: 279 creating+peering, 132 active+clean, 489 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 802 B/s rd, 0 op/s 2024-02-13T03:27:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:26:59.160957+0000 mon.c (mon.2) 96 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:26:59.161241+0000 mon.c (mon.2) 97 : audit [INF] from='client.? v1:172.21.15.161:0/953175453' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: cluster 2024-02-13T03:26:59.161420+0000 mon.a (mon.0) 741 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-13T03:27:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:26:59.174057+0000 mon.a (mon.0) 742 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:26:59.175201+0000 mon.a (mon.0) 743 : audit [INF] from='client.25013 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:26:59.189409+0000 mon.c (mon.2) 98 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:26:59.190274+0000 mon.a (mon.0) 744 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:01.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:26:59.192169+0000 mon.c (mon.2) 99 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:01.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:26:59.192656+0000 mon.a (mon.0) 745 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:01.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:26:59.193365+0000 mon.c (mon.2) 100 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi161-62012-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:01.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:26:59.193979+0000 mon.a (mon.0) 746 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi161-62012-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:01.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:26:59.435796+0000 mon.c (mon.2) 101 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:01.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:27:00.165254+0000 mon.a (mon.0) 747 : audit [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi161-62096-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]': finished 2024-02-13T03:27:01.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:27:00.165937+0000 mon.a (mon.0) 748 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:01.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:27:00.166161+0000 mon.a (mon.0) 749 : audit [INF] from='client.25013 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:01.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:27:00.166633+0000 mon.a (mon.0) 750 : audit [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi161-62012-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:01.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: cluster 2024-02-13T03:27:00.167240+0000 mon.a (mon.0) 751 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-13T03:27:01.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[27879]: audit 2024-02-13T03:27:00.174259+0000 mon.c (mon.2) 102 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi161-62012-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:01.102 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:26:58.434399+0000 mon.c (mon.2) 95 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:01.102 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: cluster 2024-02-13T03:26:58.695851+0000 mgr.y (mgr.24473) 224 : cluster [DBG] pgmap v171: 900 pgs: 279 creating+peering, 132 active+clean, 489 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 802 B/s rd, 0 op/s 2024-02-13T03:27:01.102 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:26:59.160957+0000 mon.c (mon.2) 96 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:01.102 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:26:59.161241+0000 mon.c (mon.2) 97 : audit [INF] from='client.? v1:172.21.15.161:0/953175453' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:01.102 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: cluster 2024-02-13T03:26:59.161420+0000 mon.a (mon.0) 741 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-13T03:27:01.102 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:26:59.174057+0000 mon.a (mon.0) 742 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:01.102 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:26:59.175201+0000 mon.a (mon.0) 743 : audit [INF] from='client.25013 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:01.102 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:26:59.189409+0000 mon.c (mon.2) 98 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:01.102 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:26:59.190274+0000 mon.a (mon.0) 744 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:01.102 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:26:59.192169+0000 mon.c (mon.2) 99 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:01.102 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:26:59.192656+0000 mon.a (mon.0) 745 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:01.102 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:26:59.193365+0000 mon.c (mon.2) 100 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi161-62012-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:01.102 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:26:59.193979+0000 mon.a (mon.0) 746 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi161-62012-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:01.103 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:26:59.435796+0000 mon.c (mon.2) 101 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:01.103 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:27:00.165254+0000 mon.a (mon.0) 747 : audit [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi161-62096-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]': finished 2024-02-13T03:27:01.103 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:27:00.165937+0000 mon.a (mon.0) 748 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:01.103 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:27:00.166161+0000 mon.a (mon.0) 749 : audit [INF] from='client.25013 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:01.103 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:27:00.166633+0000 mon.a (mon.0) 750 : audit [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi161-62012-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:01.103 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: cluster 2024-02-13T03:27:00.167240+0000 mon.a (mon.0) 751 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-13T03:27:01.103 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:00 smithi161 bash[19769]: audit 2024-02-13T03:27:00.174259+0000 mon.c (mon.2) 102 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi161-62012-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:01.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:27:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:27:01] "GET /metrics HTTP/1.1" 200 177028 "" "Prometheus/2.33.4" 2024-02-13T03:27:02.425 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.165254+0000 mon.a [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi161-62096-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]': finished 2024-02-13T03:27:02.425 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.165937+0000 mon.a [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:02.425 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.166161+0000 mon.a [INF] from='client.25013 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:02.425 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.166633+0000 mon.a [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi161-62012-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:02.425 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.174259+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi161-62012-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:02.425 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.186714+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/1886629756' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.425 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.197830+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/686252831' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi161-61719-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.425 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.246040+0000 client.admin [INF] onexx 2024-02-13T03:27:02.425 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.248382+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-13T03:27:02.425 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.264453+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:02.425 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.278162+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:02.426 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.292757+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:02.426 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.524321+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-13T03:27:02.426 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.525175+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-13T03:27:02.426 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.525943+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-13T03:27:02.426 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.526626+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-13T03:27:02.426 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.527290+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-13T03:27:02.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.186714+0000 mon.b (mon.1) 33 : audit [INF] from='client.? v1:172.21.15.161:0/1886629756' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.197830+0000 mon.c (mon.2) 103 : audit [INF] from='client.? v1:172.21.15.161:0/686252831' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi161-61719-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: cluster 2024-02-13T03:27:00.246040+0000 client.admin (client.25150) 0 : cluster [INF] onexx 2024-02-13T03:27:02.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.248382+0000 mon.b (mon.1) 34 : audit [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-13T03:27:02.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.264453+0000 mon.b (mon.1) 35 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.278162+0000 mon.b (mon.1) 36 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.292757+0000 mon.b (mon.1) 37 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.437184+0000 mon.c (mon.2) 104 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.524321+0000 mon.c (mon.2) 105 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-13T03:27:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.525175+0000 mon.c (mon.2) 106 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-13T03:27:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.525943+0000 mon.c (mon.2) 107 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-13T03:27:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.526626+0000 mon.c (mon.2) 108 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-13T03:27:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.527290+0000 mon.c (mon.2) 109 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-13T03:27:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.527981+0000 mon.c (mon.2) 110 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-13T03:27:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.528627+0000 mon.c (mon.2) 111 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-13T03:27:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.529264+0000 mon.c (mon.2) 112 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-13T03:27:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.186714+0000 mon.b (mon.1) 33 : audit [INF] from='client.? v1:172.21.15.161:0/1886629756' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.197830+0000 mon.c (mon.2) 103 : audit [INF] from='client.? v1:172.21.15.161:0/686252831' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi161-61719-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: cluster 2024-02-13T03:27:00.246040+0000 client.admin (client.25150) 0 : cluster [INF] onexx 2024-02-13T03:27:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.248382+0000 mon.b (mon.1) 34 : audit [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-13T03:27:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.264453+0000 mon.b (mon.1) 35 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.278162+0000 mon.b (mon.1) 36 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.292757+0000 mon.b (mon.1) 37 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.437184+0000 mon.c (mon.2) 104 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.524321+0000 mon.c (mon.2) 105 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-13T03:27:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.525175+0000 mon.c (mon.2) 106 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-13T03:27:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.525943+0000 mon.c (mon.2) 107 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-13T03:27:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.526626+0000 mon.c (mon.2) 108 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-13T03:27:02.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.527290+0000 mon.c (mon.2) 109 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-13T03:27:02.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.527981+0000 mon.c (mon.2) 110 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-13T03:27:02.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.528627+0000 mon.c (mon.2) 111 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-13T03:27:02.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.529264+0000 mon.c (mon.2) 112 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-13T03:27:02.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.529932+0000 mon.c (mon.2) 113 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-13T03:27:02.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.530631+0000 mon.c (mon.2) 114 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-13T03:27:02.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:00.680984+0000 mon.a (mon.0) 752 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi161-62012-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:02.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:01.107778+0000 mon.a (mon.0) 753 : audit [INF] from='client.25135 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:01.108074+0000 mon.a (mon.0) 754 : audit [INF] from='client.25130 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi161-61719-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:01.438373+0000 mon.c (mon.2) 115 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:02.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[27879]: audit 2024-02-13T03:27:01.543738+0000 mon.a (mon.0) 755 : audit [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-13T03:27:02.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.529932+0000 mon.c (mon.2) 113 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-13T03:27:02.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.530631+0000 mon.c (mon.2) 114 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-13T03:27:02.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:00.680984+0000 mon.a (mon.0) 752 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi161-62012-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:02.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:01.107778+0000 mon.a (mon.0) 753 : audit [INF] from='client.25135 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:01.108074+0000 mon.a (mon.0) 754 : audit [INF] from='client.25130 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi161-61719-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:01.438373+0000 mon.c (mon.2) 115 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:02.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:02 smithi161 bash[19769]: audit 2024-02-13T03:27:01.543738+0000 mon.a (mon.0) 755 : audit [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-13T03:27:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.186714+0000 mon.b (mon.1) 33 : audit [INF] from='client.? v1:172.21.15.161:0/1886629756' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.197830+0000 mon.c (mon.2) 103 : audit [INF] from='client.? v1:172.21.15.161:0/686252831' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi161-61719-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: cluster 2024-02-13T03:27:00.246040+0000 client.admin (client.25150) 0 : cluster [INF] onexx 2024-02-13T03:27:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.248382+0000 mon.b (mon.1) 34 : audit [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-13T03:27:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.264453+0000 mon.b (mon.1) 35 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.278162+0000 mon.b (mon.1) 36 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:02.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.292757+0000 mon.b (mon.1) 37 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:02.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.437184+0000 mon.c (mon.2) 104 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:02.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.524321+0000 mon.c (mon.2) 105 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-13T03:27:02.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.525175+0000 mon.c (mon.2) 106 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-13T03:27:02.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.525943+0000 mon.c (mon.2) 107 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-13T03:27:02.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.526626+0000 mon.c (mon.2) 108 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-13T03:27:02.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.527290+0000 mon.c (mon.2) 109 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-13T03:27:02.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.527981+0000 mon.c (mon.2) 110 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-13T03:27:02.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.528627+0000 mon.c (mon.2) 111 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-13T03:27:02.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.529264+0000 mon.c (mon.2) 112 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-13T03:27:02.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.529932+0000 mon.c (mon.2) 113 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-13T03:27:02.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.530631+0000 mon.c (mon.2) 114 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-13T03:27:02.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:00.680984+0000 mon.a (mon.0) 752 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi161-62012-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:02.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:01.107778+0000 mon.a (mon.0) 753 : audit [INF] from='client.25135 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:01.108074+0000 mon.a (mon.0) 754 : audit [INF] from='client.25130 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi161-61719-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:02.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:01.438373+0000 mon.c (mon.2) 115 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:02.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:02 smithi183 bash[20824]: audit 2024-02-13T03:27:01.543738+0000 mon.a (mon.0) 755 : audit [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-13T03:27:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:00.524576+0000 mgr.y (mgr.24473) 225 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-13T03:27:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:00.525364+0000 mgr.y (mgr.24473) 226 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:00.526159+0000 mgr.y (mgr.24473) 227 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:00.526799+0000 mgr.y (mgr.24473) 228 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:00.527434+0000 mgr.y (mgr.24473) 229 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:00.528147+0000 mgr.y (mgr.24473) 230 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:00.528799+0000 mgr.y (mgr.24473) 231 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:00.529426+0000 mgr.y (mgr.24473) 232 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:00.530146+0000 mgr.y (mgr.24473) 233 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:00.530826+0000 mgr.y (mgr.24473) 234 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: cluster 2024-02-13T03:27:00.697909+0000 mgr.y (mgr.24473) 235 : cluster [DBG] pgmap v174: 620 pgs: 72 creating+peering, 548 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 553 op/s 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: cluster 2024-02-13T03:27:01.168256+0000 osd.2 (osd.2) 3 : cluster [DBG] 18.7 deep-scrub starts 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: cluster 2024-02-13T03:27:01.211666+0000 osd.1 (osd.1) 3 : cluster [DBG] 18.3 deep-scrub starts 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: cluster 2024-02-13T03:27:01.215340+0000 osd.1 (osd.1) 4 : cluster [DBG] 18.3 deep-scrub ok 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: cluster 2024-02-13T03:27:01.259397+0000 osd.2 (osd.2) 4 : cluster [DBG] 18.7 deep-scrub ok 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:01.570112+0000 mon.c (mon.2) 116 : audit [INF] from='client.? v1:172.21.15.161:0/953175453' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:01.612480+0000 mon.b (mon.1) 38 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:01.617514+0000 mon.c (mon.2) 117 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:01.977578+0000 mon.a (mon.0) 756 : audit [INF] from='client.25135 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:01.977896+0000 mon.a (mon.0) 757 : audit [INF] from='client.25130 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi161-61719-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: cluster 2024-02-13T03:27:01.978539+0000 mon.a (mon.0) 758 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:01.979246+0000 mon.c (mon.2) 118 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:01.981048+0000 mon.c (mon.2) 119 : audit [INF] from='client.? v1:172.21.15.161:0/1890051921' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:01.982475+0000 mon.a (mon.0) 759 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:01.986849+0000 mon.b (mon.1) 39 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: cluster 2024-02-13T03:27:02.420468+0000 osd.7 (osd.7) 3 : cluster [DBG] 18.2 deep-scrub starts 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.422995+0000 mon.a (mon.0) 760 : audit [INF] from='client.25150 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: cluster 2024-02-13T03:27:02.424310+0000 osd.7 (osd.7) 4 : cluster [DBG] 18.2 deep-scrub ok 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.424491+0000 mon.a (mon.0) 761 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.426526+0000 mon.a (mon.0) 762 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.426892+0000 mon.a (mon.0) 763 : audit [INF] from='client.25013 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.432259+0000 mon.b (mon.1) 40 : audit [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.432501+0000 mon.b (mon.1) 41 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.432708+0000 mon.b (mon.1) 42 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.439505+0000 mon.c (mon.2) 120 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: cluster 2024-02-13T03:27:02.563736+0000 osd.3 (osd.3) 3 : cluster [DBG] 18.1 deep-scrub starts 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: cluster 2024-02-13T03:27:02.618418+0000 osd.3 (osd.3) 4 : cluster [DBG] 18.1 deep-scrub ok 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: cluster 2024-02-13T03:27:02.698913+0000 mgr.y (mgr.24473) 236 : cluster [DBG] pgmap v176: 620 pgs: 96 unknown, 72 creating+peering, 452 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 9.5 MiB/s rd, 29 MiB/s wr, 431 op/s 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: cluster 2024-02-13T03:27:02.855132+0000 client.admin (client.25150) 0 : cluster [INF] twoxx 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.855813+0000 mon.a (mon.0) 764 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.862410+0000 mon.a (mon.0) 765 : audit [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi161-62012-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi161-62012-6"}]': finished 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.862598+0000 mon.a (mon.0) 766 : audit [INF] from='client.25013 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: cluster 2024-02-13T03:27:02.864903+0000 mon.a (mon.0) 767 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.866258+0000 mon.c (mon.2) 121 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.868879+0000 mon.a (mon.0) 768 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.869801+0000 mon.c (mon.2) 122 : audit [INF] from='client.? v1:172.21.15.161:0/953175453' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi161-61726-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-13T03:27:03.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.870409+0000 mon.a (mon.0) 769 : audit [INF] from='client.25151 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.873809+0000 mon.a (mon.0) 770 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:03.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.874537+0000 mon.a (mon.0) 771 : audit [INF] from='client.25013 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi161-61726-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-13T03:27:03.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.877296+0000 mon.a (mon.0) 772 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:03.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.877429+0000 mon.c (mon.2) 123 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.877887+0000 mon.a (mon.0) 773 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.882341+0000 mon.a (mon.0) 774 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:03.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.882549+0000 mon.b (mon.1) 43 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:03.741 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.882639+0000 mon.a (mon.0) 775 : audit [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-13T03:27:03.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.882885+0000 mon.a (mon.0) 776 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:03.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.883141+0000 mon.a (mon.0) 777 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:03.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.883700+0000 mon.a (mon.0) 778 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:03.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.886561+0000 mon.a (mon.0) 779 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi161-61735-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.886647+0000 mon.b (mon.1) 44 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi161-61735-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.888053+0000 mon.a (mon.0) 780 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.888424+0000 mon.a (mon.0) 781 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.890245+0000 mon.b (mon.1) 45 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.890619+0000 mon.b (mon.1) 46 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.890958+0000 mon.b (mon.1) 47 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi161-61765-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:03 smithi183 bash[20824]: audit 2024-02-13T03:27:02.893279+0000 mon.a (mon.0) 782 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi161-61765-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:00.524576+0000 mgr.y (mgr.24473) 225 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-13T03:27:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:00.525364+0000 mgr.y (mgr.24473) 226 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-13T03:27:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:00.526159+0000 mgr.y (mgr.24473) 227 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-13T03:27:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:00.526799+0000 mgr.y (mgr.24473) 228 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-13T03:27:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:00.527434+0000 mgr.y (mgr.24473) 229 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-13T03:27:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:00.528147+0000 mgr.y (mgr.24473) 230 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-13T03:27:03.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:00.528799+0000 mgr.y (mgr.24473) 231 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-13T03:27:03.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:00.529426+0000 mgr.y (mgr.24473) 232 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-13T03:27:03.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:00.530146+0000 mgr.y (mgr.24473) 233 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-13T03:27:03.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:00.530826+0000 mgr.y (mgr.24473) 234 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-13T03:27:03.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: cluster 2024-02-13T03:27:00.697909+0000 mgr.y (mgr.24473) 235 : cluster [DBG] pgmap v174: 620 pgs: 72 creating+peering, 548 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 553 op/s 2024-02-13T03:27:03.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: cluster 2024-02-13T03:27:01.168256+0000 osd.2 (osd.2) 3 : cluster [DBG] 18.7 deep-scrub starts 2024-02-13T03:27:03.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: cluster 2024-02-13T03:27:01.211666+0000 osd.1 (osd.1) 3 : cluster [DBG] 18.3 deep-scrub starts 2024-02-13T03:27:03.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: cluster 2024-02-13T03:27:01.215340+0000 osd.1 (osd.1) 4 : cluster [DBG] 18.3 deep-scrub ok 2024-02-13T03:27:03.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: cluster 2024-02-13T03:27:01.259397+0000 osd.2 (osd.2) 4 : cluster [DBG] 18.7 deep-scrub ok 2024-02-13T03:27:03.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:01.570112+0000 mon.c (mon.2) 116 : audit [INF] from='client.? v1:172.21.15.161:0/953175453' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:01.612480+0000 mon.b (mon.1) 38 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:03.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:01.617514+0000 mon.c (mon.2) 117 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:03.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:00.524576+0000 mgr.y (mgr.24473) 225 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-13T03:27:03.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:00.525364+0000 mgr.y (mgr.24473) 226 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-13T03:27:03.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:00.526159+0000 mgr.y (mgr.24473) 227 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-13T03:27:03.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:00.526799+0000 mgr.y (mgr.24473) 228 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-13T03:27:03.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:00.527434+0000 mgr.y (mgr.24473) 229 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-13T03:27:03.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:00.528147+0000 mgr.y (mgr.24473) 230 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-13T03:27:03.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:00.528799+0000 mgr.y (mgr.24473) 231 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-13T03:27:03.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:00.529426+0000 mgr.y (mgr.24473) 232 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-13T03:27:03.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:00.530146+0000 mgr.y (mgr.24473) 233 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-13T03:27:03.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:00.530826+0000 mgr.y (mgr.24473) 234 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-13T03:27:03.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: cluster 2024-02-13T03:27:00.697909+0000 mgr.y (mgr.24473) 235 : cluster [DBG] pgmap v174: 620 pgs: 72 creating+peering, 548 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 553 op/s 2024-02-13T03:27:03.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: cluster 2024-02-13T03:27:01.168256+0000 osd.2 (osd.2) 3 : cluster [DBG] 18.7 deep-scrub starts 2024-02-13T03:27:03.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: cluster 2024-02-13T03:27:01.211666+0000 osd.1 (osd.1) 3 : cluster [DBG] 18.3 deep-scrub starts 2024-02-13T03:27:03.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: cluster 2024-02-13T03:27:01.215340+0000 osd.1 (osd.1) 4 : cluster [DBG] 18.3 deep-scrub ok 2024-02-13T03:27:03.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: cluster 2024-02-13T03:27:01.259397+0000 osd.2 (osd.2) 4 : cluster [DBG] 18.7 deep-scrub ok 2024-02-13T03:27:03.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:01.570112+0000 mon.c (mon.2) 116 : audit [INF] from='client.? v1:172.21.15.161:0/953175453' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:01.612480+0000 mon.b (mon.1) 38 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:03.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:01.617514+0000 mon.c (mon.2) 117 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:03.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:01.977578+0000 mon.a (mon.0) 756 : audit [INF] from='client.25135 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:03.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:01.977896+0000 mon.a (mon.0) 757 : audit [INF] from='client.25130 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi161-61719-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:03.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: cluster 2024-02-13T03:27:01.978539+0000 mon.a (mon.0) 758 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-13T03:27:03.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:01.979246+0000 mon.c (mon.2) 118 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:01.981048+0000 mon.c (mon.2) 119 : audit [INF] from='client.? v1:172.21.15.161:0/1890051921' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:01.982475+0000 mon.a (mon.0) 759 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:03.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:01.986849+0000 mon.b (mon.1) 39 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:03.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: cluster 2024-02-13T03:27:02.420468+0000 osd.7 (osd.7) 3 : cluster [DBG] 18.2 deep-scrub starts 2024-02-13T03:27:03.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.422995+0000 mon.a (mon.0) 760 : audit [INF] from='client.25150 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-13T03:27:03.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: cluster 2024-02-13T03:27:02.424310+0000 osd.7 (osd.7) 4 : cluster [DBG] 18.2 deep-scrub ok 2024-02-13T03:27:03.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.424491+0000 mon.a (mon.0) 761 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:03.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.426526+0000 mon.a (mon.0) 762 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:03.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.426892+0000 mon.a (mon.0) 763 : audit [INF] from='client.25013 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.432259+0000 mon.b (mon.1) 40 : audit [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-13T03:27:03.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.432501+0000 mon.b (mon.1) 41 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:03.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.432708+0000 mon.b (mon.1) 42 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:03.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.439505+0000 mon.c (mon.2) 120 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:03.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: cluster 2024-02-13T03:27:02.563736+0000 osd.3 (osd.3) 3 : cluster [DBG] 18.1 deep-scrub starts 2024-02-13T03:27:03.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: cluster 2024-02-13T03:27:02.618418+0000 osd.3 (osd.3) 4 : cluster [DBG] 18.1 deep-scrub ok 2024-02-13T03:27:03.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: cluster 2024-02-13T03:27:02.698913+0000 mgr.y (mgr.24473) 236 : cluster [DBG] pgmap v176: 620 pgs: 96 unknown, 72 creating+peering, 452 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 9.5 MiB/s rd, 29 MiB/s wr, 431 op/s 2024-02-13T03:27:03.935 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: cluster 2024-02-13T03:27:02.855132+0000 client.admin (client.25150) 0 : cluster [INF] twoxx 2024-02-13T03:27:03.935 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.855813+0000 mon.a (mon.0) 764 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:03.935 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.862410+0000 mon.a (mon.0) 765 : audit [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi161-62012-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi161-62012-6"}]': finished 2024-02-13T03:27:03.935 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.862598+0000 mon.a (mon.0) 766 : audit [INF] from='client.25013 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:03.935 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: cluster 2024-02-13T03:27:02.864903+0000 mon.a (mon.0) 767 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-13T03:27:03.935 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.866258+0000 mon.c (mon.2) 121 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:03.935 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.868879+0000 mon.a (mon.0) 768 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.935 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.869801+0000 mon.c (mon.2) 122 : audit [INF] from='client.? v1:172.21.15.161:0/953175453' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi161-61726-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-13T03:27:03.935 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.870409+0000 mon.a (mon.0) 769 : audit [INF] from='client.25151 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.935 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.873809+0000 mon.a (mon.0) 770 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:03.935 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.874537+0000 mon.a (mon.0) 771 : audit [INF] from='client.25013 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi161-61726-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-13T03:27:03.935 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.877296+0000 mon.a (mon.0) 772 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:03.936 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.877429+0000 mon.c (mon.2) 123 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.936 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.877887+0000 mon.a (mon.0) 773 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.936 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.882341+0000 mon.a (mon.0) 774 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:03.936 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.882549+0000 mon.b (mon.1) 43 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:03.936 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.882639+0000 mon.a (mon.0) 775 : audit [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-13T03:27:03.936 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.882885+0000 mon.a (mon.0) 776 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:03.936 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.883141+0000 mon.a (mon.0) 777 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:03.936 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.883700+0000 mon.a (mon.0) 778 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:03.936 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.886561+0000 mon.a (mon.0) 779 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi161-61735-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.936 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.886647+0000 mon.b (mon.1) 44 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi161-61735-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.936 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.888053+0000 mon.a (mon.0) 780 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.936 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.888424+0000 mon.a (mon.0) 781 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.937 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.890245+0000 mon.b (mon.1) 45 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.937 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.890619+0000 mon.b (mon.1) 46 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:01.977578+0000 mon.a (mon.0) 756 : audit [INF] from='client.25135 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:03.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:01.977896+0000 mon.a (mon.0) 757 : audit [INF] from='client.25130 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi161-61719-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:03.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: cluster 2024-02-13T03:27:01.978539+0000 mon.a (mon.0) 758 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-13T03:27:03.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:01.979246+0000 mon.c (mon.2) 118 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:01.981048+0000 mon.c (mon.2) 119 : audit [INF] from='client.? v1:172.21.15.161:0/1890051921' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:01.982475+0000 mon.a (mon.0) 759 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:03.937 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:01.986849+0000 mon.b (mon.1) 39 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: cluster 2024-02-13T03:27:02.420468+0000 osd.7 (osd.7) 3 : cluster [DBG] 18.2 deep-scrub starts 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.422995+0000 mon.a (mon.0) 760 : audit [INF] from='client.25150 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: cluster 2024-02-13T03:27:02.424310+0000 osd.7 (osd.7) 4 : cluster [DBG] 18.2 deep-scrub ok 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.424491+0000 mon.a (mon.0) 761 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.426526+0000 mon.a (mon.0) 762 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.426892+0000 mon.a (mon.0) 763 : audit [INF] from='client.25013 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.432259+0000 mon.b (mon.1) 40 : audit [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.432501+0000 mon.b (mon.1) 41 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.432708+0000 mon.b (mon.1) 42 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.439505+0000 mon.c (mon.2) 120 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: cluster 2024-02-13T03:27:02.563736+0000 osd.3 (osd.3) 3 : cluster [DBG] 18.1 deep-scrub starts 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: cluster 2024-02-13T03:27:02.618418+0000 osd.3 (osd.3) 4 : cluster [DBG] 18.1 deep-scrub ok 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: cluster 2024-02-13T03:27:02.698913+0000 mgr.y (mgr.24473) 236 : cluster [DBG] pgmap v176: 620 pgs: 96 unknown, 72 creating+peering, 452 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 9.5 MiB/s rd, 29 MiB/s wr, 431 op/s 2024-02-13T03:27:03.938 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: cluster 2024-02-13T03:27:02.855132+0000 client.admin (client.25150) 0 : cluster [INF] twoxx 2024-02-13T03:27:03.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.855813+0000 mon.a (mon.0) 764 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:03.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.862410+0000 mon.a (mon.0) 765 : audit [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi161-62012-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi161-62012-6"}]': finished 2024-02-13T03:27:03.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.862598+0000 mon.a (mon.0) 766 : audit [INF] from='client.25013 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi161-61726-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:03.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: cluster 2024-02-13T03:27:02.864903+0000 mon.a (mon.0) 767 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-13T03:27:03.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.866258+0000 mon.c (mon.2) 121 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:03.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.868879+0000 mon.a (mon.0) 768 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.869801+0000 mon.c (mon.2) 122 : audit [INF] from='client.? v1:172.21.15.161:0/953175453' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi161-61726-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-13T03:27:03.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.870409+0000 mon.a (mon.0) 769 : audit [INF] from='client.25151 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:03.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.873809+0000 mon.a (mon.0) 770 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:03.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.874537+0000 mon.a (mon.0) 771 : audit [INF] from='client.25013 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi161-61726-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-13T03:27:03.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.877296+0000 mon.a (mon.0) 772 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:03.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.877429+0000 mon.c (mon.2) 123 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.877887+0000 mon.a (mon.0) 773 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.882341+0000 mon.a (mon.0) 774 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.882549+0000 mon.b (mon.1) 43 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.882639+0000 mon.a (mon.0) 775 : audit [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.882885+0000 mon.a (mon.0) 776 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.883141+0000 mon.a (mon.0) 777 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.883700+0000 mon.a (mon.0) 778 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.886561+0000 mon.a (mon.0) 779 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi161-61735-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.886647+0000 mon.b (mon.1) 44 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi161-61735-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.888053+0000 mon.a (mon.0) 780 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.888424+0000 mon.a (mon.0) 781 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.890245+0000 mon.b (mon.1) 45 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.890619+0000 mon.b (mon.1) 46 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.890958+0000 mon.b (mon.1) 47 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi161-61765-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.941 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[27879]: audit 2024-02-13T03:27:02.893279+0000 mon.a (mon.0) 782 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi161-61765-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.941 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.890958+0000 mon.b (mon.1) 47 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi161-61765-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:03.941 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:03 smithi161 bash[19769]: audit 2024-02-13T03:27:02.893279+0000 mon.a (mon.0) 782 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi161-61765-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:05.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: cluster 2024-02-13T03:27:03.417586+0000 mon.a (mon.0) 783 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:27:05.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.440431+0000 mon.c (mon.2) 124 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:05.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.444840+0000 mon.a (mon.0) 784 : audit [INF] from='client.25150 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-13T03:27:05.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.869562+0000 mon.a (mon.0) 785 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:05.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.869831+0000 mon.a (mon.0) 786 : audit [INF] from='client.25151 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:05.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.870016+0000 mon.a (mon.0) 787 : audit [INF] from='client.25013 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi161-61726-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-13T03:27:05.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.870138+0000 mon.a (mon.0) 788 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.870323+0000 mon.a (mon.0) 789 : audit [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi161-61735-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.870424+0000 mon.a (mon.0) 790 : audit [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.870563+0000 mon.a (mon.0) 791 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.870647+0000 mon.a (mon.0) 792 : audit [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi161-61765-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: cluster 2024-02-13T03:27:03.870772+0000 mon.a (mon.0) 793 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-13T03:27:05.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.874753+0000 mon.c (mon.2) 125 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:27:05.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.877785+0000 mon.b (mon.1) 48 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi161-61765-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:05.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: cluster 2024-02-13T03:27:03.417586+0000 mon.a (mon.0) 783 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:27:05.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.440431+0000 mon.c (mon.2) 124 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:05.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.444840+0000 mon.a (mon.0) 784 : audit [INF] from='client.25150 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-13T03:27:05.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.869562+0000 mon.a (mon.0) 785 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:05.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.869831+0000 mon.a (mon.0) 786 : audit [INF] from='client.25151 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:05.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.870016+0000 mon.a (mon.0) 787 : audit [INF] from='client.25013 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi161-61726-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-13T03:27:05.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.870138+0000 mon.a (mon.0) 788 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.870323+0000 mon.a (mon.0) 789 : audit [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi161-61735-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.870424+0000 mon.a (mon.0) 790 : audit [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.870563+0000 mon.a (mon.0) 791 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.870647+0000 mon.a (mon.0) 792 : audit [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi161-61765-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: cluster 2024-02-13T03:27:03.870772+0000 mon.a (mon.0) 793 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-13T03:27:05.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.874753+0000 mon.c (mon.2) 125 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:27:05.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.877785+0000 mon.b (mon.1) 48 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi161-61765-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:05.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.878196+0000 mon.c (mon.2) 126 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi161-61798-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:05.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.878306+0000 mon.b (mon.1) 49 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi161-61743-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:05.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.878543+0000 mon.b (mon.1) 50 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi161-62041-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:05.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.878871+0000 mon.b (mon.1) 51 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi161-61735-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:05.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.880357+0000 mon.c (mon.2) 127 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:05.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.882188+0000 mon.a (mon.0) 794 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi161-61765-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:05.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.882905+0000 mon.a (mon.0) 795 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi161-61798-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:05.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.883508+0000 mon.a (mon.0) 796 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi161-61743-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:05.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.883821+0000 mon.b (mon.1) 52 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:05.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.884040+0000 mon.a (mon.0) 797 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:05.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.884376+0000 mon.a (mon.0) 798 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi161-62041-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:05.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.878196+0000 mon.c (mon.2) 126 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi161-61798-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:05.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.878306+0000 mon.b (mon.1) 49 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi161-61743-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:05.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.878543+0000 mon.b (mon.1) 50 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi161-62041-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:05.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.878871+0000 mon.b (mon.1) 51 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi161-61735-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:05.182 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.880357+0000 mon.c (mon.2) 127 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:05.182 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.882188+0000 mon.a (mon.0) 794 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi161-61765-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:05.182 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.882905+0000 mon.a (mon.0) 795 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi161-61798-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:05.182 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.883508+0000 mon.a (mon.0) 796 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi161-61743-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:05.182 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.883821+0000 mon.b (mon.1) 52 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:05.182 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.884040+0000 mon.a (mon.0) 797 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:05.182 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.884376+0000 mon.a (mon.0) 798 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi161-62041-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:05.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.884929+0000 mon.a (mon.0) 799 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi161-61735-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:05.182 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.898336+0000 mon.a (mon.0) 800 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:05.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.898355+0000 mon.c (mon.2) 128 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:05.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:03.898975+0000 mon.a (mon.0) 801 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:05.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:04.248185+0000 mon.c (mon.2) 129 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:27:05.183 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[27879]: audit 2024-02-13T03:27:04.249630+0000 mon.c (mon.2) 130 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:27:05.183 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.884929+0000 mon.a (mon.0) 799 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi161-61735-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:05.183 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.898336+0000 mon.a (mon.0) 800 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:05.183 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.898355+0000 mon.c (mon.2) 128 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:05.183 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:03.898975+0000 mon.a (mon.0) 801 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:05.183 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:04.248185+0000 mon.c (mon.2) 129 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:27:05.183 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:04 smithi161 bash[19769]: audit 2024-02-13T03:27:04.249630+0000 mon.c (mon.2) 130 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:27:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: cluster 2024-02-13T03:27:03.417586+0000 mon.a (mon.0) 783 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:27:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.440431+0000 mon.c (mon.2) 124 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.444840+0000 mon.a (mon.0) 784 : audit [INF] from='client.25150 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-13T03:27:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.869562+0000 mon.a (mon.0) 785 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.869831+0000 mon.a (mon.0) 786 : audit [INF] from='client.25151 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.870016+0000 mon.a (mon.0) 787 : audit [INF] from='client.25013 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi161-61726-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.870138+0000 mon.a (mon.0) 788 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.870323+0000 mon.a (mon.0) 789 : audit [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi161-61735-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.870424+0000 mon.a (mon.0) 790 : audit [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.870563+0000 mon.a (mon.0) 791 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.870647+0000 mon.a (mon.0) 792 : audit [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi161-61765-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: cluster 2024-02-13T03:27:03.870772+0000 mon.a (mon.0) 793 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.874753+0000 mon.c (mon.2) 125 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.877785+0000 mon.b (mon.1) 48 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi161-61765-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.878196+0000 mon.c (mon.2) 126 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi161-61798-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.878306+0000 mon.b (mon.1) 49 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi161-61743-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.878543+0000 mon.b (mon.1) 50 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi161-62041-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.878871+0000 mon.b (mon.1) 51 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi161-61735-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.880357+0000 mon.c (mon.2) 127 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.882188+0000 mon.a (mon.0) 794 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi161-61765-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:05.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.882905+0000 mon.a (mon.0) 795 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi161-61798-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:05.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.883508+0000 mon.a (mon.0) 796 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi161-61743-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:05.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.883821+0000 mon.b (mon.1) 52 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:05.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.884040+0000 mon.a (mon.0) 797 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:05.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.884376+0000 mon.a (mon.0) 798 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi161-62041-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:05.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.884929+0000 mon.a (mon.0) 799 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi161-61735-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:05.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.898336+0000 mon.a (mon.0) 800 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:05.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.898355+0000 mon.c (mon.2) 128 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:05.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:03.898975+0000 mon.a (mon.0) 801 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:05.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:04.248185+0000 mon.c (mon.2) 129 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:27:05.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:04 smithi183 bash[20824]: audit 2024-02-13T03:27:04.249630+0000 mon.c (mon.2) 130 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:27:05.448 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.527981+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-13T03:27:05.448 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.528627+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-13T03:27:05.448 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.529264+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-13T03:27:05.448 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.529932+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-13T03:27:05.448 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.530631+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-13T03:27:05.448 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:00.680984+0000 mon.a [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi161-62012-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:05.448 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:01.107778+0000 mon.a [INF] from='client.25135 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:01.108074+0000 mon.a [INF] from='client.25130 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi161-61719-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:01.543738+0000 mon.a [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.417586+0000 mon.a [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.444840+0000 mon.a [INF] from='client.25150 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.869562+0000 mon.a [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.869831+0000 mon.a [INF] from='client.25151 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.870016+0000 mon.a [INF] from='client.25013 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi161-61726-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.870138+0000 mon.a [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.870323+0000 mon.a [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi161-61735-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.870424+0000 mon.a [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.870563+0000 mon.a [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.870647+0000 mon.a [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi161-61765-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.877785+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi161-61765-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:05.449 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.878196+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi161-61798-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:05.450 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.878306+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi161-61743-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:05.450 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.878543+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi161-62041-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:05.450 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.878871+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi161-61735-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:05.450 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.880357+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:05.450 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.882188+0000 mon.a [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi161-61765-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:05.450 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.882905+0000 mon.a [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi161-61798-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:05.450 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.883508+0000 mon.a [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi161-61743-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:05.450 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.883821+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:06.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:27:05 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:27:07.000 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.884040+0000 mon.a [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache delete_pools_parallel: process_1_[62388]: starting. 2024-02-13T03:27:07.000 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_1_[62388]: creating pool ceph_test_rados_delete_pools_parallel.smithi161-62334 2024-02-13T03:27:07.000 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_1_[62388]: created object 0... 2024-02-13T03:27:07.000 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_1_[62388]: created object 25... 2024-02-13T03:27:07.000 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_1_[62388]: created object 49... 2024-02-13T03:27:07.000 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_1_[62388]: finishing. 2024-02-13T03:27:07.000 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_1_[62388]: shutting down. 2024-02-13T03:27:07.000 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_2_[62389]: starting. 2024-02-13T03:27:07.000 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_2_[62389]: deleting pool ceph_test_rados_delete_pools_parallel.smithi161-62334 2024-02-13T03:27:07.000 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_2_[62389]: shutting down. 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: ******************************* 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_3_[63280]: starting. 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_3_[63280]: creating pool ceph_test_rados_delete_pools_parallel.smithi161-62334 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_3_[63280]: created object 0... 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_3_[63280]: created object 25... 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_3_[63280]: created object 49... 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_3_[63280]: finishing. 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_3_[63280]: shutting down. 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: ******************************* 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_5_[63282]: starting. 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_5_[63282]: listing objects. 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_5_[63282]: listed object 0... 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_5_[63282]: listed object 25... 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_5_[63282]: saw 50 objects 2024-02-13T03:27:07.001 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_5_[63282]: shutting down. 2024-02-13T03:27:07.002 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: ******************************* 2024-02-13T03:27:07.002 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_4_[63281]: starting. 2024-02-13T03:27:07.002 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_4_[63281]: deleting pool ceph_test_rados_delete_pools_parallel.smithi161-62334 2024-02-13T03:27:07.002 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: process_4_[63281]: shutting down. 2024-02-13T03:27:07.002 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: ******************************* 2024-02-13T03:27:07.002 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: ******************************* 2024-02-13T03:27:07.002 INFO:tasks.workunit.client.0.smithi161.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-02-13T03:27:07.002 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_1_[62288]: starting. 2024-02-13T03:27:07.002 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_1_[62288]: creating pool ceph_test_rados_open_pools_parallel.smithi161-62275 2024-02-13T03:27:07.002 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_1_[62288]: created object 0... 2024-02-13T03:27:07.003 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_1_[62288]: created object 25... 2024-02-13T03:27:07.003 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_1_[62288]: created object 49... 2024-02-13T03:27:07.003 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_1_[62288]: finishing. 2024-02-13T03:27:07.003 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_1_[62288]: shutting down. 2024-02-13T03:27:07.003 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_2_[62289]: starting. 2024-02-13T03:27:07.003 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_2_[62289]: rados_pool_create. 2024-02-13T03:27:07.003 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_2_[62289]: rados_ioctx_create. 2024-02-13T03:27:07.003 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_2_[62289]: shutting down. 2024-02-13T03:27:07.003 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: ******************************* 2024-02-13T03:27:07.003 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_3_[63278]: starting. 2024-02-13T03:27:07.003 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_3_[63278]: creating pool ceph_test_rados_open_pools_parallel.smithi161-62275 2024-02-13T03:27:07.003 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_3_[63278]: created object 0... 2024-02-13T03:27:07.003 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_3_[63278]: created object 25... 2024-02-13T03:27:07.004 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_3_[63278]: created object 49... 2024-02-13T03:27:07.004 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_3_[63278]: finishing. 2024-02-13T03:27:07.004 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_3_[63278]: shutting down. 2024-02-13T03:27:07.004 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: ******************************* 2024-02-13T03:27:07.004 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_4_[63279]: starting. 2024-02-13T03:27:07.004 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_4_[63279]: rados_pool_create. 2024-02-13T03:27:07.004 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_4_[63279]: rados_ioctx_create. 2024-02-13T03:27:07.004 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: process_4_[63279]: shutting down. 2024-02-13T03:27:07.004 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: ******************************* 2024-02-13T03:27:07.004 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: ******************************* 2024-02-13T03:27:07.004 INFO:tasks.workunit.client.0.smithi161.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-02-13T03:27:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:07 smithi161 bash[27879]: cluster 2024-02-13T03:27:03.609239+0000 osd.3 (osd.3) 5 : cluster [DBG] 18.9 deep-scrub starts 2024-02-13T03:27:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:07 smithi161 bash[27879]: cluster 2024-02-13T03:27:03.612690+0000 osd.3 (osd.3) 6 : cluster [DBG] 18.9 deep-scrub ok 2024-02-13T03:27:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:07 smithi161 bash[27879]: cluster 2024-02-13T03:27:04.253222+0000 mgr.y (mgr.24473) 237 : cluster [DBG] pgmap v179: 620 pgs: 200 unknown, 420 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 535 op/s 2024-02-13T03:27:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:07 smithi161 bash[27879]: audit 2024-02-13T03:27:04.441438+0000 mon.c (mon.2) 131 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:07 smithi161 bash[27879]: audit 2024-02-13T03:27:04.824078+0000 mon.a (mon.0) 802 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:07 smithi161 bash[27879]: audit 2024-02-13T03:27:04.827406+0000 mon.a (mon.0) 803 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:07.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:07 smithi161 bash[19769]: cluster 2024-02-13T03:27:03.609239+0000 osd.3 (osd.3) 5 : cluster [DBG] 18.9 deep-scrub starts 2024-02-13T03:27:07.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:07 smithi161 bash[19769]: cluster 2024-02-13T03:27:03.612690+0000 osd.3 (osd.3) 6 : cluster [DBG] 18.9 deep-scrub ok 2024-02-13T03:27:07.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:07 smithi161 bash[19769]: cluster 2024-02-13T03:27:04.253222+0000 mgr.y (mgr.24473) 237 : cluster [DBG] pgmap v179: 620 pgs: 200 unknown, 420 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 535 op/s 2024-02-13T03:27:07.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:07 smithi161 bash[19769]: audit 2024-02-13T03:27:04.441438+0000 mon.c (mon.2) 131 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:07.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:07 smithi161 bash[19769]: audit 2024-02-13T03:27:04.824078+0000 mon.a (mon.0) 802 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:07.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:07 smithi161 bash[19769]: audit 2024-02-13T03:27:04.827406+0000 mon.a (mon.0) 803 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:07.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:07 smithi183 bash[20824]: cluster 2024-02-13T03:27:03.609239+0000 osd.3 (osd.3) 5 : cluster [DBG] 18.9 deep-scrub starts 2024-02-13T03:27:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:07 smithi183 bash[20824]: cluster 2024-02-13T03:27:03.612690+0000 osd.3 (osd.3) 6 : cluster [DBG] 18.9 deep-scrub ok 2024-02-13T03:27:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:07 smithi183 bash[20824]: cluster 2024-02-13T03:27:04.253222+0000 mgr.y (mgr.24473) 237 : cluster [DBG] pgmap v179: 620 pgs: 200 unknown, 420 active+clean; 144 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 535 op/s 2024-02-13T03:27:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:07 smithi183 bash[20824]: audit 2024-02-13T03:27:04.441438+0000 mon.c (mon.2) 131 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:07 smithi183 bash[20824]: audit 2024-02-13T03:27:04.824078+0000 mon.a (mon.0) 802 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:07 smithi183 bash[20824]: audit 2024-02-13T03:27:04.827406+0000 mon.a (mon.0) 803 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:07.588 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: Running main() from gmock_main.cc 2024-02-13T03:27:07.588 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-02-13T03:27:07.588 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [----------] Global test environment set-up. 2024-02-13T03:27:07.588 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-02-13T03:27:07.588 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: seed 61743 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (269 ms) 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (10 ms) 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (13 ms) 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (4 ms) 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (5 ms) 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-02-13T03:27:07.589 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (5 ms) 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (3 ms) 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (6 ms) 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (5 ms) 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (5 ms) 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (5 ms) 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (5 ms) 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (5 ms) 2024-02-13T03:27:07.590 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (6 ms) 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (12 ms) 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (7 ms) 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (7 ms) 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (4 ms) 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (5 ms) 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (382 ms total) 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-02-13T03:27:07.591 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (537 ms) 2024-02-13T03:27:07.592 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-02-13T03:27:07.592 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (14 ms) 2024-02-13T03:27:07.592 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-02-13T03:27:07.592 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-02-13T03:27:07.592 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-02-13T03:27:07.592 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (5 ms) 2024-02-13T03:27:07.592 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-02-13T03:27:07.592 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (5 ms) 2024-02-13T03:27:08.022 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ]"}]: dispatch 2024-02-13T03:27:08.022 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.884376+0000 mon.a [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi161-62041-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:08.022 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.884929+0000 mon.a [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi161-61735-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:08.022 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.898336+0000 mon.a [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:08.022 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.898355+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.022 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:03.898975+0000 mon.a [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.022 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:04.249630+0000 mon.c [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:27:08.022 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:04.824078+0000 mon.a [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:08.022 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:04.827406+0000 mon.a [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:08.022 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:05.446426+0000 client.admin [INF] threexx 2024-02-13T03:27:08.022 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:05.449319+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-13T03:27:08.023 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:06.151079+0000 mon.a [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]': finished 2024-02-13T03:27:08.023 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:06.152043+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-4", "overlaypool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:08.023 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:06.991873+0000 mon.a [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]': finished 2024-02-13T03:27:08.023 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:06.992130+0000 mon.a [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:08.023 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:06.994092+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/671714200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi161-61719-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.023 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:06.994492+0000 mon.a [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-13T03:27:08.023 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:06.996229+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:08.023 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:06.996539+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.023 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:06.997568+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/2599947766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.023 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.005912+0000 mon.a [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-4", "overlaypool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:08.023 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.007707+0000 mon.a [INF] from='client.25208 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi161-61719-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.023 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.007917+0000 mon.a [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.023 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.014718+0000 mon.a [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi161-61765-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi161-61765-10"}]': finished 2024-02-13T03:27:08.024 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.014814+0000 mon.a [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi161-61798-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi161-61798-10"}]': finished 2024-02-13T03:27:08.024 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.014888+0000 mon.a [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi161-61743-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi161-61743-22"}]': finished 2024-02-13T03:27:08.024 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.014953+0000 mon.a [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi161-62041-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi161-62041-7"}]': finished 2024-02-13T03:27:08.024 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.015028+0000 mon.a [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi161-61735-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi161-61735-15"}]': finished 2024-02-13T03:27:08.024 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.021385+0000 mon.a [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-13T03:27:08.024 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.021686+0000 mon.a [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:08.024 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.022396+0000 mon.a [INF] from='client.25216 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.024 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.022592+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.024 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.023159+0000 mon.a [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: cluster 2024-02-13T03:27:04.147591+0000 osd.0 (osd.0) 3 : cluster [DBG] 18.0 deep-scrub starts 2024-02-13T03:27:08.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: cluster 2024-02-13T03:27:04.151133+0000 osd.0 (osd.0) 4 : cluster [DBG] 18.0 deep-scrub ok 2024-02-13T03:27:08.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: cluster 2024-02-13T03:27:04.412141+0000 osd.7 (osd.7) 5 : cluster [DBG] 18.6 deep-scrub starts 2024-02-13T03:27:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: cluster 2024-02-13T03:27:04.415322+0000 osd.7 (osd.7) 6 : cluster [DBG] 18.6 deep-scrub ok 2024-02-13T03:27:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:05.442897+0000 mon.c (mon.2) 132 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: cluster 2024-02-13T03:27:05.446426+0000 client.admin (client.25150) 0 : cluster [INF] threexx 2024-02-13T03:27:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:05.449319+0000 mon.b (mon.1) 53 : audit [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-13T03:27:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:05.808312+0000 mgr.y (mgr.24473) 238 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:06.151079+0000 mon.a (mon.0) 804 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]': finished 2024-02-13T03:27:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:06.152043+0000 mon.c (mon.2) 133 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-4", "overlaypool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: cluster 2024-02-13T03:27:06.257438+0000 mgr.y (mgr.24473) 239 : cluster [DBG] pgmap v181: 580 pgs: 37 creating+peering, 27 unknown, 516 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.3 KiB/s wr, 2 op/s 2024-02-13T03:27:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:06.444368+0000 mon.c (mon.2) 134 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:06.991873+0000 mon.a (mon.0) 805 : audit [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]': finished 2024-02-13T03:27:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:06.992130+0000 mon.a (mon.0) 806 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: cluster 2024-02-13T03:27:06.993277+0000 mon.a (mon.0) 807 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-13T03:27:08.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:06.994092+0000 mon.c (mon.2) 135 : audit [INF] from='client.? v1:172.21.15.161:0/671714200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi161-61719-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: cluster 2024-02-13T03:27:06.994492+0000 mon.a (mon.0) 808 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-13T03:27:08.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:06.996229+0000 mon.b (mon.1) 54 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:08.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: cluster 2024-02-13T03:27:04.147591+0000 osd.0 (osd.0) 3 : cluster [DBG] 18.0 deep-scrub starts 2024-02-13T03:27:08.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: cluster 2024-02-13T03:27:04.151133+0000 osd.0 (osd.0) 4 : cluster [DBG] 18.0 deep-scrub ok 2024-02-13T03:27:08.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: cluster 2024-02-13T03:27:04.412141+0000 osd.7 (osd.7) 5 : cluster [DBG] 18.6 deep-scrub starts 2024-02-13T03:27:08.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: cluster 2024-02-13T03:27:04.415322+0000 osd.7 (osd.7) 6 : cluster [DBG] 18.6 deep-scrub ok 2024-02-13T03:27:08.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:05.442897+0000 mon.c (mon.2) 132 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:08.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: cluster 2024-02-13T03:27:05.446426+0000 client.admin (client.25150) 0 : cluster [INF] threexx 2024-02-13T03:27:08.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:05.449319+0000 mon.b (mon.1) 53 : audit [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-13T03:27:08.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:05.808312+0000 mgr.y (mgr.24473) 238 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:08.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:06.151079+0000 mon.a (mon.0) 804 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]': finished 2024-02-13T03:27:08.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:06.152043+0000 mon.c (mon.2) 133 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-4", "overlaypool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:08.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: cluster 2024-02-13T03:27:06.257438+0000 mgr.y (mgr.24473) 239 : cluster [DBG] pgmap v181: 580 pgs: 37 creating+peering, 27 unknown, 516 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.3 KiB/s wr, 2 op/s 2024-02-13T03:27:08.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:06.444368+0000 mon.c (mon.2) 134 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:08.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:06.991873+0000 mon.a (mon.0) 805 : audit [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]': finished 2024-02-13T03:27:08.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:06.992130+0000 mon.a (mon.0) 806 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:08.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: cluster 2024-02-13T03:27:06.993277+0000 mon.a (mon.0) 807 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-13T03:27:08.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:06.994092+0000 mon.c (mon.2) 135 : audit [INF] from='client.? v1:172.21.15.161:0/671714200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi161-61719-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: cluster 2024-02-13T03:27:06.994492+0000 mon.a (mon.0) 808 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-13T03:27:08.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:06.996229+0000 mon.b (mon.1) 54 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:08.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:06.996539+0000 mon.c (mon.2) 136 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:06.997568+0000 mon.b (mon.1) 55 : audit [INF] from='client.? v1:172.21.15.161:0/2599947766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.005912+0000 mon.a (mon.0) 809 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-4", "overlaypool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:08.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.007707+0000 mon.a (mon.0) 810 : audit [INF] from='client.25208 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi161-61719-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.007917+0000 mon.a (mon.0) 811 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.433 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.014718+0000 mon.a (mon.0) 812 : audit [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi161-61765-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi161-61765-10"}]': finished 2024-02-13T03:27:08.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.014814+0000 mon.a (mon.0) 813 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi161-61798-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi161-61798-10"}]': finished 2024-02-13T03:27:08.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.014888+0000 mon.a (mon.0) 814 : audit [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi161-61743-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi161-61743-22"}]': finished 2024-02-13T03:27:08.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.014953+0000 mon.a (mon.0) 815 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi161-62041-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi161-62041-7"}]': finished 2024-02-13T03:27:08.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.015028+0000 mon.a (mon.0) 816 : audit [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi161-61735-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi161-61735-15"}]': finished 2024-02-13T03:27:08.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: cluster 2024-02-13T03:27:07.016184+0000 mon.a (mon.0) 817 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-13T03:27:08.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.021385+0000 mon.a (mon.0) 818 : audit [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-13T03:27:08.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.021686+0000 mon.a (mon.0) 819 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:08.434 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.021940+0000 mon.c (mon.2) 137 : audit [DBG] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-13T03:27:08.434 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:06.996539+0000 mon.c (mon.2) 136 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.434 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:06.997568+0000 mon.b (mon.1) 55 : audit [INF] from='client.? v1:172.21.15.161:0/2599947766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.434 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.005912+0000 mon.a (mon.0) 809 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-4", "overlaypool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:08.434 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.007707+0000 mon.a (mon.0) 810 : audit [INF] from='client.25208 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi161-61719-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.435 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.007917+0000 mon.a (mon.0) 811 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.435 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.014718+0000 mon.a (mon.0) 812 : audit [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi161-61765-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi161-61765-10"}]': finished 2024-02-13T03:27:08.435 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.014814+0000 mon.a (mon.0) 813 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi161-61798-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi161-61798-10"}]': finished 2024-02-13T03:27:08.435 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.014888+0000 mon.a (mon.0) 814 : audit [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi161-61743-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi161-61743-22"}]': finished 2024-02-13T03:27:08.435 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.014953+0000 mon.a (mon.0) 815 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi161-62041-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi161-62041-7"}]': finished 2024-02-13T03:27:08.435 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.015028+0000 mon.a (mon.0) 816 : audit [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi161-61735-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi161-61735-15"}]': finished 2024-02-13T03:27:08.435 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: cluster 2024-02-13T03:27:07.016184+0000 mon.a (mon.0) 817 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-13T03:27:08.435 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.021385+0000 mon.a (mon.0) 818 : audit [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-13T03:27:08.435 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.021686+0000 mon.a (mon.0) 819 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:08.435 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.021940+0000 mon.c (mon.2) 137 : audit [DBG] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-13T03:27:08.435 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.022396+0000 mon.a (mon.0) 820 : audit [INF] from='client.25216 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.022592+0000 mon.c (mon.2) 138 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.023159+0000 mon.a (mon.0) 821 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.445940+0000 mon.c (mon.2) 139 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:08.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.478591+0000 mon.a (mon.0) 822 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-4", "overlaypool": "test-rados-api-smithi161-61906-4-cache"}]': finished 2024-02-13T03:27:08.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.478729+0000 mon.a (mon.0) 823 : audit [INF] from='client.25208 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi161-61719-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:08.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.478822+0000 mon.a (mon.0) 824 : audit [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]': finished 2024-02-13T03:27:08.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.478938+0000 mon.a (mon.0) 825 : audit [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]': finished 2024-02-13T03:27:08.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.479023+0000 mon.a (mon.0) 826 : audit [INF] from='client.25216 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:08.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.479135+0000 mon.a (mon.0) 827 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:08.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: cluster 2024-02-13T03:27:07.479380+0000 mon.a (mon.0) 828 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-13T03:27:08.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.479867+0000 mon.c (mon.2) 140 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.480099+0000 mon.c (mon.2) 141 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-4-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:27:08.436 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.480998+0000 mon.c (mon.2) 142 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2"}]: dispatch 2024-02-13T03:27:08.437 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.022396+0000 mon.a (mon.0) 820 : audit [INF] from='client.25216 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.437 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.022592+0000 mon.c (mon.2) 138 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.437 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.023159+0000 mon.a (mon.0) 821 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.437 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.445940+0000 mon.c (mon.2) 139 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:08.437 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.478591+0000 mon.a (mon.0) 822 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-4", "overlaypool": "test-rados-api-smithi161-61906-4-cache"}]': finished 2024-02-13T03:27:08.437 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.478729+0000 mon.a (mon.0) 823 : audit [INF] from='client.25208 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi161-61719-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:08.437 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.478822+0000 mon.a (mon.0) 824 : audit [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]': finished 2024-02-13T03:27:08.437 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.478938+0000 mon.a (mon.0) 825 : audit [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]': finished 2024-02-13T03:27:08.437 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.479023+0000 mon.a (mon.0) 826 : audit [INF] from='client.25216 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:08.437 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.479135+0000 mon.a (mon.0) 827 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:08.437 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: cluster 2024-02-13T03:27:07.479380+0000 mon.a (mon.0) 828 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-13T03:27:08.437 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.479867+0000 mon.c (mon.2) 140 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.438 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.480099+0000 mon.c (mon.2) 141 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-4-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:27:08.438 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.480998+0000 mon.c (mon.2) 142 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2"}]: dispatch 2024-02-13T03:27:08.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.481756+0000 mon.c (mon.2) 143 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.482577+0000 mon.a (mon.0) 829 : audit [INF] from='client.25222 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.482884+0000 mon.a (mon.0) 830 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.483204+0000 mon.a (mon.0) 831 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-4-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:27:08.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.483418+0000 mon.a (mon.0) 832 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.438 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[19769]: audit 2024-02-13T03:27:07.484559+0000 mon.b (mon.1) 56 : audit [INF] from='client.? v1:172.21.15.161:0/2325996547' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.439 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.481756+0000 mon.c (mon.2) 143 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.439 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.482577+0000 mon.a (mon.0) 829 : audit [INF] from='client.25222 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.439 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.482884+0000 mon.a (mon.0) 830 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.439 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.483204+0000 mon.a (mon.0) 831 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-4-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:27:08.439 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.483418+0000 mon.a (mon.0) 832 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.439 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:08 smithi161 bash[27879]: audit 2024-02-13T03:27:07.484559+0000 mon.b (mon.1) 56 : audit [INF] from='client.? v1:172.21.15.161:0/2325996547' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: cluster 2024-02-13T03:27:04.147591+0000 osd.0 (osd.0) 3 : cluster [DBG] 18.0 deep-scrub starts 2024-02-13T03:27:08.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: cluster 2024-02-13T03:27:04.151133+0000 osd.0 (osd.0) 4 : cluster [DBG] 18.0 deep-scrub ok 2024-02-13T03:27:08.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: cluster 2024-02-13T03:27:04.412141+0000 osd.7 (osd.7) 5 : cluster [DBG] 18.6 deep-scrub starts 2024-02-13T03:27:08.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: cluster 2024-02-13T03:27:04.415322+0000 osd.7 (osd.7) 6 : cluster [DBG] 18.6 deep-scrub ok 2024-02-13T03:27:08.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:05.442897+0000 mon.c (mon.2) 132 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:08.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: cluster 2024-02-13T03:27:05.446426+0000 client.admin (client.25150) 0 : cluster [INF] threexx 2024-02-13T03:27:08.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:05.449319+0000 mon.b (mon.1) 53 : audit [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-13T03:27:08.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:05.808312+0000 mgr.y (mgr.24473) 238 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:08.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:06.151079+0000 mon.a (mon.0) 804 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]': finished 2024-02-13T03:27:08.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:06.152043+0000 mon.c (mon.2) 133 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-4", "overlaypool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:08.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: cluster 2024-02-13T03:27:06.257438+0000 mgr.y (mgr.24473) 239 : cluster [DBG] pgmap v181: 580 pgs: 37 creating+peering, 27 unknown, 516 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.3 KiB/s wr, 2 op/s 2024-02-13T03:27:08.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:06.444368+0000 mon.c (mon.2) 134 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:08.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:06.991873+0000 mon.a (mon.0) 805 : audit [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi161-62096-1"}]': finished 2024-02-13T03:27:08.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:06.992130+0000 mon.a (mon.0) 806 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:08.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: cluster 2024-02-13T03:27:06.993277+0000 mon.a (mon.0) 807 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-13T03:27:08.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:06.994092+0000 mon.c (mon.2) 135 : audit [INF] from='client.? v1:172.21.15.161:0/671714200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi161-61719-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: cluster 2024-02-13T03:27:06.994492+0000 mon.a (mon.0) 808 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-13T03:27:08.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:06.996229+0000 mon.b (mon.1) 54 : audit [INF] from='client.? v1:172.21.15.161:0/3238656429' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:08.489 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.478591+0000 mon.a [INF] from='cl api_stat: Running main() from gmock_main.cc 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [==========] Running 8 tests from 2 test suites. 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [----------] Global test environment set-up. 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [----------] 4 tests from LibRadosStat 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ OK ] LibRadosStat.Stat (260 ms) 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ OK ] LibRadosStat.StatNS (25 ms) 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (3 ms) 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [----------] 4 tests from LibRadosStat (288 ms total) 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-02-13T03:27:08.490 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-02-13T03:27:08.491 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1118 ms) 2024-02-13T03:27:08.491 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-02-13T03:27:08.491 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (8 ms) 2024-02-13T03:27:08.491 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-02-13T03:27:08.491 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (0 ms) 2024-02-13T03:27:08.491 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-02-13T03:27:08.491 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (2 ms) 2024-02-13T03:27:08.491 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1128 ms total) 2024-02-13T03:27:08.491 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: 2024-02-13T03:27:08.491 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [----------] Global test environment tear-down 2024-02-13T03:27:08.491 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [==========] 8 tests from 2 test suites ran. (22154 ms total) 2024-02-13T03:27:08.491 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat: [ PASSED ] 8 tests. 2024-02-13T03:27:08.492 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:06.996539+0000 mon.c (mon.2) 136 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.492 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:06.997568+0000 mon.b (mon.1) 55 : audit [INF] from='client.? v1:172.21.15.161:0/2599947766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.492 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.005912+0000 mon.a (mon.0) 809 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-4", "overlaypool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:08.492 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.007707+0000 mon.a (mon.0) 810 : audit [INF] from='client.25208 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi161-61719-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.492 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.007917+0000 mon.a (mon.0) 811 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.492 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.014718+0000 mon.a (mon.0) 812 : audit [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi161-61765-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi161-61765-10"}]': finished 2024-02-13T03:27:08.492 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.014814+0000 mon.a (mon.0) 813 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi161-61798-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi161-61798-10"}]': finished 2024-02-13T03:27:08.492 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.014888+0000 mon.a (mon.0) 814 : audit [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi161-61743-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi161-61743-22"}]': finished 2024-02-13T03:27:08.492 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.014953+0000 mon.a (mon.0) 815 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi161-62041-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi161-62041-7"}]': finished 2024-02-13T03:27:08.492 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.015028+0000 mon.a (mon.0) 816 : audit [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi161-61735-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi161-61735-15"}]': finished 2024-02-13T03:27:08.492 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: cluster 2024-02-13T03:27:07.016184+0000 mon.a (mon.0) 817 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.021385+0000 mon.a (mon.0) 818 : audit [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.021686+0000 mon.a (mon.0) 819 : audit [INF] from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]: dispatch 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.021940+0000 mon.c (mon.2) 137 : audit [DBG] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.022396+0000 mon.a (mon.0) 820 : audit [INF] from='client.25216 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.022592+0000 mon.c (mon.2) 138 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.023159+0000 mon.a (mon.0) 821 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.445940+0000 mon.c (mon.2) 139 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.478591+0000 mon.a (mon.0) 822 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-4", "overlaypool": "test-rados-api-smithi161-61906-4-cache"}]': finished 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.478729+0000 mon.a (mon.0) 823 : audit [INF] from='client.25208 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi161-61719-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.478822+0000 mon.a (mon.0) 824 : audit [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]': finished 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.478938+0000 mon.a (mon.0) 825 : audit [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]': finished 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.479023+0000 mon.a (mon.0) 826 : audit [INF] from='client.25216 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.479135+0000 mon.a (mon.0) 827 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:08.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: cluster 2024-02-13T03:27:07.479380+0000 mon.a (mon.0) 828 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-13T03:27:08.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.479867+0000 mon.c (mon.2) 140 : audit [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.480099+0000 mon.c (mon.2) 141 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-4-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:27:08.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.480998+0000 mon.c (mon.2) 142 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2"}]: dispatch 2024-02-13T03:27:08.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.481756+0000 mon.c (mon.2) 143 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.482577+0000 mon.a (mon.0) 829 : audit [INF] from='client.25222 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.482884+0000 mon.a (mon.0) 830 : audit [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:08.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.483204+0000 mon.a (mon.0) 831 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-4-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:27:08.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.483418+0000 mon.a (mon.0) 832 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:08.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:08 smithi183 bash[20824]: audit 2024-02-13T03:27:07.484559+0000 mon.b (mon.1) 56 : audit [INF] from='client.? v1:172.21.15.161:0/2325996547' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:09.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:27:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:27:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: cluster 2024-02-13T03:27:07.168383+0000 osd.0 (osd.0) 5 : cluster [DBG] 18.4 deep-scrub starts 2024-02-13T03:27:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: cluster 2024-02-13T03:27:07.172895+0000 osd.0 (osd.0) 6 : cluster [DBG] 18.4 deep-scrub ok 2024-02-13T03:27:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: cluster 2024-02-13T03:27:07.362911+0000 osd.1 (osd.1) 5 : cluster [DBG] 18.5 deep-scrub starts 2024-02-13T03:27:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: cluster 2024-02-13T03:27:07.366864+0000 osd.1 (osd.1) 6 : cluster [DBG] 18.5 deep-scrub ok 2024-02-13T03:27:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.021258+0000 mon.a (mon.0) 833 : audit [INF] from='client.25150 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-13T03:27:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: cluster 2024-02-13T03:27:08.023080+0000 client.admin (client.25150) 0 : cluster [INF] fourxx 2024-02-13T03:27:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.023323+0000 mon.a (mon.0) 834 : audit [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-13T03:27:09.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.026159+0000 mon.b (mon.1) 57 : audit [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-13T03:27:09.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: cluster 2024-02-13T03:27:08.258753+0000 mgr.y (mgr.24473) 240 : cluster [DBG] pgmap v184: 652 pgs: 37 creating+peering, 99 unknown, 516 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-13T03:27:09.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.447770+0000 mon.c (mon.2) 144 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:09.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: cluster 2024-02-13T03:27:08.469694+0000 mon.a (mon.0) 835 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:09.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.477916+0000 mon.a (mon.0) 836 : audit [INF] from='client.25222 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:09.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.478012+0000 mon.a (mon.0) 837 : audit [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]': finished 2024-02-13T03:27:09.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.478095+0000 mon.a (mon.0) 838 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-4-cache", "mode": "writeback"}]': finished 2024-02-13T03:27:09.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.478174+0000 mon.a (mon.0) 839 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:09.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: cluster 2024-02-13T03:27:08.478934+0000 mon.a (mon.0) 840 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-13T03:27:09.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.482751+0000 mon.c (mon.2) 145 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:09.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.483188+0000 mon.c (mon.2) 146 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-13T03:27:09.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.485974+0000 mon.c (mon.2) 147 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:27:09.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.490062+0000 mon.b (mon.1) 58 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:09.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.491548+0000 mon.b (mon.1) 59 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:09.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.491832+0000 mon.b (mon.1) 60 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:09.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.505971+0000 mon.a (mon.0) 841 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:09.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.506318+0000 mon.a (mon.0) 842 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:09.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.506643+0000 mon.a (mon.0) 843 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:09.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.506969+0000 mon.a (mon.0) 844 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:27:09.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.507288+0000 mon.a (mon.0) 845 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:09.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.560880+0000 mon.c (mon.2) 148 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-4"}]: dispatch 2024-02-13T03:27:09.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:09 smithi183 bash[20824]: audit 2024-02-13T03:27:08.993424+0000 mon.a (mon.0) 846 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-4"}]: dispatch 2024-02-13T03:27:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: cluster 2024-02-13T03:27:07.168383+0000 osd.0 (osd.0) 5 : cluster [DBG] 18.4 deep-scrub starts 2024-02-13T03:27:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: cluster 2024-02-13T03:27:07.172895+0000 osd.0 (osd.0) 6 : cluster [DBG] 18.4 deep-scrub ok 2024-02-13T03:27:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: cluster 2024-02-13T03:27:07.362911+0000 osd.1 (osd.1) 5 : cluster [DBG] 18.5 deep-scrub starts 2024-02-13T03:27:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: cluster 2024-02-13T03:27:07.366864+0000 osd.1 (osd.1) 6 : cluster [DBG] 18.5 deep-scrub ok 2024-02-13T03:27:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.021258+0000 mon.a (mon.0) 833 : audit [INF] from='client.25150 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-13T03:27:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: cluster 2024-02-13T03:27:08.023080+0000 client.admin (client.25150) 0 : cluster [INF] fourxx 2024-02-13T03:27:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.023323+0000 mon.a (mon.0) 834 : audit [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-13T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.026159+0000 mon.b (mon.1) 57 : audit [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-13T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: cluster 2024-02-13T03:27:08.258753+0000 mgr.y (mgr.24473) 240 : cluster [DBG] pgmap v184: 652 pgs: 37 creating+peering, 99 unknown, 516 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-13T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.447770+0000 mon.c (mon.2) 144 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: cluster 2024-02-13T03:27:08.469694+0000 mon.a (mon.0) 835 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.477916+0000 mon.a (mon.0) 836 : audit [INF] from='client.25222 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.478012+0000 mon.a (mon.0) 837 : audit [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]': finished 2024-02-13T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.478095+0000 mon.a (mon.0) 838 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-4-cache", "mode": "writeback"}]': finished 2024-02-13T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.478174+0000 mon.a (mon.0) 839 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: cluster 2024-02-13T03:27:08.478934+0000 mon.a (mon.0) 840 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-13T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.482751+0000 mon.c (mon.2) 145 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.483188+0000 mon.c (mon.2) 146 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-13T03:27:09.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: cluster 2024-02-13T03:27:07.168383+0000 osd.0 (osd.0) 5 : cluster [DBG] 18.4 deep-scrub starts 2024-02-13T03:27:09.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: cluster 2024-02-13T03:27:07.172895+0000 osd.0 (osd.0) 6 : cluster [DBG] 18.4 deep-scrub ok 2024-02-13T03:27:09.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: cluster 2024-02-13T03:27:07.362911+0000 osd.1 (osd.1) 5 : cluster [DBG] 18.5 deep-scrub starts 2024-02-13T03:27:09.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: cluster 2024-02-13T03:27:07.366864+0000 osd.1 (osd.1) 6 : cluster [DBG] 18.5 deep-scrub ok 2024-02-13T03:27:09.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.021258+0000 mon.a (mon.0) 833 : audit [INF] from='client.25150 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-13T03:27:09.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: cluster 2024-02-13T03:27:08.023080+0000 client.admin (client.25150) 0 : cluster [INF] fourxx 2024-02-13T03:27:09.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.023323+0000 mon.a (mon.0) 834 : audit [INF] from='client.25150 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-13T03:27:09.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.026159+0000 mon.b (mon.1) 57 : audit [INF] from='client.? v1:172.21.15.161:0/3416141498' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-13T03:27:09.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: cluster 2024-02-13T03:27:08.258753+0000 mgr.y (mgr.24473) 240 : cluster [DBG] pgmap v184: 652 pgs: 37 creating+peering, 99 unknown, 516 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-13T03:27:09.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.447770+0000 mon.c (mon.2) 144 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:09.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: cluster 2024-02-13T03:27:08.469694+0000 mon.a (mon.0) 835 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:09.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.477916+0000 mon.a (mon.0) 836 : audit [INF] from='client.25222 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:09.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.478012+0000 mon.a (mon.0) 837 : audit [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]': finished 2024-02-13T03:27:09.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.478095+0000 mon.a (mon.0) 838 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-4-cache", "mode": "writeback"}]': finished 2024-02-13T03:27:09.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.478174+0000 mon.a (mon.0) 839 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:09.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: cluster 2024-02-13T03:27:08.478934+0000 mon.a (mon.0) 840 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-13T03:27:09.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.482751+0000 mon.c (mon.2) 145 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:09.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.483188+0000 mon.c (mon.2) 146 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-13T03:27:09.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.485974+0000 mon.c (mon.2) 147 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:27:09.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.490062+0000 mon.b (mon.1) 58 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:09.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.491548+0000 mon.b (mon.1) 59 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:09.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.491832+0000 mon.b (mon.1) 60 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:09.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.505971+0000 mon.a (mon.0) 841 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:09.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.506318+0000 mon.a (mon.0) 842 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:09.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.506643+0000 mon.a (mon.0) 843 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:09.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.506969+0000 mon.a (mon.0) 844 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:27:09.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.507288+0000 mon.a (mon.0) 845 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:09.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.560880+0000 mon.c (mon.2) 148 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-4"}]: dispatch 2024-02-13T03:27:09.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[27879]: audit 2024-02-13T03:27:08.993424+0000 mon.a (mon.0) 846 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-4"}]: dispatch 2024-02-13T03:27:09.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.485974+0000 mon.c (mon.2) 147 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:27:09.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.490062+0000 mon.b (mon.1) 58 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:09.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.491548+0000 mon.b (mon.1) 59 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:09.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.491832+0000 mon.b (mon.1) 60 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:09.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.505971+0000 mon.a (mon.0) 841 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:09.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.506318+0000 mon.a (mon.0) 842 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:09.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.506643+0000 mon.a (mon.0) 843 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:09.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.506969+0000 mon.a (mon.0) 844 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:27:09.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.507288+0000 mon.a (mon.0) 845 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:09.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.560880+0000 mon.c (mon.2) 148 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-4"}]: dispatch 2024-02-13T03:27:09.934 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:09 smithi161 bash[19769]: audit 2024-02-13T03:27:08.993424+0000 mon.a (mon.0) 846 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-4"}]: dispatch 2024-02-13T03:27:10.497 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: Running main() from gmock_main.cc 2024-02-13T03:27:10.497 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [==========] Running 23 tests from 2 test suites. 2024-02-13T03:27:10.497 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [----------] Global test environment set-up. 2024-02-13T03:27:10.497 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [----------] 13 tests from LibRadosIo 2024-02-13T03:27:10.497 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-02-13T03:27:10.497 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (234 ms) 2024-02-13T03:27:10.497 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-02-13T03:27:10.497 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-02-13T03:27:10.497 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-02-13T03:27:10.497 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: no timeout :/ 2024-02-13T03:27:10.497 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: no timeout :/ 2024-02-13T03:27:10.497 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: no timeout :/ 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: no timeout :/ 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: no timeout :/ 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (75 ms) 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (6 ms) 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIo.Checksum (4 ms) 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (6 ms) 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (5 ms) 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (7 ms) 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-02-13T03:27:10.498 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIo.TruncTest (6 ms) 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (6 ms) 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (4 ms) 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIo.RmXattr (15 ms) 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIo.XattrIter (7 ms) 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [----------] 13 tests from LibRadosIo (376 ms total) 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (524 ms) 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-02-13T03:27:10.499 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (11 ms) 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (5 ms) 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (7 ms) 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (9 ms) 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (4 ms) 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (4 ms) 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-02-13T03:27:10.500 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (10 ms) 2024-02-13T03:27:10.501 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-02-13T03:27:10.501 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (5 ms) 2024-02-13T03:27:10.501 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [----------] 10 tests from LibRadosIoEC (582 ms total) 2024-02-13T03:27:10.501 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: 2024-02-13T03:27:10.501 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [----------] Global test environment tear-down 2024-02-13T03:27:10.501 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [==========] 23 tests from 2 test suites ran. (24287 ms total) 2024-02-13T03:27:10.501 INFO:tasks.workunit.client.0.smithi161.stdout: LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-02-13T03:27:10.501 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (4 ms) 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (5 ms) 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (4 ms) 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (6 ms) 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (5 ms) 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (4 ms) 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-02-13T03:27:10.502 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (9 ms) 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (8 ms) 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (5 ms) 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (6 ms) 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (629 ms total) 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [----------] Global test environment tear-down 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (24281 ms total) 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_io: [ PASSED ] 23 tests. 2024-02-13T03:27:10.503 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: seed 62041 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (256 ms) 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (22 ms) 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (3 ms) 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (13 ms) 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (16 ms) 2024-02-13T03:27:10.504 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (310 ms total) 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (520 ms) 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (5 ms) 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (12 ms) 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (537 ms total) 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (24158 ms total) 2024-02-13T03:27:10.505 INFO:tasks.workunit.client.0.smithi161.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-02-13T03:27:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.449224+0000 mon.c (mon.2) 149 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.449367+0000 mon.a (mon.0) 847 : audit [INF] from='client.25150 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-13T03:27:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.479077+0000 mon.a (mon.0) 848 : audit [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]': finished 2024-02-13T03:27:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.479182+0000 mon.a (mon.0) 849 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:10.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.479267+0000 mon.a (mon.0) 850 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]': finished 2024-02-13T03:27:10.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.479343+0000 mon.a (mon.0) 851 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-13T03:27:10.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.479427+0000 mon.a (mon.0) 852 : audit [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]': finished 2024-02-13T03:27:10.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.479638+0000 mon.a (mon.0) 853 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-4"}]': finished 2024-02-13T03:27:10.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: cluster 2024-02-13T03:27:09.479747+0000 mon.a (mon.0) 854 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-13T03:27:10.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.481696+0000 mon.c (mon.2) 150 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:10.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.483027+0000 mon.c (mon.2) 151 : audit [INF] from='client.? v1:172.21.15.161:0/1807536862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi161-61719-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:10.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.483074+0000 mon.a (mon.0) 855 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:10.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.483439+0000 mon.a (mon.0) 856 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:10.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.483716+0000 mon.a (mon.0) 857 : audit [INF] from='client.25243 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:10.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.483958+0000 mon.b (mon.1) 61 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:10.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.484024+0000 mon.a (mon.0) 858 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi161-61719-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:10.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.484992+0000 mon.b (mon.1) 62 : audit [INF] from='client.? v1:172.21.15.161:0/696527319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.485256+0000 mon.c (mon.2) 152 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.485837+0000 mon.a (mon.0) 859 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.489525+0000 mon.a (mon.0) 860 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.489855+0000 mon.a (mon.0) 861 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.491856+0000 mon.b (mon.1) 63 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.492185+0000 mon.b (mon.1) 64 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.549456+0000 mon.b (mon.1) 65 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.557057+0000 mon.c (mon.2) 153 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pg_num": 4}]: dispatch 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:09.953916+0000 mon.a (mon.0) 862 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pg_num": 4}]: dispatch 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: cluster 2024-02-13T03:27:10.259663+0000 mgr.y (mgr.24473) 241 : cluster [DBG] pgmap v187: 596 pgs: 112 unknown, 484 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:10.378337+0000 mon.a (mon.0) 863 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:10.380031+0000 mon.a (mon.0) 864 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:10.381683+0000 mon.a (mon.0) 865 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:10.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:10.382594+0000 mon.b (mon.1) 66 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:10.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:10 smithi183 bash[20824]: audit 2024-02-13T03:27:10.384294+0000 mon.b (mon.1) 67 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.449224+0000 mon.c (mon.2) 149 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.449367+0000 mon.a (mon.0) 847 : audit [INF] from='client.25150 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-13T03:27:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.479077+0000 mon.a (mon.0) 848 : audit [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]': finished 2024-02-13T03:27:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.479182+0000 mon.a (mon.0) 849 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.479267+0000 mon.a (mon.0) 850 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]': finished 2024-02-13T03:27:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.479343+0000 mon.a (mon.0) 851 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-13T03:27:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.479427+0000 mon.a (mon.0) 852 : audit [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]': finished 2024-02-13T03:27:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.479638+0000 mon.a (mon.0) 853 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-4"}]': finished 2024-02-13T03:27:10.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: cluster 2024-02-13T03:27:09.479747+0000 mon.a (mon.0) 854 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-13T03:27:10.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.481696+0000 mon.c (mon.2) 150 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:10.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.483027+0000 mon.c (mon.2) 151 : audit [INF] from='client.? v1:172.21.15.161:0/1807536862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi161-61719-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:10.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.483074+0000 mon.a (mon.0) 855 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:10.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.483439+0000 mon.a (mon.0) 856 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:10.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.483716+0000 mon.a (mon.0) 857 : audit [INF] from='client.25243 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:10.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.483958+0000 mon.b (mon.1) 61 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:10.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.449224+0000 mon.c (mon.2) 149 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:10.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.449367+0000 mon.a (mon.0) 847 : audit [INF] from='client.25150 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-13T03:27:10.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.479077+0000 mon.a (mon.0) 848 : audit [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi161-61735-15"}]': finished 2024-02-13T03:27:10.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.479182+0000 mon.a (mon.0) 849 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:10.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.479267+0000 mon.a (mon.0) 850 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi161-62041-7"}]': finished 2024-02-13T03:27:10.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.479343+0000 mon.a (mon.0) 851 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-13T03:27:10.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.479427+0000 mon.a (mon.0) 852 : audit [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi161-61743-22"}]': finished 2024-02-13T03:27:10.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.479638+0000 mon.a (mon.0) 853 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-4"}]': finished 2024-02-13T03:27:10.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: cluster 2024-02-13T03:27:09.479747+0000 mon.a (mon.0) 854 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-13T03:27:10.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.481696+0000 mon.c (mon.2) 150 : audit [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:10.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.483027+0000 mon.c (mon.2) 151 : audit [INF] from='client.? v1:172.21.15.161:0/1807536862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi161-61719-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:10.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.483074+0000 mon.a (mon.0) 855 : audit [INF] from='client.25165 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:10.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.483439+0000 mon.a (mon.0) 856 : audit [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]: dispatch 2024-02-13T03:27:10.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.483716+0000 mon.a (mon.0) 857 : audit [INF] from='client.25243 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:10.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.483958+0000 mon.b (mon.1) 61 : audit [INF] from='client.? v1:172.21.15.161:0/3156599118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]: dispatch 2024-02-13T03:27:10.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.484024+0000 mon.a (mon.0) 858 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi161-61719-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:10.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.484992+0000 mon.b (mon.1) 62 : audit [INF] from='client.? v1:172.21.15.161:0/696527319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:10.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.485256+0000 mon.c (mon.2) 152 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-13T03:27:10.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.485837+0000 mon.a (mon.0) 859 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-13T03:27:10.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.489525+0000 mon.a (mon.0) 860 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:10.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.489855+0000 mon.a (mon.0) 861 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:10.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.491856+0000 mon.b (mon.1) 63 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:10.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.492185+0000 mon.b (mon.1) 64 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:10.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.549456+0000 mon.b (mon.1) 65 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:10.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.557057+0000 mon.c (mon.2) 153 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pg_num": 4}]: dispatch 2024-02-13T03:27:10.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:09.953916+0000 mon.a (mon.0) 862 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pg_num": 4}]: dispatch 2024-02-13T03:27:10.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: cluster 2024-02-13T03:27:10.259663+0000 mgr.y (mgr.24473) 241 : cluster [DBG] pgmap v187: 596 pgs: 112 unknown, 484 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:10.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:10.378337+0000 mon.a (mon.0) 863 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:10.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:10.380031+0000 mon.a (mon.0) 864 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:10.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:10.381683+0000 mon.a (mon.0) 865 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:10.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:10.382594+0000 mon.b (mon.1) 66 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:10.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[27879]: audit 2024-02-13T03:27:10.384294+0000 mon.b (mon.1) 67 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:10.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.484024+0000 mon.a (mon.0) 858 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi161-61719-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:10.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.484992+0000 mon.b (mon.1) 62 : audit [INF] from='client.? v1:172.21.15.161:0/696527319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:10.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.485256+0000 mon.c (mon.2) 152 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-13T03:27:10.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.485837+0000 mon.a (mon.0) 859 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-13T03:27:10.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.489525+0000 mon.a (mon.0) 860 : audit [INF] from='client.25174 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:10.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.489855+0000 mon.a (mon.0) 861 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:10.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.491856+0000 mon.b (mon.1) 63 : audit [INF] from='client.? v1:172.21.15.161:0/3388669218' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]: dispatch 2024-02-13T03:27:10.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.492185+0000 mon.b (mon.1) 64 : audit [INF] from='client.? v1:172.21.15.161:0/4210620294' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]: dispatch 2024-02-13T03:27:10.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.549456+0000 mon.b (mon.1) 65 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:10.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.557057+0000 mon.c (mon.2) 153 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pg_num": 4}]: dispatch 2024-02-13T03:27:10.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:09.953916+0000 mon.a (mon.0) 862 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pg_num": 4}]: dispatch 2024-02-13T03:27:10.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: cluster 2024-02-13T03:27:10.259663+0000 mgr.y (mgr.24473) 241 : cluster [DBG] pgmap v187: 596 pgs: 112 unknown, 484 active+clean; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:10.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:10.378337+0000 mon.a (mon.0) 863 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:10.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:10.380031+0000 mon.a (mon.0) 864 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:10.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:10.381683+0000 mon.a (mon.0) 865 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:10.932 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:10.382594+0000 mon.b (mon.1) 66 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:10.933 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:10 smithi161 bash[19769]: audit 2024-02-13T03:27:10.384294+0000 mon.b (mon.1) 67 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:11.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:27:11] "GET /metrics HTTP/1.1" 200 212629 "" "Prometheus/2.33.4" 2024-02-13T03:27:11.457 INFO:tasks.workunit.client.0.smithi161.stdout:ient.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-4", "overlaypool": "test-rados-api-smithi161-61906-4-cache"}]': finished 2024-02-13T03:27:11.457 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.478729+0000 mon.a [INF] from='client.25208 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi161-61719-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:11.457 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.478822+0000 mon.a [INF] from='client.25127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi161-62012-6"}]': finished 2024-02-13T03:27:11.457 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.478938+0000 mon.a [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi161-62096-1"}]': finished 2024-02-13T03:27:11.457 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.479023+0000 mon.a [INF] from='client.25216 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:11.457 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.479135+0000 mon.a [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.479867+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/195956287' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.480099+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/525846338' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-4-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.480998+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2"}]: dispatch 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.481756+0000 mon.c [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.482577+0000 mon.a [INF] from='client.25222 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.482884+0000 mon.a [INF] from='client.25127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi161-62012-6"}]: dispatch 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.483204+0000 mon.a [INF] from='client.25034 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-4-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.483418+0000 mon.a [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi161-61860-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: got: 2024-02-13T03:27:07.484559+0000 mon.b [INF] from='client.? v1:172.21.15.161:0/2325996547' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (11278 ms) 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (25072 ms total) 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [----------] Global test environment tear-down 2024-02-13T03:27:11.458 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (25072 ms total) 2024-02-13T03:27:11.459 INFO:tasks.workunit.client.0.smithi161.stdout: api_cmd: [ PASSED ] 4 tests. 2024-02-13T03:27:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: cluster 2024-02-13T03:27:10.243216+0000 osd.0 (osd.0) 7 : cluster [DBG] 18.8 deep-scrub starts 2024-02-13T03:27:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: cluster 2024-02-13T03:27:10.248912+0000 osd.0 (osd.0) 8 : cluster [DBG] 18.8 deep-scrub ok 2024-02-13T03:27:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.454554+0000 mon.c (mon.2) 154 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: cluster 2024-02-13T03:27:10.476476+0000 mon.a (mon.0) 866 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:27:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.487827+0000 mon.a (mon.0) 867 : audit [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]': finished 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.487970+0000 mon.a (mon.0) 868 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]': finished 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.488084+0000 mon.a (mon.0) 869 : audit [INF] from='client.25243 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.488202+0000 mon.a (mon.0) 870 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi161-61719-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.488333+0000 mon.a (mon.0) 871 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.488518+0000 mon.a (mon.0) 872 : audit [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]': finished 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.488652+0000 mon.a (mon.0) 873 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]': finished 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.489325+0000 mon.a (mon.0) 874 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pg_num": 4}]': finished 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.489631+0000 mon.a (mon.0) 875 : audit [INF] from='client.25249 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: cluster 2024-02-13T03:27:10.489791+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.491364+0000 mon.c (mon.2) 155 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.491666+0000 mon.c (mon.2) 156 : audit [INF] from='client.? v1:172.21.15.161:0/3890674510' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.491984+0000 mon.c (mon.2) 157 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.493769+0000 mon.c (mon.2) 158 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1"}]: dispatch 2024-02-13T03:27:11.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.498666+0000 mon.b (mon.1) 68 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:11.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.499318+0000 mon.a (mon.0) 877 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:11.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.500426+0000 mon.a (mon.0) 878 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:11.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.501089+0000 mon.a (mon.0) 879 : audit [INF] from='client.25247 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:11.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.501737+0000 mon.a (mon.0) 880 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:11.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.502082+0000 mon.a (mon.0) 881 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1"}]: dispatch 2024-02-13T03:27:11.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.503795+0000 mon.a (mon.0) 882 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi161-62063-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:11.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:11 smithi183 bash[20824]: audit 2024-02-13T03:27:10.505451+0000 mon.b (mon.1) 69 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi161-62063-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:11.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: cluster 2024-02-13T03:27:10.243216+0000 osd.0 (osd.0) 7 : cluster [DBG] 18.8 deep-scrub starts 2024-02-13T03:27:11.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: cluster 2024-02-13T03:27:10.248912+0000 osd.0 (osd.0) 8 : cluster [DBG] 18.8 deep-scrub ok 2024-02-13T03:27:11.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.454554+0000 mon.c (mon.2) 154 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:11.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: cluster 2024-02-13T03:27:10.476476+0000 mon.a (mon.0) 866 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:27:11.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.487827+0000 mon.a (mon.0) 867 : audit [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]': finished 2024-02-13T03:27:11.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.487970+0000 mon.a (mon.0) 868 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]': finished 2024-02-13T03:27:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.488084+0000 mon.a (mon.0) 869 : audit [INF] from='client.25243 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.488202+0000 mon.a (mon.0) 870 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi161-61719-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.488333+0000 mon.a (mon.0) 871 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-13T03:27:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.488518+0000 mon.a (mon.0) 872 : audit [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]': finished 2024-02-13T03:27:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.488652+0000 mon.a (mon.0) 873 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]': finished 2024-02-13T03:27:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.489325+0000 mon.a (mon.0) 874 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pg_num": 4}]': finished 2024-02-13T03:27:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.489631+0000 mon.a (mon.0) 875 : audit [INF] from='client.25249 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: cluster 2024-02-13T03:27:10.489791+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-13T03:27:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.491364+0000 mon.c (mon.2) 155 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.491666+0000 mon.c (mon.2) 156 : audit [INF] from='client.? v1:172.21.15.161:0/3890674510' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.491984+0000 mon.c (mon.2) 157 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.493769+0000 mon.c (mon.2) 158 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1"}]: dispatch 2024-02-13T03:27:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.498666+0000 mon.b (mon.1) 68 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.499318+0000 mon.a (mon.0) 877 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.500426+0000 mon.a (mon.0) 878 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.501089+0000 mon.a (mon.0) 879 : audit [INF] from='client.25247 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.501737+0000 mon.a (mon.0) 880 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.502082+0000 mon.a (mon.0) 881 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1"}]: dispatch 2024-02-13T03:27:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.503795+0000 mon.a (mon.0) 882 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi161-62063-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[19769]: audit 2024-02-13T03:27:10.505451+0000 mon.b (mon.1) 69 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi161-62063-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:11.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: cluster 2024-02-13T03:27:10.243216+0000 osd.0 (osd.0) 7 : cluster [DBG] 18.8 deep-scrub starts 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: cluster 2024-02-13T03:27:10.248912+0000 osd.0 (osd.0) 8 : cluster [DBG] 18.8 deep-scrub ok 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.454554+0000 mon.c (mon.2) 154 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: cluster 2024-02-13T03:27:10.476476+0000 mon.a (mon.0) 866 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.487827+0000 mon.a (mon.0) 867 : audit [INF] from='client.25165 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi161-61735-15"}]': finished 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.487970+0000 mon.a (mon.0) 868 : audit [INF] from='client.25034 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-4", "tierpool": "test-rados-api-smithi161-61906-4-cache"}]': finished 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.488084+0000 mon.a (mon.0) 869 : audit [INF] from='client.25243 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.488202+0000 mon.a (mon.0) 870 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi161-61719-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.488333+0000 mon.a (mon.0) 871 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.488518+0000 mon.a (mon.0) 872 : audit [INF] from='client.25174 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi161-61743-22"}]': finished 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.488652+0000 mon.a (mon.0) 873 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi161-62041-7"}]': finished 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.489325+0000 mon.a (mon.0) 874 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pg_num": 4}]': finished 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.489631+0000 mon.a (mon.0) 875 : audit [INF] from='client.25249 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: cluster 2024-02-13T03:27:10.489791+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-13T03:27:11.931 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.491364+0000 mon.c (mon.2) 155 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:11.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.491666+0000 mon.c (mon.2) 156 : audit [INF] from='client.? v1:172.21.15.161:0/3890674510' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:11.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.491984+0000 mon.c (mon.2) 157 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:11.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.493769+0000 mon.c (mon.2) 158 : audit [INF] from='client.? v1:172.21.15.161:0/829854564' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1"}]: dispatch 2024-02-13T03:27:11.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.498666+0000 mon.b (mon.1) 68 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:11.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.499318+0000 mon.a (mon.0) 877 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:11.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.500426+0000 mon.a (mon.0) 878 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:11.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.501089+0000 mon.a (mon.0) 879 : audit [INF] from='client.25247 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:11.932 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.501737+0000 mon.a (mon.0) 880 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:11.933 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.502082+0000 mon.a (mon.0) 881 : audit [INF] from='client.25202 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1"}]: dispatch 2024-02-13T03:27:11.933 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.503795+0000 mon.a (mon.0) 882 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi161-62063-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:11.933 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:11 smithi161 bash[27879]: audit 2024-02-13T03:27:10.505451+0000 mon.b (mon.1) 69 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi161-62063-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:12.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-02-13T03:27:12.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-02-13T03:27:12.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-02-13T03:27:12.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-02-13T03:27:12.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: seed 61798 2024-02-13T03:27:12.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-02-13T03:27:12.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (265 ms) 2024-02-13T03:27:12.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-02-13T03:27:12.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (12 ms) 2024-02-13T03:27:12.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-02-13T03:27:12.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1006 ms) 2024-02-13T03:27:12.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-02-13T03:27:12.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1007 ms) 2024-02-13T03:27:12.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-02-13T03:27:12.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (11 ms) 2024-02-13T03:27:12.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-02-13T03:27:12.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (12 ms) 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (8 ms) 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (4 ms) 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2325 ms total) 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (531 ms) 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (5 ms) 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1006 ms) 2024-02-13T03:27:12.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1005 ms) 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (5 ms) 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (4 ms) 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (2564 ms total) 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (26269 ms total) 2024-02-13T03:27:12.522 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-02-13T03:27:12.536 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: Running main() from gmock_main.cc 2024-02-13T03:27:12.537 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-02-13T03:27:12.537 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [----------] Global test environment set-up. 2024-02-13T03:27:12.537 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-02-13T03:27:12.537 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-02-13T03:27:12.537 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (271 ms) 2024-02-13T03:27:12.537 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-02-13T03:27:12.537 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLock.LockShared (6 ms) 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1006 ms) 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1005 ms) 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (10 ms) 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLock.Unlock (5 ms) 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (13 ms) 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (6 ms) 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [----------] 8 tests from LibRadosLock (2322 ms total) 2024-02-13T03:27:12.538 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (518 ms) 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (7 ms) 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1011 ms) 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1005 ms) 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (5 ms) 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (4 ms) 2024-02-13T03:27:12.539 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-02-13T03:27:12.540 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (6 ms) 2024-02-13T03:27:12.540 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-02-13T03:27:12.540 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (4 ms) 2024-02-13T03:27:12.540 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (2560 ms total) 2024-02-13T03:27:12.540 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: 2024-02-13T03:27:12.540 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [----------] Global test environment tear-down 2024-02-13T03:27:12.540 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (26307 ms total) 2024-02-13T03:27:12.540 INFO:tasks.workunit.client.0.smithi161.stdout: api_lock: [ PASSED ] 16 tests. 2024-02-13T03:27:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:12 smithi183 bash[20824]: audit 2024-02-13T03:27:11.458745+0000 mon.c (mon.2) 159 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:12 smithi183 bash[20824]: audit 2024-02-13T03:27:11.495306+0000 mon.a (mon.0) 883 : audit [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]': finished 2024-02-13T03:27:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:12 smithi183 bash[20824]: audit 2024-02-13T03:27:11.495893+0000 mon.a (mon.0) 884 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]': finished 2024-02-13T03:27:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:12 smithi183 bash[20824]: audit 2024-02-13T03:27:11.496085+0000 mon.a (mon.0) 885 : audit [INF] from='client.25247 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:12 smithi183 bash[20824]: audit 2024-02-13T03:27:11.496207+0000 mon.a (mon.0) 886 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:12 smithi183 bash[20824]: audit 2024-02-13T03:27:11.496323+0000 mon.a (mon.0) 887 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1"}]': finished 2024-02-13T03:27:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:12 smithi183 bash[20824]: audit 2024-02-13T03:27:11.498080+0000 mon.c (mon.2) 160 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:12 smithi183 bash[20824]: audit 2024-02-13T03:27:11.498383+0000 mon.c (mon.2) 161 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]: dispatch 2024-02-13T03:27:12.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:12 smithi183 bash[20824]: audit 2024-02-13T03:27:11.500396+0000 mon.b (mon.1) 70 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:12.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:12 smithi183 bash[20824]: cluster 2024-02-13T03:27:11.992584+0000 mon.a (mon.0) 888 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-13T03:27:12.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:12 smithi183 bash[20824]: audit 2024-02-13T03:27:11.994537+0000 mon.a (mon.0) 889 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:12.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:12 smithi183 bash[20824]: audit 2024-02-13T03:27:11.994785+0000 mon.a (mon.0) 890 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:12.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:12 smithi183 bash[20824]: audit 2024-02-13T03:27:11.995370+0000 mon.a (mon.0) 891 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]: dispatch 2024-02-13T03:27:12.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[27879]: audit 2024-02-13T03:27:11.458745+0000 mon.c (mon.2) 159 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:12.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[27879]: audit 2024-02-13T03:27:11.495306+0000 mon.a (mon.0) 883 : audit [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]': finished 2024-02-13T03:27:12.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[27879]: audit 2024-02-13T03:27:11.495893+0000 mon.a (mon.0) 884 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]': finished 2024-02-13T03:27:12.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[27879]: audit 2024-02-13T03:27:11.496085+0000 mon.a (mon.0) 885 : audit [INF] from='client.25247 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:12.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[27879]: audit 2024-02-13T03:27:11.496207+0000 mon.a (mon.0) 886 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:12.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[27879]: audit 2024-02-13T03:27:11.496323+0000 mon.a (mon.0) 887 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1"}]': finished 2024-02-13T03:27:12.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[27879]: audit 2024-02-13T03:27:11.498080+0000 mon.c (mon.2) 160 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:12.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[27879]: audit 2024-02-13T03:27:11.498383+0000 mon.c (mon.2) 161 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]: dispatch 2024-02-13T03:27:12.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[27879]: audit 2024-02-13T03:27:11.500396+0000 mon.b (mon.1) 70 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:12.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[27879]: cluster 2024-02-13T03:27:11.992584+0000 mon.a (mon.0) 888 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-13T03:27:12.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[27879]: audit 2024-02-13T03:27:11.994537+0000 mon.a (mon.0) 889 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:12.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[27879]: audit 2024-02-13T03:27:11.994785+0000 mon.a (mon.0) 890 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:12.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[27879]: audit 2024-02-13T03:27:11.995370+0000 mon.a (mon.0) 891 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]: dispatch 2024-02-13T03:27:12.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[19769]: audit 2024-02-13T03:27:11.458745+0000 mon.c (mon.2) 159 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:12.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[19769]: audit 2024-02-13T03:27:11.495306+0000 mon.a (mon.0) 883 : audit [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi161-61765-10"}]': finished 2024-02-13T03:27:12.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[19769]: audit 2024-02-13T03:27:11.495893+0000 mon.a (mon.0) 884 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi161-61798-10"}]': finished 2024-02-13T03:27:12.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[19769]: audit 2024-02-13T03:27:11.496085+0000 mon.a (mon.0) 885 : audit [INF] from='client.25247 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:12.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[19769]: audit 2024-02-13T03:27:11.496207+0000 mon.a (mon.0) 886 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:12.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[19769]: audit 2024-02-13T03:27:11.496323+0000 mon.a (mon.0) 887 : audit [INF] from='client.25202 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi161-61860-1","app":"app1","key":"key1"}]': finished 2024-02-13T03:27:12.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[19769]: audit 2024-02-13T03:27:11.498080+0000 mon.c (mon.2) 160 : audit [INF] from='client.? v1:172.21.15.161:0/1259280962' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:12.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[19769]: audit 2024-02-13T03:27:11.498383+0000 mon.c (mon.2) 161 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]: dispatch 2024-02-13T03:27:12.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[19769]: audit 2024-02-13T03:27:11.500396+0000 mon.b (mon.1) 70 : audit [INF] from='client.? v1:172.21.15.161:0/2452856677' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:12.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[19769]: cluster 2024-02-13T03:27:11.992584+0000 mon.a (mon.0) 888 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-13T03:27:12.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[19769]: audit 2024-02-13T03:27:11.994537+0000 mon.a (mon.0) 889 : audit [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]: dispatch 2024-02-13T03:27:12.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[19769]: audit 2024-02-13T03:27:11.994785+0000 mon.a (mon.0) 890 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]: dispatch 2024-02-13T03:27:12.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:12 smithi161 bash[19769]: audit 2024-02-13T03:27:11.995370+0000 mon.a (mon.0) 891 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]: dispatch 2024-02-13T03:27:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: audit 2024-02-13T03:27:11.995245+0000 mon.c (mon.2) 162 : audit [INF] from='client.? v1:172.21.15.161:0/3774265899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi161-61936-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: cluster 2024-02-13T03:27:12.260378+0000 mgr.y (mgr.24473) 242 : cluster [DBG] pgmap v190: 520 pgs: 36 creating+peering, 32 unknown, 452 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-02-13T03:27:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: audit 2024-02-13T03:27:12.459959+0000 mon.c (mon.2) 163 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: cluster 2024-02-13T03:27:12.477368+0000 mon.a (mon.0) 892 : cluster [WRN] pool 'PoolQuotaPP_smithi161-61726-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-13T03:27:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: cluster 2024-02-13T03:27:12.479538+0000 mon.a (mon.0) 893 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-13T03:27:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: audit 2024-02-13T03:27:12.481175+0000 mon.a (mon.0) 894 : audit [INF] from='client.25256 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi161-61936-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: audit 2024-02-13T03:27:12.496758+0000 mon.a (mon.0) 895 : audit [INF] from='client.25249 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi161-62063-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]': finished 2024-02-13T03:27:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: audit 2024-02-13T03:27:12.496932+0000 mon.a (mon.0) 896 : audit [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]': finished 2024-02-13T03:27:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: audit 2024-02-13T03:27:12.497003+0000 mon.a (mon.0) 897 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]': finished 2024-02-13T03:27:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: audit 2024-02-13T03:27:12.497099+0000 mon.a (mon.0) 898 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]': finished 2024-02-13T03:27:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: audit 2024-02-13T03:27:12.501404+0000 mon.c (mon.2) 164 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "mode": "writeback"}]: dispatch 2024-02-13T03:27:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: audit 2024-02-13T03:27:12.513979+0000 mon.c (mon.2) 165 : audit [INF] from='client.? v1:172.21.15.161:0/4219162521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: cluster 2024-02-13T03:27:12.979168+0000 mon.a (mon.0) 899 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-13T03:27:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: audit 2024-02-13T03:27:12.981274+0000 mon.a (mon.0) 900 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "mode": "writeback"}]: dispatch 2024-02-13T03:27:13.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[19769]: audit 2024-02-13T03:27:12.981988+0000 mon.a (mon.0) 901 : audit [INF] from='client.25271 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:13.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: audit 2024-02-13T03:27:11.995245+0000 mon.c (mon.2) 162 : audit [INF] from='client.? v1:172.21.15.161:0/3774265899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi161-61936-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:13.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: cluster 2024-02-13T03:27:12.260378+0000 mgr.y (mgr.24473) 242 : cluster [DBG] pgmap v190: 520 pgs: 36 creating+peering, 32 unknown, 452 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-02-13T03:27:13.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: audit 2024-02-13T03:27:12.459959+0000 mon.c (mon.2) 163 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:13.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: cluster 2024-02-13T03:27:12.477368+0000 mon.a (mon.0) 892 : cluster [WRN] pool 'PoolQuotaPP_smithi161-61726-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-13T03:27:13.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: cluster 2024-02-13T03:27:12.479538+0000 mon.a (mon.0) 893 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-13T03:27:13.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: audit 2024-02-13T03:27:12.481175+0000 mon.a (mon.0) 894 : audit [INF] from='client.25256 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi161-61936-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:13.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: audit 2024-02-13T03:27:12.496758+0000 mon.a (mon.0) 895 : audit [INF] from='client.25249 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi161-62063-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]': finished 2024-02-13T03:27:13.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: audit 2024-02-13T03:27:12.496932+0000 mon.a (mon.0) 896 : audit [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]': finished 2024-02-13T03:27:13.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: audit 2024-02-13T03:27:12.497003+0000 mon.a (mon.0) 897 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]': finished 2024-02-13T03:27:13.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: audit 2024-02-13T03:27:12.497099+0000 mon.a (mon.0) 898 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]': finished 2024-02-13T03:27:13.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: audit 2024-02-13T03:27:12.501404+0000 mon.c (mon.2) 164 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "mode": "writeback"}]: dispatch 2024-02-13T03:27:13.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: audit 2024-02-13T03:27:12.513979+0000 mon.c (mon.2) 165 : audit [INF] from='client.? v1:172.21.15.161:0/4219162521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:13.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: cluster 2024-02-13T03:27:12.979168+0000 mon.a (mon.0) 899 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-13T03:27:13.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: audit 2024-02-13T03:27:12.981274+0000 mon.a (mon.0) 900 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "mode": "writeback"}]: dispatch 2024-02-13T03:27:13.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:13 smithi161 bash[27879]: audit 2024-02-13T03:27:12.981988+0000 mon.a (mon.0) 901 : audit [INF] from='client.25271 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: audit 2024-02-13T03:27:11.995245+0000 mon.c (mon.2) 162 : audit [INF] from='client.? v1:172.21.15.161:0/3774265899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi161-61936-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: cluster 2024-02-13T03:27:12.260378+0000 mgr.y (mgr.24473) 242 : cluster [DBG] pgmap v190: 520 pgs: 36 creating+peering, 32 unknown, 452 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-02-13T03:27:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: audit 2024-02-13T03:27:12.459959+0000 mon.c (mon.2) 163 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: cluster 2024-02-13T03:27:12.477368+0000 mon.a (mon.0) 892 : cluster [WRN] pool 'PoolQuotaPP_smithi161-61726-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-13T03:27:13.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: cluster 2024-02-13T03:27:12.479538+0000 mon.a (mon.0) 893 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-13T03:27:13.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: audit 2024-02-13T03:27:12.481175+0000 mon.a (mon.0) 894 : audit [INF] from='client.25256 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi161-61936-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:13.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: audit 2024-02-13T03:27:12.496758+0000 mon.a (mon.0) 895 : audit [INF] from='client.25249 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi161-62063-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]': finished 2024-02-13T03:27:13.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: audit 2024-02-13T03:27:12.496932+0000 mon.a (mon.0) 896 : audit [INF] from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi161-61765-10"}]': finished 2024-02-13T03:27:13.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: audit 2024-02-13T03:27:12.497003+0000 mon.a (mon.0) 897 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi161-61798-10"}]': finished 2024-02-13T03:27:13.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: audit 2024-02-13T03:27:12.497099+0000 mon.a (mon.0) 898 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]': finished 2024-02-13T03:27:13.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: audit 2024-02-13T03:27:12.501404+0000 mon.c (mon.2) 164 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "mode": "writeback"}]: dispatch 2024-02-13T03:27:13.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: audit 2024-02-13T03:27:12.513979+0000 mon.c (mon.2) 165 : audit [INF] from='client.? v1:172.21.15.161:0/4219162521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:13.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: cluster 2024-02-13T03:27:12.979168+0000 mon.a (mon.0) 899 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-13T03:27:13.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: audit 2024-02-13T03:27:12.981274+0000 mon.a (mon.0) 900 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "mode": "writeback"}]: dispatch 2024-02-13T03:27:13.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:13 smithi183 bash[20824]: audit 2024-02-13T03:27:12.981988+0000 mon.a (mon.0) 901 : audit [INF] from='client.25271 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:14.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[27879]: audit 2024-02-13T03:27:13.461062+0000 mon.c (mon.2) 166 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:14.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[27879]: cluster 2024-02-13T03:27:13.491223+0000 mon.a (mon.0) 902 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:14.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[27879]: audit 2024-02-13T03:27:13.504514+0000 mon.a (mon.0) 903 : audit [INF] from='client.25256 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi161-61936-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:14.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[27879]: audit 2024-02-13T03:27:13.504691+0000 mon.a (mon.0) 904 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "mode": "writeback"}]': finished 2024-02-13T03:27:14.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[27879]: audit 2024-02-13T03:27:13.504761+0000 mon.a (mon.0) 905 : audit [INF] from='client.25271 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:14.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[27879]: cluster 2024-02-13T03:27:13.505230+0000 mon.a (mon.0) 906 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-13T03:27:14.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[27879]: audit 2024-02-13T03:27:13.512689+0000 mon.c (mon.2) 167 : audit [INF] from='client.? v1:172.21.15.161:0/487517093' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:14.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[27879]: audit 2024-02-13T03:27:13.513174+0000 mon.c (mon.2) 168 : audit [INF] from='client.? v1:172.21.15.161:0/3212895654' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi161-61719-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:14.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[27879]: audit 2024-02-13T03:27:13.513252+0000 mon.a (mon.0) 907 : audit [INF] from='client.25484 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:14.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[27879]: audit 2024-02-13T03:27:13.515983+0000 mon.a (mon.0) 908 : audit [INF] from='client.25289 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi161-61719-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:14.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[27879]: cluster 2024-02-13T03:27:14.261129+0000 mgr.y (mgr.24473) 243 : cluster [DBG] pgmap v193: 528 pgs: 4 creating+peering, 136 unknown, 388 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 6.5 KiB/s wr, 3 op/s 2024-02-13T03:27:14.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[27879]: audit 2024-02-13T03:27:14.462210+0000 mon.c (mon.2) 169 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:14.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[19769]: audit 2024-02-13T03:27:13.461062+0000 mon.c (mon.2) 166 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:14.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[19769]: cluster 2024-02-13T03:27:13.491223+0000 mon.a (mon.0) 902 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:14.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[19769]: audit 2024-02-13T03:27:13.504514+0000 mon.a (mon.0) 903 : audit [INF] from='client.25256 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi161-61936-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:14.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[19769]: audit 2024-02-13T03:27:13.504691+0000 mon.a (mon.0) 904 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "mode": "writeback"}]': finished 2024-02-13T03:27:14.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[19769]: audit 2024-02-13T03:27:13.504761+0000 mon.a (mon.0) 905 : audit [INF] from='client.25271 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:14.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[19769]: cluster 2024-02-13T03:27:13.505230+0000 mon.a (mon.0) 906 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-13T03:27:14.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[19769]: audit 2024-02-13T03:27:13.512689+0000 mon.c (mon.2) 167 : audit [INF] from='client.? v1:172.21.15.161:0/487517093' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:14.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[19769]: audit 2024-02-13T03:27:13.513174+0000 mon.c (mon.2) 168 : audit [INF] from='client.? v1:172.21.15.161:0/3212895654' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi161-61719-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:14.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[19769]: audit 2024-02-13T03:27:13.513252+0000 mon.a (mon.0) 907 : audit [INF] from='client.25484 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:14.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[19769]: audit 2024-02-13T03:27:13.515983+0000 mon.a (mon.0) 908 : audit [INF] from='client.25289 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi161-61719-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:14.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[19769]: cluster 2024-02-13T03:27:14.261129+0000 mgr.y (mgr.24473) 243 : cluster [DBG] pgmap v193: 528 pgs: 4 creating+peering, 136 unknown, 388 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 6.5 KiB/s wr, 3 op/s 2024-02-13T03:27:14.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:14 smithi161 bash[19769]: audit 2024-02-13T03:27:14.462210+0000 mon.c (mon.2) 169 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:14 smithi183 bash[20824]: audit 2024-02-13T03:27:13.461062+0000 mon.c (mon.2) 166 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:14 smithi183 bash[20824]: cluster 2024-02-13T03:27:13.491223+0000 mon.a (mon.0) 902 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:14 smithi183 bash[20824]: audit 2024-02-13T03:27:13.504514+0000 mon.a (mon.0) 903 : audit [INF] from='client.25256 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi161-61936-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:14 smithi183 bash[20824]: audit 2024-02-13T03:27:13.504691+0000 mon.a (mon.0) 904 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "mode": "writeback"}]': finished 2024-02-13T03:27:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:14 smithi183 bash[20824]: audit 2024-02-13T03:27:13.504761+0000 mon.a (mon.0) 905 : audit [INF] from='client.25271 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:14 smithi183 bash[20824]: cluster 2024-02-13T03:27:13.505230+0000 mon.a (mon.0) 906 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-13T03:27:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:14 smithi183 bash[20824]: audit 2024-02-13T03:27:13.512689+0000 mon.c (mon.2) 167 : audit [INF] from='client.? v1:172.21.15.161:0/487517093' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:14 smithi183 bash[20824]: audit 2024-02-13T03:27:13.513174+0000 mon.c (mon.2) 168 : audit [INF] from='client.? v1:172.21.15.161:0/3212895654' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi161-61719-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:14.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:14 smithi183 bash[20824]: audit 2024-02-13T03:27:13.513252+0000 mon.a (mon.0) 907 : audit [INF] from='client.25484 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:14.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:14 smithi183 bash[20824]: audit 2024-02-13T03:27:13.515983+0000 mon.a (mon.0) 908 : audit [INF] from='client.25289 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi161-61719-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:14.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:14 smithi183 bash[20824]: cluster 2024-02-13T03:27:14.261129+0000 mgr.y (mgr.24473) 243 : cluster [DBG] pgmap v193: 528 pgs: 4 creating+peering, 136 unknown, 388 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 6.5 KiB/s wr, 3 op/s 2024-02-13T03:27:14.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:14 smithi183 bash[20824]: audit 2024-02-13T03:27:14.462210+0000 mon.c (mon.2) 169 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:15.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:15 smithi183 bash[20824]: audit 2024-02-13T03:27:14.515576+0000 mon.a (mon.0) 909 : audit [INF] from='client.25484 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:15.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:15 smithi183 bash[20824]: audit 2024-02-13T03:27:14.515684+0000 mon.a (mon.0) 910 : audit [INF] from='client.25289 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi161-61719-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:15.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:15 smithi183 bash[20824]: cluster 2024-02-13T03:27:14.515924+0000 mon.a (mon.0) 911 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-13T03:27:15.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:15 smithi183 bash[20824]: audit 2024-02-13T03:27:14.533800+0000 mon.a (mon.0) 912 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:15.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:15 smithi183 bash[20824]: audit 2024-02-13T03:27:14.536292+0000 mon.b (mon.1) 71 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:15.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:15 smithi183 bash[20824]: audit 2024-02-13T03:27:14.538002+0000 mon.a (mon.0) 913 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:15.818 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:15 smithi183 bash[20824]: audit 2024-02-13T03:27:14.538175+0000 mon.b (mon.1) 72 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:15.819 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:15 smithi183 bash[20824]: audit 2024-02-13T03:27:15.463954+0000 mon.c (mon.2) 170 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:15.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[27879]: audit 2024-02-13T03:27:14.515576+0000 mon.a (mon.0) 909 : audit [INF] from='client.25484 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:15.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[27879]: audit 2024-02-13T03:27:14.515684+0000 mon.a (mon.0) 910 : audit [INF] from='client.25289 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi161-61719-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:15.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[27879]: cluster 2024-02-13T03:27:14.515924+0000 mon.a (mon.0) 911 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-13T03:27:15.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[27879]: audit 2024-02-13T03:27:14.533800+0000 mon.a (mon.0) 912 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:15.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[27879]: audit 2024-02-13T03:27:14.536292+0000 mon.b (mon.1) 71 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:15.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[27879]: audit 2024-02-13T03:27:14.538002+0000 mon.a (mon.0) 913 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:15.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[27879]: audit 2024-02-13T03:27:14.538175+0000 mon.b (mon.1) 72 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:15.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[27879]: audit 2024-02-13T03:27:15.463954+0000 mon.c (mon.2) 170 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:15.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[19769]: audit 2024-02-13T03:27:14.515576+0000 mon.a (mon.0) 909 : audit [INF] from='client.25484 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:15.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[19769]: audit 2024-02-13T03:27:14.515684+0000 mon.a (mon.0) 910 : audit [INF] from='client.25289 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi161-61719-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:15.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[19769]: cluster 2024-02-13T03:27:14.515924+0000 mon.a (mon.0) 911 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-13T03:27:15.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[19769]: audit 2024-02-13T03:27:14.533800+0000 mon.a (mon.0) 912 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:15.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[19769]: audit 2024-02-13T03:27:14.536292+0000 mon.b (mon.1) 71 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:15.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[19769]: audit 2024-02-13T03:27:14.538002+0000 mon.a (mon.0) 913 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:15.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[19769]: audit 2024-02-13T03:27:14.538175+0000 mon.b (mon.1) 72 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:15.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:15 smithi161 bash[19769]: audit 2024-02-13T03:27:15.463954+0000 mon.c (mon.2) 170 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:16.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:27:15 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:27:16.663 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-02-13T03:27:16.663 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-02-13T03:27:16.663 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-02-13T03:27:16.663 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-02-13T03:27:16.664 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-02-13T03:27:16.664 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify_test_cb 2024-02-13T03:27:16.664 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (278 ms) 2024-02-13T03:27:16.664 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-02-13T03:27:16.664 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-13T03:27:16.664 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94846614489232 err -107 2024-02-13T03:27:16.664 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1005 ms) 2024-02-13T03:27:16.664 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-02-13T03:27:16.664 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-13T03:27:16.664 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94846614505184 err -107 2024-02-13T03:27:16.664 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1007 ms) 2024-02-13T03:27:16.664 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-02-13T03:27:16.664 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify2_test_cb from 24788 notify_id 292057776128 cookie 94846614511184 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (11 ms) 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify2_test_cb from 24788 notify_id 292057776128 cookie 94846614534128 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (27 ms) 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify2_test_cb from 24788 notify_id 292057776129 cookie 94846614534128 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (17 ms) 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify2_test_cb from 24788 notify_id 292057776128 cookie 94846614534128 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify2_test_cb from 24788 notify_id 292057776128 cookie 94846614557488 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (7 ms) 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify2_test_cb from 24788 notify_id 292057776128 cookie 94846614534128 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify2_test_cb from 24788 notify_id 296352743425 cookie 94846614534128 2024-02-13T03:27:16.665 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3008 ms) 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94846614557488 err -107 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify2_test_cb from 24788 notify_id 322122547202 cookie 94846614557408 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5010 ms) 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94846614575952 err -107 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1006 ms) 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (11376 ms total) 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-02-13T03:27:16.666 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: watch_notify_test_cb 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1068 ms) 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1068 ms total) 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (30302 ms total) 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notif api_watch_notify_pp: Running main() from gmock_main.cc 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: notify 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1466 ms) 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-02-13T03:27:16.667 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (4 ms) 2024-02-13T03:27:16.668 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1470 ms total) 2024-02-13T03:27:16.668 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: 2024-02-13T03:27:16.668 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-02-13T03:27:16.668 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-02-13T03:27:16.668 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: notify 2024-02-13T03:27:16.668 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (72 ms) 2024-02-13T03:27:16.668 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-02-13T03:27:16.668 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: notify 2024-02-13T03:27:16.668 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (4006 ms) 2024-02-13T03:27:16.668 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-02-13T03:27:16.668 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (14 ms) 2024-02-13T03:27:16.668 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-02-13T03:27:16.668 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (33 ms) 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: handle_notify cookie 94224382735760 notify_id 347892350978 notifier_gid 25223 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (7 ms) 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: handle_notify cookie 94224382735760 notify_id 347892350979 notifier_gid 25223 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (10 ms) 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: handle_notify cookie 94224382735760 notify_id 347892350980 notifier_gid 25223 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (6 ms) 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: handle_notify cookie 94224382735760 notify_id 347892350981 notifier_gid 25223 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (5 ms) 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: handle_notify cookie 94224382735760 notify_id 347892350982 notifier_gid 25223 2024-02-13T03:27:16.669 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (5 ms) 2024-02-13T03:27:16.670 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-02-13T03:27:16.670 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: handle_notify cookie 94224382735760 notify_id 347892350983 notifier_gid 25223 2024-02-13T03:27:16.670 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (9 ms) 2024-02-13T03:27:16.670 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-02-13T03:27:16.670 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: trying... 2024-02-13T03:27:16.670 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: handle_notify cookie 94224382735760 notify_id 347892350980 notifier_gid 25223 2024-02-13T03:27:16.670 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: timed out 2024-02-13T03:27:16.670 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: flushing 2024-02-13T03:27:16.670 INFO:tasks.workunit.client.0.smithi161.stdout: api_way: [ PASSED ] 11 tests. 2024-02-13T03:27:16.690 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-02-13T03:27:16.690 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [----------] Global test environment set-up. 2024-02-13T03:27:16.690 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-02-13T03:27:16.690 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-02-13T03:27:16.690 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5028 ms) 2024-02-13T03:27:16.690 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-02-13T03:27:16.690 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (34 ms) 2024-02-13T03:27:16.690 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-02-13T03:27:16.690 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20024 ms) 2024-02-13T03:27:16.690 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-02-13T03:27:16.690 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: attempt 0 of 20 2024-02-13T03:27:16.690 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: failed to deregister: 2024-02-13T03:27:16.690 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: { 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "epoch": 18, 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "modified": "2024-02-13T03:27:12.259921+0000", 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "services": { 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "laundry": { 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "daemons": { 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "summary": "", 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "close-test-pid62163": { 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "start_epoch": 18, 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "start_stamp": "2024-02-13T03:27:11.576200+0000", 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "gid": 25262, 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "addr": "172.21.15.161:0/42959384", 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "metadata": { 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "arch": "x86_64", 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-13T03:27:16.691 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)", 2024-02-13T03:27:16.692 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "ceph_version_short": "16.2.14-521-g0e714d9a", 2024-02-13T03:27:16.692 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-13T03:27:16.692 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "distro": "ubuntu", 2024-02-13T03:27:16.692 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "distro_description": "Ubuntu 18.04.6 LTS", 2024-02-13T03:27:16.692 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "distro_version": "18.04", 2024-02-13T03:27:16.692 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "foo": "bar", 2024-02-13T03:27:16.692 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "hostname": "smithi161", 2024-02-13T03:27:16.692 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "kernel_description": "#224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023", 2024-02-13T03:27:16.692 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "kernel_version": "4.15.0-213-generic", 2024-02-13T03:27:16.692 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "mem_swap_kb": "2097148", 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "mem_total_kb": "32823864", 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "os": "Linux", 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "this": "that" 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: }, 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "task_status": {} 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: }, 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "pid62150": { 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "start_epoch": 17, 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "start_stamp": "2024-02-13T03:26:57.941942+0000", 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "gid": 25030, 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "addr": "172.21.15.161:0/2344103445", 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "metadata": { 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "arch": "x86_64", 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-13T03:27:16.693 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)", 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "ceph_version_short": "16.2.14-521-g0e714d9a", 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "distro": "ubuntu", 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "distro_description": "Ubuntu 18.04.6 LTS", 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "distro_version": "18.04", 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "foo": "bar", 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "hostname": "smithi161", 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "kernel_description": "#224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023", 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "kernel_version": "4.15.0-213-generic", 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "mem_swap_kb": "2097148", 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "mem_total_kb": "32823864", 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "os": "Linux", 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "this": "that" 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: }, 2024-02-13T03:27:16.694 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "task_status": {} 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: } 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: } 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: }, 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "rgw": { 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "daemons": { 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "summary": "", 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "24398": { 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "start_epoch": 13, 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "start_stamp": "2024-02-13T03:21:16.823468+0000", 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "gid": 24398, 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "addr": "172.21.15.161:0/629238266", 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "metadata": { 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "arch": "x86_64", 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-13T03:27:16.695 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "ceph_version_short": "16.2.14-521-g0e714d9a", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "container_hostname": "smithi161", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "distro": "centos", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "distro_version": "8", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "frontend_type#0": "beast", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "hostname": "smithi161", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "id": "foo.a", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "kernel_description": "#224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "kernel_version": "4.15.0-213-generic", 2024-02-13T03:27:16.696 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "mem_swap_kb": "2097148", 2024-02-13T03:27:16.697 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "mem_total_kb": "32823864", 2024-02-13T03:27:16.697 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "num_handles": "1", 2024-02-13T03:27:16.697 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "os": "Linux", 2024-02-13T03:27:16.697 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "pid": "7", 2024-02-13T03:27:16.697 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "realm_id": "", 2024-02-13T03:27:16.697 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "realm_name": "", 2024-02-13T03:27:16.697 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "zone_id": "e6b5d406-dc41-4020-8ba8-6146122dc832", 2024-02-13T03:27:16.697 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "zone_name": "default", 2024-02-13T03:27:16.697 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "zonegroup_id": "974130b2-f99d-4f08-82e8-d9564fbeb842", 2024-02-13T03:27:16.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:16 smithi161 bash[27879]: audit 2024-02-13T03:27:15.525692+0000 mon.a (mon.0) 914 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:16.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:16 smithi161 bash[27879]: audit 2024-02-13T03:27:15.525798+0000 mon.a (mon.0) 915 : audit [INF] from='client.25249 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]': finished 2024-02-13T03:27:16.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:16 smithi161 bash[27879]: cluster 2024-02-13T03:27:15.526026+0000 mon.a (mon.0) 916 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-13T03:27:16.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:16 smithi161 bash[27879]: audit 2024-02-13T03:27:15.548702+0000 mon.b (mon.1) 73 : audit [INF] from='client.? v1:172.21.15.161:0/4201818420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:16.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:16 smithi161 bash[27879]: audit 2024-02-13T03:27:15.550525+0000 mon.b (mon.1) 74 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:16.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:16 smithi161 bash[27879]: audit 2024-02-13T03:27:16.080298+0000 mon.a (mon.0) 917 : audit [INF] from='client.26911 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:16 smithi161 bash[19769]: audit 2024-02-13T03:27:15.525692+0000 mon.a (mon.0) 914 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:16 smithi161 bash[19769]: audit 2024-02-13T03:27:15.525798+0000 mon.a (mon.0) 915 : audit [INF] from='client.25249 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]': finished 2024-02-13T03:27:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:16 smithi161 bash[19769]: cluster 2024-02-13T03:27:15.526026+0000 mon.a (mon.0) 916 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-13T03:27:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:16 smithi161 bash[19769]: audit 2024-02-13T03:27:15.548702+0000 mon.b (mon.1) 73 : audit [INF] from='client.? v1:172.21.15.161:0/4201818420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:16.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:16 smithi161 bash[19769]: audit 2024-02-13T03:27:15.550525+0000 mon.b (mon.1) 74 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:16.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:16 smithi161 bash[19769]: audit 2024-02-13T03:27:16.080298+0000 mon.a (mon.0) 917 : audit [INF] from='client.26911 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:16 smithi183 bash[20824]: audit 2024-02-13T03:27:15.525692+0000 mon.a (mon.0) 914 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:16 smithi183 bash[20824]: audit 2024-02-13T03:27:15.525798+0000 mon.a (mon.0) 915 : audit [INF] from='client.25249 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi161-62063-12"}]': finished 2024-02-13T03:27:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:16 smithi183 bash[20824]: cluster 2024-02-13T03:27:15.526026+0000 mon.a (mon.0) 916 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-13T03:27:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:16 smithi183 bash[20824]: audit 2024-02-13T03:27:15.548702+0000 mon.b (mon.1) 73 : audit [INF] from='client.? v1:172.21.15.161:0/4201818420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:16 smithi183 bash[20824]: audit 2024-02-13T03:27:15.550525+0000 mon.b (mon.1) 74 : audit [INF] from='client.? v1:172.21.15.161:0/1786769180' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:16.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:16 smithi183 bash[20824]: audit 2024-02-13T03:27:16.080298+0000 mon.a (mon.0) 917 : audit [INF] from='client.26911 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:17.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: " mon: 3 daemons, quorum a,b,c (age 11m) 2024-02-13T03:27:17.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: mgr: y(active, since 5m), standbys: x 2024-02-13T03:27:17.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: osd: 8 osds: 8 up (since 6m), 8 in (since 6m) 2024-02-13T03:27:17.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-02-13T03:27:17.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-13T03:27:17.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-13T03:27:17.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:27:17.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: data: 2024-02-13T03:27:17.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: pools: 30 pools, 900 pgs 2024-02-13T03:27:17.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: objects: 224 objects, 5.7 KiB 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: pgs: 50.778% pgs unknown 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 34.556% pgs not active 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 457 unknown 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 311 creating+peering 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 132 active+clean 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: io: 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [ OK ] LibRadosService.StatusFormat (6391 ms) 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [ RUN ] LibRadosService.Status 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [ OK ] LibRadosService.Status (20070 ms) 2024-02-13T03:27:17.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [----------] 4 tests from LibRadosService (31537 ms total) 2024-02-13T03:27:17.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: 2024-02-13T03:27:17.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [----------] Global test environment tear-down 2024-02-13T03:27:17.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [==========] 4 tests from 1 test suite ran. (31537 ms total) 2024-02-13T03:27:17.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_service: [ PASSED ] 4 tests. 2024-02-13T03:27:18.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:17 smithi161 bash[27879]: audit 2024-02-13T03:27:15.815481+0000 mgr.y (mgr.24473) 244 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:18.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:17 smithi161 bash[27879]: audit 2024-02-13T03:27:16.081257+0000 mon.a (mon.0) 918 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:18.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:17 smithi161 bash[27879]: cluster 2024-02-13T03:27:16.263124+0000 mgr.y (mgr.24473) 245 : cluster [DBG] pgmap v196: 456 pgs: 32 creating+peering, 12 creating+activating, 412 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-02-13T03:27:18.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:17 smithi161 bash[27879]: audit 2024-02-13T03:27:16.464867+0000 mon.c (mon.2) 171 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:18.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:17 smithi161 bash[19769]: audit 2024-02-13T03:27:15.815481+0000 mgr.y (mgr.24473) 244 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:18.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:17 smithi161 bash[19769]: audit 2024-02-13T03:27:16.081257+0000 mon.a (mon.0) 918 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:18.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:17 smithi161 bash[19769]: cluster 2024-02-13T03:27:16.263124+0000 mgr.y (mgr.24473) 245 : cluster [DBG] pgmap v196: 456 pgs: 32 creating+peering, 12 creating+activating, 412 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-02-13T03:27:18.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:17 smithi161 bash[19769]: audit 2024-02-13T03:27:16.464867+0000 mon.c (mon.2) 171 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:17 smithi183 bash[20824]: audit 2024-02-13T03:27:15.815481+0000 mgr.y (mgr.24473) 244 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:17 smithi183 bash[20824]: audit 2024-02-13T03:27:16.081257+0000 mon.a (mon.0) 918 : audit [INF] from='client.25249 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]: dispatch 2024-02-13T03:27:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:17 smithi183 bash[20824]: cluster 2024-02-13T03:27:16.263124+0000 mgr.y (mgr.24473) 245 : cluster [DBG] pgmap v196: 456 pgs: 32 creating+peering, 12 creating+activating, 412 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-02-13T03:27:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:17 smithi183 bash[20824]: audit 2024-02-13T03:27:16.464867+0000 mon.c (mon.2) 171 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:16.626646+0000 mon.a (mon.0) 919 : audit [INF] from='client.26911 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:16.626773+0000 mon.a (mon.0) 920 : audit [INF] from='client.25249 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]': finished 2024-02-13T03:27:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:16.658302+0000 mon.b (mon.1) 75 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:16.658618+0000 mon.c (mon.2) 172 : audit [INF] from='client.? v1:172.21.15.161:0/2077273465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi161-61719-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:16.677283+0000 mgr.y (mgr.24473) 246 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-13T03:27:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:16.680250+0000 mon.b (mon.1) 76 : audit [DBG] from='client.? v1:172.21.15.161:0/1440961648' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-13T03:27:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:17.466385+0000 mon.c (mon.2) 173 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: cluster 2024-02-13T03:27:17.712474+0000 mon.a (mon.0) 921 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-13T03:27:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:17.723517+0000 mon.a (mon.0) 922 : audit [INF] from='client.27764 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi161-61719-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:17.729896+0000 mon.a (mon.0) 923 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: cluster 2024-02-13T03:27:17.742771+0000 mon.a (mon.0) 924 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-13T03:27:19.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:17.766659+0000 mon.a (mon.0) 925 : audit [INF] from='client.27826 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:17.766953+0000 mon.a (mon.0) 926 : audit [INF] from='client.27799 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi161-61993-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:17.768398+0000 mon.b (mon.1) 77 : audit [INF] from='client.? v1:172.21.15.161:0/4100663023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:17.768738+0000 mon.b (mon.1) 78 : audit [INF] from='client.? v1:172.21.15.161:0/3414177897' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi161-61993-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: cluster 2024-02-13T03:27:18.263865+0000 mgr.y (mgr.24473) 247 : cluster [DBG] pgmap v199: 584 pgs: 128 unknown, 32 creating+peering, 12 creating+activating, 412 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 4 op/s 2024-02-13T03:27:19.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[27879]: audit 2024-02-13T03:27:18.467664+0000 mon.c (mon.2) 174 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:19.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:16.626646+0000 mon.a (mon.0) 919 : audit [INF] from='client.26911 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:19.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:16.626773+0000 mon.a (mon.0) 920 : audit [INF] from='client.25249 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]': finished 2024-02-13T03:27:19.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:16.658302+0000 mon.b (mon.1) 75 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:16.658618+0000 mon.c (mon.2) 172 : audit [INF] from='client.? v1:172.21.15.161:0/2077273465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi161-61719-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:16.677283+0000 mgr.y (mgr.24473) 246 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-13T03:27:19.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:16.680250+0000 mon.b (mon.1) 76 : audit [DBG] from='client.? v1:172.21.15.161:0/1440961648' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-13T03:27:19.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:17.466385+0000 mon.c (mon.2) 173 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:19.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: cluster 2024-02-13T03:27:17.712474+0000 mon.a (mon.0) 921 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-13T03:27:19.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:17.723517+0000 mon.a (mon.0) 922 : audit [INF] from='client.27764 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi161-61719-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:17.729896+0000 mon.a (mon.0) 923 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: cluster 2024-02-13T03:27:17.742771+0000 mon.a (mon.0) 924 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-13T03:27:19.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:17.766659+0000 mon.a (mon.0) 925 : audit [INF] from='client.27826 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:17.766953+0000 mon.a (mon.0) 926 : audit [INF] from='client.27799 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi161-61993-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:17.768398+0000 mon.b (mon.1) 77 : audit [INF] from='client.? v1:172.21.15.161:0/4100663023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:17.768738+0000 mon.b (mon.1) 78 : audit [INF] from='client.? v1:172.21.15.161:0/3414177897' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi161-61993-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: cluster 2024-02-13T03:27:18.263865+0000 mgr.y (mgr.24473) 247 : cluster [DBG] pgmap v199: 584 pgs: 128 unknown, 32 creating+peering, 12 creating+activating, 412 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 4 op/s 2024-02-13T03:27:19.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:18 smithi161 bash[19769]: audit 2024-02-13T03:27:18.467664+0000 mon.c (mon.2) 174 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:19.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:16.626646+0000 mon.a (mon.0) 919 : audit [INF] from='client.26911 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:19.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:16.626773+0000 mon.a (mon.0) 920 : audit [INF] from='client.25249 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi161-62063-12"}]': finished 2024-02-13T03:27:19.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:16.658302+0000 mon.b (mon.1) 75 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:16.658618+0000 mon.c (mon.2) 172 : audit [INF] from='client.? v1:172.21.15.161:0/2077273465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi161-61719-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:16.677283+0000 mgr.y (mgr.24473) 246 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-13T03:27:19.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:16.680250+0000 mon.b (mon.1) 76 : audit [DBG] from='client.? v1:172.21.15.161:0/1440961648' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-13T03:27:19.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:17.466385+0000 mon.c (mon.2) 173 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:19.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: cluster 2024-02-13T03:27:17.712474+0000 mon.a (mon.0) 921 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-13T03:27:19.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:17.723517+0000 mon.a (mon.0) 922 : audit [INF] from='client.27764 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi161-61719-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:17.729896+0000 mon.a (mon.0) 923 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: cluster 2024-02-13T03:27:17.742771+0000 mon.a (mon.0) 924 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-13T03:27:19.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:17.766659+0000 mon.a (mon.0) 925 : audit [INF] from='client.27826 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:17.766953+0000 mon.a (mon.0) 926 : audit [INF] from='client.27799 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi161-61993-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:17.768398+0000 mon.b (mon.1) 77 : audit [INF] from='client.? v1:172.21.15.161:0/4100663023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:17.768738+0000 mon.b (mon.1) 78 : audit [INF] from='client.? v1:172.21.15.161:0/3414177897' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi161-61993-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:19.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: cluster 2024-02-13T03:27:18.263865+0000 mgr.y (mgr.24473) 247 : cluster [DBG] pgmap v199: 584 pgs: 128 unknown, 32 creating+peering, 12 creating+activating, 412 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 4 op/s 2024-02-13T03:27:19.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:18 smithi183 bash[20824]: audit 2024-02-13T03:27:18.467664+0000 mon.c (mon.2) 174 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:19.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:27:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:27:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:27:20.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[19769]: cluster 2024-02-13T03:27:18.738316+0000 mon.a (mon.0) 927 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-13T03:27:20.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[19769]: audit 2024-02-13T03:27:18.748264+0000 mon.a (mon.0) 928 : audit [INF] from='client.27764 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi161-61719-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:20.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[19769]: audit 2024-02-13T03:27:18.748913+0000 mon.a (mon.0) 929 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:20.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[19769]: audit 2024-02-13T03:27:18.749063+0000 mon.a (mon.0) 930 : audit [INF] from='client.27826 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:20.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[19769]: audit 2024-02-13T03:27:18.749120+0000 mon.a (mon.0) 931 : audit [INF] from='client.27799 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi161-61993-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:20.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[19769]: cluster 2024-02-13T03:27:18.749191+0000 mon.a (mon.0) 932 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-13T03:27:20.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[19769]: audit 2024-02-13T03:27:18.869667+0000 mon.b (mon.1) 79 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:20.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[19769]: audit 2024-02-13T03:27:19.356180+0000 mon.a (mon.0) 933 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:20.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[19769]: audit 2024-02-13T03:27:19.468626+0000 mon.c (mon.2) 175 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:20.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[19769]: audit 2024-02-13T03:27:19.749918+0000 mon.a (mon.0) 934 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:20.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[19769]: cluster 2024-02-13T03:27:19.750033+0000 mon.a (mon.0) 935 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-13T03:27:20.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[27879]: cluster 2024-02-13T03:27:18.738316+0000 mon.a (mon.0) 927 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-13T03:27:20.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[27879]: audit 2024-02-13T03:27:18.748264+0000 mon.a (mon.0) 928 : audit [INF] from='client.27764 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi161-61719-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:20.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[27879]: audit 2024-02-13T03:27:18.748913+0000 mon.a (mon.0) 929 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:20.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[27879]: audit 2024-02-13T03:27:18.749063+0000 mon.a (mon.0) 930 : audit [INF] from='client.27826 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:20.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[27879]: audit 2024-02-13T03:27:18.749120+0000 mon.a (mon.0) 931 : audit [INF] from='client.27799 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi161-61993-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:20.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[27879]: cluster 2024-02-13T03:27:18.749191+0000 mon.a (mon.0) 932 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-13T03:27:20.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[27879]: audit 2024-02-13T03:27:18.869667+0000 mon.b (mon.1) 79 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:20.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[27879]: audit 2024-02-13T03:27:19.356180+0000 mon.a (mon.0) 933 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:20.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[27879]: audit 2024-02-13T03:27:19.468626+0000 mon.c (mon.2) 175 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:20.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[27879]: audit 2024-02-13T03:27:19.749918+0000 mon.a (mon.0) 934 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:20.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:19 smithi161 bash[27879]: cluster 2024-02-13T03:27:19.750033+0000 mon.a (mon.0) 935 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-13T03:27:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:19 smithi183 bash[20824]: cluster 2024-02-13T03:27:18.738316+0000 mon.a (mon.0) 927 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-13T03:27:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:19 smithi183 bash[20824]: audit 2024-02-13T03:27:18.748264+0000 mon.a (mon.0) 928 : audit [INF] from='client.27764 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi161-61719-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:19 smithi183 bash[20824]: audit 2024-02-13T03:27:18.748913+0000 mon.a (mon.0) 929 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:19 smithi183 bash[20824]: audit 2024-02-13T03:27:18.749063+0000 mon.a (mon.0) 930 : audit [INF] from='client.27826 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:19 smithi183 bash[20824]: audit 2024-02-13T03:27:18.749120+0000 mon.a (mon.0) 931 : audit [INF] from='client.27799 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi161-61993-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:19 smithi183 bash[20824]: cluster 2024-02-13T03:27:18.749191+0000 mon.a (mon.0) 932 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-13T03:27:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:19 smithi183 bash[20824]: audit 2024-02-13T03:27:18.869667+0000 mon.b (mon.1) 79 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:19 smithi183 bash[20824]: audit 2024-02-13T03:27:19.356180+0000 mon.a (mon.0) 933 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:19 smithi183 bash[20824]: audit 2024-02-13T03:27:19.468626+0000 mon.c (mon.2) 175 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:19 smithi183 bash[20824]: audit 2024-02-13T03:27:19.749918+0000 mon.a (mon.0) 934 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:20.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:19 smithi183 bash[20824]: cluster 2024-02-13T03:27:19.750033+0000 mon.a (mon.0) 935 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-13T03:27:21.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:27:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:27:21] "GET /metrics HTTP/1.1" 200 201936 "" "Prometheus/2.33.4" 2024-02-13T03:27:22.066 INFO:tasks.workunit.client.0.smithi161.stdout:zonegroup_name": "default" 2024-02-13T03:27:22.066 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: }, 2024-02-13T03:27:22.066 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: "task_status": {} 2024-02-13T03:27:22.066 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: } 2024-02-13T03:27:22.066 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: } 2024-02-13T03:27:22.066 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: } 2024-02-13T03:27:22.066 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: } 2024-02-13T03:27:22.066 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: } 2024-02-13T03:27:22.066 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: 2024-02-13T03:27:22.066 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: attempt 1 of 20 2024-02-13T03:27:22.066 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (10565 ms) 2024-02-13T03:27:22.067 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (35651 ms total) 2024-02-13T03:27:22.067 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: 2024-02-13T03:27:22.067 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [----------] Global test environment tear-down 2024-02-13T03:27:22.067 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (35651 ms total) 2024-02-13T03:27:22.067 INFO:tasks.workunit.client.0.smithi161.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-02-13T03:27:22.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:21 smithi183 bash[20824]: audit 2024-02-13T03:27:19.768374+0000 mon.b (mon.1) 80 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-7"}]: dispatch 2024-02-13T03:27:22.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:21 smithi183 bash[20824]: audit 2024-02-13T03:27:19.820741+0000 mon.a (mon.0) 936 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-7"}]: dispatch 2024-02-13T03:27:22.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:21 smithi183 bash[20824]: audit 2024-02-13T03:27:19.838641+0000 mon.a (mon.0) 937 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:22.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:21 smithi183 bash[20824]: audit 2024-02-13T03:27:20.470537+0000 mon.c (mon.2) 176 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:22.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:21 smithi161 bash[27879]: audit 2024-02-13T03:27:19.768374+0000 mon.b (mon.1) 80 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-7"}]: dispatch 2024-02-13T03:27:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:21 smithi161 bash[27879]: audit 2024-02-13T03:27:19.820741+0000 mon.a (mon.0) 936 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-7"}]: dispatch 2024-02-13T03:27:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:21 smithi161 bash[27879]: audit 2024-02-13T03:27:19.838641+0000 mon.a (mon.0) 937 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:21 smithi161 bash[27879]: audit 2024-02-13T03:27:20.470537+0000 mon.c (mon.2) 176 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:22.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:21 smithi161 bash[19769]: audit 2024-02-13T03:27:19.768374+0000 mon.b (mon.1) 80 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-7"}]: dispatch 2024-02-13T03:27:22.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:21 smithi161 bash[19769]: audit 2024-02-13T03:27:19.820741+0000 mon.a (mon.0) 936 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-7"}]: dispatch 2024-02-13T03:27:22.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:21 smithi161 bash[19769]: audit 2024-02-13T03:27:19.838641+0000 mon.a (mon.0) 937 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:22.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:21 smithi161 bash[19769]: audit 2024-02-13T03:27:20.470537+0000 mon.c (mon.2) 176 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:23.007 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-02-13T03:27:23.007 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-02-13T03:27:23.007 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-02-13T03:27:23.008 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-02-13T03:27:23.008 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-02-13T03:27:23.008 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-02-13T03:27:23.008 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-02-13T03:27:23.008 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: 2024-02-13T03:27:23.008 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-02-13T03:27:23.008 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-02-13T03:27:23.008 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (13756 ms) 2024-02-13T03:27:23.008 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-02-13T03:27:23.008 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (6824 ms) 2024-02-13T03:27:23.008 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-02-13T03:27:23.008 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (2492 ms) 2024-02-13T03:27:23.009 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-02-13T03:27:23.009 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (3020 ms) 2024-02-13T03:27:23.009 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-02-13T03:27:23.009 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3028 ms) 2024-02-13T03:27:23.009 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-02-13T03:27:23.009 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (4236 ms) 2024-02-13T03:27:23.009 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-02-13T03:27:23.009 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (3226 ms) 2024-02-13T03:27:23.009 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (36582 ms total) 2024-02-13T03:27:23.009 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: 2024-02-13T03:27:23.009 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-02-13T03:27:23.009 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (36582 ms total) 2024-02-13T03:27:23.009 INFO:tasks.workunit.client.0.smithi161.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-02-13T03:27:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: cluster 2024-02-13T03:27:20.264435+0000 mgr.y (mgr.24473) 248 : cluster [DBG] pgmap v202: 456 pgs: 96 unknown, 12 creating+activating, 348 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:20.927428+0000 mon.a (mon.0) 938 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-7"}]': finished 2024-02-13T03:27:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:20.947655+0000 mon.b (mon.1) 81 : audit [INF] from='client.? v1:172.21.15.161:0/2512757190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:20.986346+0000 mon.b (mon.1) 82 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: cluster 2024-02-13T03:27:21.455618+0000 mon.a (mon.0) 939 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-13T03:27:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:21.463369+0000 mon.a (mon.0) 940 : audit [INF] from='client.31663 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:21.464145+0000 mon.a (mon.0) 941 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:23.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:21.471583+0000 mon.c (mon.2) 177 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:23.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:21.960806+0000 mon.a (mon.0) 942 : audit [INF] from='client.? v1:172.21.15.161:0/650760154' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:21.980919+0000 mon.a (mon.0) 943 : audit [INF] from='client.31663 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:23.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:21.981018+0000 mon.a (mon.0) 944 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:27:23.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: cluster 2024-02-13T03:27:21.981199+0000 mon.a (mon.0) 945 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-13T03:27:23.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:22.002058+0000 mon.b (mon.1) 83 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7"}]: dispatch 2024-02-13T03:27:23.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:22.014034+0000 mon.b (mon.1) 84 : audit [INF] from='client.? v1:172.21.15.161:0/514138712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi161-61719-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:22.014345+0000 mon.b (mon.1) 85 : audit [INF] from='client.? v1:172.21.15.161:0/1276613639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:22.024004+0000 mon.a (mon.0) 946 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7"}]: dispatch 2024-02-13T03:27:23.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:22.024466+0000 mon.a (mon.0) 947 : audit [INF] from='client.31687 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi161-61719-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:22.024743+0000 mon.a (mon.0) 948 : audit [INF] from='client.32722 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:22.041649+0000 mgr.y (mgr.24473) 249 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-13T03:27:23.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:22.043052+0000 mon.b (mon.1) 86 : audit [DBG] from='client.? v1:172.21.15.161:0/26723338' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-13T03:27:23.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: cluster 2024-02-13T03:27:22.265127+0000 mgr.y (mgr.24473) 250 : cluster [DBG] pgmap v205: 552 pgs: 128 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:27:23.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:22 smithi183 bash[20824]: audit 2024-02-13T03:27:22.472426+0000 mon.c (mon.2) 178 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: cluster 2024-02-13T03:27:20.264435+0000 mgr.y (mgr.24473) 248 : cluster [DBG] pgmap v202: 456 pgs: 96 unknown, 12 creating+activating, 348 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:20.927428+0000 mon.a (mon.0) 938 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-7"}]': finished 2024-02-13T03:27:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:20.947655+0000 mon.b (mon.1) 81 : audit [INF] from='client.? v1:172.21.15.161:0/2512757190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:20.986346+0000 mon.b (mon.1) 82 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: cluster 2024-02-13T03:27:21.455618+0000 mon.a (mon.0) 939 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-13T03:27:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:21.463369+0000 mon.a (mon.0) 940 : audit [INF] from='client.31663 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:21.464145+0000 mon.a (mon.0) 941 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:21.471583+0000 mon.c (mon.2) 177 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:21.960806+0000 mon.a (mon.0) 942 : audit [INF] from='client.? v1:172.21.15.161:0/650760154' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:21.980919+0000 mon.a (mon.0) 943 : audit [INF] from='client.31663 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:21.981018+0000 mon.a (mon.0) 944 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:27:23.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: cluster 2024-02-13T03:27:21.981199+0000 mon.a (mon.0) 945 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-13T03:27:23.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:22.002058+0000 mon.b (mon.1) 83 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7"}]: dispatch 2024-02-13T03:27:23.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:22.014034+0000 mon.b (mon.1) 84 : audit [INF] from='client.? v1:172.21.15.161:0/514138712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi161-61719-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:22.014345+0000 mon.b (mon.1) 85 : audit [INF] from='client.? v1:172.21.15.161:0/1276613639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:22.024004+0000 mon.a (mon.0) 946 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7"}]: dispatch 2024-02-13T03:27:23.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:22.024466+0000 mon.a (mon.0) 947 : audit [INF] from='client.31687 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi161-61719-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:22.024743+0000 mon.a (mon.0) 948 : audit [INF] from='client.32722 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:22.041649+0000 mgr.y (mgr.24473) 249 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-13T03:27:23.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:22.043052+0000 mon.b (mon.1) 86 : audit [DBG] from='client.? v1:172.21.15.161:0/26723338' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-13T03:27:23.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: cluster 2024-02-13T03:27:22.265127+0000 mgr.y (mgr.24473) 250 : cluster [DBG] pgmap v205: 552 pgs: 128 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:27:23.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[19769]: audit 2024-02-13T03:27:22.472426+0000 mon.c (mon.2) 178 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: cluster 2024-02-13T03:27:20.264435+0000 mgr.y (mgr.24473) 248 : cluster [DBG] pgmap v202: 456 pgs: 96 unknown, 12 creating+activating, 348 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:20.927428+0000 mon.a (mon.0) 938 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-7"}]': finished 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:20.947655+0000 mon.b (mon.1) 81 : audit [INF] from='client.? v1:172.21.15.161:0/2512757190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:20.986346+0000 mon.b (mon.1) 82 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: cluster 2024-02-13T03:27:21.455618+0000 mon.a (mon.0) 939 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:21.463369+0000 mon.a (mon.0) 940 : audit [INF] from='client.31663 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:21.464145+0000 mon.a (mon.0) 941 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:21.471583+0000 mon.c (mon.2) 177 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:21.960806+0000 mon.a (mon.0) 942 : audit [INF] from='client.? v1:172.21.15.161:0/650760154' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:21.980919+0000 mon.a (mon.0) 943 : audit [INF] from='client.31663 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-62195-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:21.981018+0000 mon.a (mon.0) 944 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: cluster 2024-02-13T03:27:21.981199+0000 mon.a (mon.0) 945 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:22.002058+0000 mon.b (mon.1) 83 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7"}]: dispatch 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:22.014034+0000 mon.b (mon.1) 84 : audit [INF] from='client.? v1:172.21.15.161:0/514138712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi161-61719-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:22.014345+0000 mon.b (mon.1) 85 : audit [INF] from='client.? v1:172.21.15.161:0/1276613639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:22.024004+0000 mon.a (mon.0) 946 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7"}]: dispatch 2024-02-13T03:27:23.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:22.024466+0000 mon.a (mon.0) 947 : audit [INF] from='client.31687 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi161-61719-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:22.024743+0000 mon.a (mon.0) 948 : audit [INF] from='client.32722 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:23.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:22.041649+0000 mgr.y (mgr.24473) 249 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-13T03:27:23.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:22.043052+0000 mon.b (mon.1) 86 : audit [DBG] from='client.? v1:172.21.15.161:0/26723338' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-13T03:27:23.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: cluster 2024-02-13T03:27:22.265127+0000 mgr.y (mgr.24473) 250 : cluster [DBG] pgmap v205: 552 pgs: 128 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:27:23.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:22 smithi161 bash[27879]: audit 2024-02-13T03:27:22.472426+0000 mon.c (mon.2) 178 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:24.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:23 smithi161 bash[27879]: audit 2024-02-13T03:27:22.992333+0000 mon.a (mon.0) 949 : audit [INF] from='client.? v1:172.21.15.161:0/650760154' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:24.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:23 smithi161 bash[27879]: audit 2024-02-13T03:27:22.992448+0000 mon.a (mon.0) 950 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7"}]': finished 2024-02-13T03:27:24.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:23 smithi161 bash[27879]: audit 2024-02-13T03:27:22.992563+0000 mon.a (mon.0) 951 : audit [INF] from='client.31687 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi161-61719-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:24.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:23 smithi161 bash[27879]: audit 2024-02-13T03:27:22.992636+0000 mon.a (mon.0) 952 : audit [INF] from='client.32722 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:24.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:23 smithi161 bash[27879]: cluster 2024-02-13T03:27:22.992818+0000 mon.a (mon.0) 953 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-13T03:27:24.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:23 smithi161 bash[27879]: audit 2024-02-13T03:27:23.473373+0000 mon.c (mon.2) 179 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:23 smithi161 bash[19769]: audit 2024-02-13T03:27:22.992333+0000 mon.a (mon.0) 949 : audit [INF] from='client.? v1:172.21.15.161:0/650760154' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:23 smithi161 bash[19769]: audit 2024-02-13T03:27:22.992448+0000 mon.a (mon.0) 950 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7"}]': finished 2024-02-13T03:27:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:23 smithi161 bash[19769]: audit 2024-02-13T03:27:22.992563+0000 mon.a (mon.0) 951 : audit [INF] from='client.31687 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi161-61719-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:24.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:23 smithi161 bash[19769]: audit 2024-02-13T03:27:22.992636+0000 mon.a (mon.0) 952 : audit [INF] from='client.32722 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:24.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:23 smithi161 bash[19769]: cluster 2024-02-13T03:27:22.992818+0000 mon.a (mon.0) 953 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-13T03:27:24.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:23 smithi161 bash[19769]: audit 2024-02-13T03:27:23.473373+0000 mon.c (mon.2) 179 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:24.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:23 smithi183 bash[20824]: audit 2024-02-13T03:27:22.992333+0000 mon.a (mon.0) 949 : audit [INF] from='client.? v1:172.21.15.161:0/650760154' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:23 smithi183 bash[20824]: audit 2024-02-13T03:27:22.992448+0000 mon.a (mon.0) 950 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-7"}]': finished 2024-02-13T03:27:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:23 smithi183 bash[20824]: audit 2024-02-13T03:27:22.992563+0000 mon.a (mon.0) 951 : audit [INF] from='client.31687 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi161-61719-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:23 smithi183 bash[20824]: audit 2024-02-13T03:27:22.992636+0000 mon.a (mon.0) 952 : audit [INF] from='client.32722 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:23 smithi183 bash[20824]: cluster 2024-02-13T03:27:22.992818+0000 mon.a (mon.0) 953 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-13T03:27:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:23 smithi183 bash[20824]: audit 2024-02-13T03:27:23.473373+0000 mon.c (mon.2) 179 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:25.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:25 smithi161 bash[27879]: cluster 2024-02-13T03:27:24.002603+0000 mon.a (mon.0) 954 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-13T03:27:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:25 smithi161 bash[27879]: audit 2024-02-13T03:27:24.127951+0000 mon.b (mon.1) 87 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:25 smithi161 bash[27879]: audit 2024-02-13T03:27:24.478993+0000 mon.c (mon.2) 180 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:25 smithi161 bash[19769]: cluster 2024-02-13T03:27:24.002603+0000 mon.a (mon.0) 954 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-13T03:27:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:25 smithi161 bash[19769]: audit 2024-02-13T03:27:24.127951+0000 mon.b (mon.1) 87 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:25 smithi161 bash[19769]: audit 2024-02-13T03:27:24.478993+0000 mon.c (mon.2) 180 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:25.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:27:25 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:27:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:25 smithi183 bash[20824]: cluster 2024-02-13T03:27:24.002603+0000 mon.a (mon.0) 954 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-13T03:27:25.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:25 smithi183 bash[20824]: audit 2024-02-13T03:27:24.127951+0000 mon.b (mon.1) 87 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:25.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:25 smithi183 bash[20824]: audit 2024-02-13T03:27:24.478993+0000 mon.c (mon.2) 180 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: cluster 2024-02-13T03:27:24.265676+0000 mgr.y (mgr.24473) 251 : cluster [DBG] pgmap v208: 424 pgs: 64 unknown, 360 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:27:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:24.555111+0000 mon.a (mon.0) 955 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:24.562809+0000 mon.b (mon.1) 88 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:25.069917+0000 mon.a (mon.0) 956 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:25.094367+0000 mon.b (mon.1) 89 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:25.118380+0000 mon.c (mon.2) 181 : audit [INF] from='client.? v1:172.21.15.161:0/4175774331' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi161-61719-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:25.481084+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: cluster 2024-02-13T03:27:25.612048+0000 mon.a (mon.0) 957 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-13T03:27:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:25.616498+0000 mon.b (mon.1) 90 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:25.618754+0000 mon.a (mon.0) 958 : audit [INF] from='client.34973 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi161-61719-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:25.619251+0000 mon.a (mon.0) 959 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:25.619868+0000 mon.a (mon.0) 960 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:25.696016+0000 mon.c (mon.2) 183 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4"}]: dispatch 2024-02-13T03:27:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:25.698478+0000 mon.a (mon.0) 961 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4"}]: dispatch 2024-02-13T03:27:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:26.078734+0000 mon.a (mon.0) 962 : audit [INF] from='client.34973 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi161-61719-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:26.078869+0000 mon.a (mon.0) 963 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:26.078968+0000 mon.a (mon.0) 964 : audit [INF] from='client.35305 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:26.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:26.079053+0000 mon.a (mon.0) 965 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4"}]': finished 2024-02-13T03:27:26.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: cluster 2024-02-13T03:27:26.079102+0000 mon.a (mon.0) 966 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-13T03:27:26.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:26.088398+0000 mon.a (mon.0) 967 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi161-61936-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:26.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:26.090025+0000 mon.b (mon.1) 91 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi161-61936-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:26.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:26.097940+0000 mon.c (mon.2) 184 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]: dispatch 2024-02-13T03:27:26.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:26.101201+0000 mon.a (mon.0) 968 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]: dispatch 2024-02-13T03:27:26.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:26.109932+0000 mon.c (mon.2) 185 : audit [INF] from='client.? v1:172.21.15.161:0/1342157760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:26.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:26.110332+0000 mon.a (mon.0) 969 : audit [INF] from='client.34970 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:26.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:26.154390+0000 mon.a (mon.0) 970 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:26.989 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:26 smithi183 bash[20824]: audit 2024-02-13T03:27:26.156654+0000 mon.b (mon.1) 92 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:27.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: cluster 2024-02-13T03:27:24.265676+0000 mgr.y (mgr.24473) 251 : cluster [DBG] pgmap v208: 424 pgs: 64 unknown, 360 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:27:27.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:24.555111+0000 mon.a (mon.0) 955 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:27.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:24.562809+0000 mon.b (mon.1) 88 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:27.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:25.069917+0000 mon.a (mon.0) 956 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:27.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:25.094367+0000 mon.b (mon.1) 89 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:27.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:25.118380+0000 mon.c (mon.2) 181 : audit [INF] from='client.? v1:172.21.15.161:0/4175774331' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi161-61719-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:25.481084+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: cluster 2024-02-13T03:27:25.612048+0000 mon.a (mon.0) 957 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:25.616498+0000 mon.b (mon.1) 90 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:25.618754+0000 mon.a (mon.0) 958 : audit [INF] from='client.34973 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi161-61719-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:25.619251+0000 mon.a (mon.0) 959 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:25.619868+0000 mon.a (mon.0) 960 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:25.696016+0000 mon.c (mon.2) 183 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4"}]: dispatch 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:25.698478+0000 mon.a (mon.0) 961 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4"}]: dispatch 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:26.078734+0000 mon.a (mon.0) 962 : audit [INF] from='client.34973 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi161-61719-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:26.078869+0000 mon.a (mon.0) 963 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:26.078968+0000 mon.a (mon.0) 964 : audit [INF] from='client.35305 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:26.079053+0000 mon.a (mon.0) 965 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4"}]': finished 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: cluster 2024-02-13T03:27:26.079102+0000 mon.a (mon.0) 966 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-13T03:27:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:26.088398+0000 mon.a (mon.0) 967 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi161-61936-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:27.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:26.090025+0000 mon.b (mon.1) 91 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi161-61936-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:27.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:26.097940+0000 mon.c (mon.2) 184 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]: dispatch 2024-02-13T03:27:27.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:26.101201+0000 mon.a (mon.0) 968 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]: dispatch 2024-02-13T03:27:27.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:26.109932+0000 mon.c (mon.2) 185 : audit [INF] from='client.? v1:172.21.15.161:0/1342157760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:27.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:26.110332+0000 mon.a (mon.0) 969 : audit [INF] from='client.34970 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:27.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:26.154390+0000 mon.a (mon.0) 970 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:27.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[27879]: audit 2024-02-13T03:27:26.156654+0000 mon.b (mon.1) 92 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:27.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: cluster 2024-02-13T03:27:24.265676+0000 mgr.y (mgr.24473) 251 : cluster [DBG] pgmap v208: 424 pgs: 64 unknown, 360 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:27:27.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:24.555111+0000 mon.a (mon.0) 955 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:27.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:24.562809+0000 mon.b (mon.1) 88 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:27.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:25.069917+0000 mon.a (mon.0) 956 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:27.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:25.094367+0000 mon.b (mon.1) 89 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:27.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:25.118380+0000 mon.c (mon.2) 181 : audit [INF] from='client.? v1:172.21.15.161:0/4175774331' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi161-61719-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:27.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:25.481084+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:27.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: cluster 2024-02-13T03:27:25.612048+0000 mon.a (mon.0) 957 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-13T03:27:27.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:25.616498+0000 mon.b (mon.1) 90 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:27.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:25.618754+0000 mon.a (mon.0) 958 : audit [INF] from='client.34973 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi161-61719-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:27.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:25.619251+0000 mon.a (mon.0) 959 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:27.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:25.619868+0000 mon.a (mon.0) 960 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:27.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:25.696016+0000 mon.c (mon.2) 183 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4"}]: dispatch 2024-02-13T03:27:27.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:25.698478+0000 mon.a (mon.0) 961 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4"}]: dispatch 2024-02-13T03:27:27.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:26.078734+0000 mon.a (mon.0) 962 : audit [INF] from='client.34973 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi161-61719-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:27.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:26.078869+0000 mon.a (mon.0) 963 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:27.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:26.078968+0000 mon.a (mon.0) 964 : audit [INF] from='client.35305 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:27.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:26.079053+0000 mon.a (mon.0) 965 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4"}]': finished 2024-02-13T03:27:27.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: cluster 2024-02-13T03:27:26.079102+0000 mon.a (mon.0) 966 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-13T03:27:27.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:26.088398+0000 mon.a (mon.0) 967 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi161-61936-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:27.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:26.090025+0000 mon.b (mon.1) 91 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi161-61936-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:27.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:26.097940+0000 mon.c (mon.2) 184 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]: dispatch 2024-02-13T03:27:27.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:26.101201+0000 mon.a (mon.0) 968 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]: dispatch 2024-02-13T03:27:27.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:26.109932+0000 mon.c (mon.2) 185 : audit [INF] from='client.? v1:172.21.15.161:0/1342157760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:27.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:26.110332+0000 mon.a (mon.0) 969 : audit [INF] from='client.34970 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:27.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:26.154390+0000 mon.a (mon.0) 970 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:27.181 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:26 smithi161 bash[19769]: audit 2024-02-13T03:27:26.156654+0000 mon.b (mon.1) 92 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: audit 2024-02-13T03:27:25.826406+0000 mgr.y (mgr.24473) 252 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: cluster 2024-02-13T03:27:26.266273+0000 mgr.y (mgr.24473) 253 : cluster [DBG] pgmap v211: 456 pgs: 42 creating+peering, 54 unknown, 360 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-13T03:27:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: audit 2024-02-13T03:27:26.482666+0000 mon.c (mon.2) 186 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: cluster 2024-02-13T03:27:27.076966+0000 mon.a (mon.0) 971 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:27:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: audit 2024-02-13T03:27:27.483818+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: audit 2024-02-13T03:27:27.596435+0000 mon.a (mon.0) 972 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]': finished 2024-02-13T03:27:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: audit 2024-02-13T03:27:27.596530+0000 mon.a (mon.0) 973 : audit [INF] from='client.34970 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:28.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: audit 2024-02-13T03:27:27.596715+0000 mon.a (mon.0) 974 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:28.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: cluster 2024-02-13T03:27:27.596877+0000 mon.a (mon.0) 975 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-13T03:27:28.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: audit 2024-02-13T03:27:27.598585+0000 mon.c (mon.2) 188 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:27:28.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: audit 2024-02-13T03:27:27.599069+0000 mon.c (mon.2) 189 : audit [INF] from='client.? v1:172.21.15.161:0/3341455651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:28.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: audit 2024-02-13T03:27:27.602409+0000 mon.b (mon.1) 93 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-9"}]: dispatch 2024-02-13T03:27:28.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: audit 2024-02-13T03:27:27.614239+0000 mon.a (mon.0) 976 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:27:28.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: audit 2024-02-13T03:27:27.614665+0000 mon.a (mon.0) 977 : audit [INF] from='client.36692 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:28.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:28 smithi183 bash[20824]: audit 2024-02-13T03:27:27.615603+0000 mon.a (mon.0) 978 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-9"}]: dispatch 2024-02-13T03:27:28.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: audit 2024-02-13T03:27:25.826406+0000 mgr.y (mgr.24473) 252 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:28.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: cluster 2024-02-13T03:27:26.266273+0000 mgr.y (mgr.24473) 253 : cluster [DBG] pgmap v211: 456 pgs: 42 creating+peering, 54 unknown, 360 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-13T03:27:28.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: audit 2024-02-13T03:27:26.482666+0000 mon.c (mon.2) 186 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:28.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: cluster 2024-02-13T03:27:27.076966+0000 mon.a (mon.0) 971 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:27:28.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: audit 2024-02-13T03:27:27.483818+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:28.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: audit 2024-02-13T03:27:27.596435+0000 mon.a (mon.0) 972 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]': finished 2024-02-13T03:27:28.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: audit 2024-02-13T03:27:27.596530+0000 mon.a (mon.0) 973 : audit [INF] from='client.34970 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:28.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: audit 2024-02-13T03:27:27.596715+0000 mon.a (mon.0) 974 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:28.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: cluster 2024-02-13T03:27:27.596877+0000 mon.a (mon.0) 975 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-13T03:27:28.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: audit 2024-02-13T03:27:27.598585+0000 mon.c (mon.2) 188 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:27:28.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: audit 2024-02-13T03:27:27.599069+0000 mon.c (mon.2) 189 : audit [INF] from='client.? v1:172.21.15.161:0/3341455651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:28.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: audit 2024-02-13T03:27:27.602409+0000 mon.b (mon.1) 93 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-9"}]: dispatch 2024-02-13T03:27:28.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: audit 2024-02-13T03:27:27.614239+0000 mon.a (mon.0) 976 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:27:28.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: audit 2024-02-13T03:27:27.614665+0000 mon.a (mon.0) 977 : audit [INF] from='client.36692 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:28.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[27879]: audit 2024-02-13T03:27:27.615603+0000 mon.a (mon.0) 978 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-9"}]: dispatch 2024-02-13T03:27:28.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: audit 2024-02-13T03:27:25.826406+0000 mgr.y (mgr.24473) 252 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:28.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: cluster 2024-02-13T03:27:26.266273+0000 mgr.y (mgr.24473) 253 : cluster [DBG] pgmap v211: 456 pgs: 42 creating+peering, 54 unknown, 360 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-13T03:27:28.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: audit 2024-02-13T03:27:26.482666+0000 mon.c (mon.2) 186 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:28.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: cluster 2024-02-13T03:27:27.076966+0000 mon.a (mon.0) 971 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:27:28.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: audit 2024-02-13T03:27:27.483818+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:28.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: audit 2024-02-13T03:27:27.596435+0000 mon.a (mon.0) 972 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6"}]': finished 2024-02-13T03:27:28.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: audit 2024-02-13T03:27:27.596530+0000 mon.a (mon.0) 973 : audit [INF] from='client.34970 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi161-61726-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:28.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: audit 2024-02-13T03:27:27.596715+0000 mon.a (mon.0) 974 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:28.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: cluster 2024-02-13T03:27:27.596877+0000 mon.a (mon.0) 975 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-13T03:27:28.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: audit 2024-02-13T03:27:27.598585+0000 mon.c (mon.2) 188 : audit [INF] from='client.? v1:172.21.15.161:0/3586838363' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:27:28.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: audit 2024-02-13T03:27:27.599069+0000 mon.c (mon.2) 189 : audit [INF] from='client.? v1:172.21.15.161:0/3341455651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:28.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: audit 2024-02-13T03:27:27.602409+0000 mon.b (mon.1) 93 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-9"}]: dispatch 2024-02-13T03:27:28.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: audit 2024-02-13T03:27:27.614239+0000 mon.a (mon.0) 976 : audit [INF] from='client.25223 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:27:28.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: audit 2024-02-13T03:27:27.614665+0000 mon.a (mon.0) 977 : audit [INF] from='client.36692 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:28.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:28 smithi161 bash[19769]: audit 2024-02-13T03:27:27.615603+0000 mon.a (mon.0) 978 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-9"}]: dispatch 2024-02-13T03:27:29.350 INFO:tasks.workunit.client.0.smithi161.stdout:tch_notify_pp: flushed 2024-02-13T03:27:29.350 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3010 ms) 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: trying... 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: handle_notify cookie 94224382735760 notify_id 360777252872 notifier_gid 25223 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: timed out 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: flushing 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: flushed 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3007 ms) 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: List watches 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: notify2 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: handle_notify cookie 94224382735760 notify_id 369367187461 notifier_gid 25223 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: notify2 done 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: watch_check 2024-02-13T03:27:29.351 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: unwatch2 2024-02-13T03:27:29.352 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: flushing 2024-02-13T03:27:29.352 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: done 2024-02-13T03:27:29.352 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3017 ms) 2024-02-13T03:27:29.352 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-02-13T03:27:29.352 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: List watches 2024-02-13T03:27:29.352 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: notify2 2024-02-13T03:27:29.352 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: handle_notify cookie 94224382755408 notify_id 382252089353 notifier_gid 25223 2024-02-13T03:27:29.352 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: notify2 done 2024-02-13T03:27:29.352 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: watch_check 2024-02-13T03:27:29.352 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: unwatch2 2024-02-13T03:27:29.352 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: flushing 2024-02-13T03:27:29.352 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: done 2024-02-13T03:27:29.352 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3009 ms) 2024-02-13T03:27:29.353 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16211 ms total) 2024-02-13T03:27:29.353 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: 2024-02-13T03:27:29.353 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-02-13T03:27:29.353 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (42976 ms total) 2024-02-13T03:27:29.353 INFO:tasks.workunit.client.0.smithi161.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-02-13T03:27:29.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[19769]: audit 2024-02-13T03:27:28.215346+0000 mon.a (mon.0) 979 : audit [INF] from='client.35305 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi161-61936-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]': finished 2024-02-13T03:27:29.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[19769]: audit 2024-02-13T03:27:28.215551+0000 mon.a (mon.0) 980 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-13T03:27:29.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[19769]: audit 2024-02-13T03:27:28.215610+0000 mon.a (mon.0) 981 : audit [INF] from='client.36692 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:29.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[19769]: audit 2024-02-13T03:27:28.215671+0000 mon.a (mon.0) 982 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-9"}]': finished 2024-02-13T03:27:29.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[19769]: cluster 2024-02-13T03:27:28.219958+0000 mon.a (mon.0) 983 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-13T03:27:29.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[19769]: audit 2024-02-13T03:27:28.228665+0000 mon.a (mon.0) 984 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-9", "mode": "writeback"}]: dispatch 2024-02-13T03:27:29.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[19769]: audit 2024-02-13T03:27:28.229605+0000 mon.b (mon.1) 94 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-9", "mode": "writeback"}]: dispatch 2024-02-13T03:27:29.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[19769]: audit 2024-02-13T03:27:28.229834+0000 mon.c (mon.2) 190 : audit [INF] from='client.? v1:172.21.15.161:0/1996406190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi161-61719-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:29.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[19769]: audit 2024-02-13T03:27:28.232887+0000 mon.a (mon.0) 985 : audit [INF] from='client.38024 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi161-61719-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:29.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[19769]: cluster 2024-02-13T03:27:28.266921+0000 mgr.y (mgr.24473) 254 : cluster [DBG] pgmap v214: 460 pgs: 20 creating+peering, 84 unknown, 356 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 op/s 2024-02-13T03:27:29.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[19769]: audit 2024-02-13T03:27:28.486439+0000 mon.c (mon.2) 191 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:29.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[27879]: audit 2024-02-13T03:27:28.215346+0000 mon.a (mon.0) 979 : audit [INF] from='client.35305 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi161-61936-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]': finished 2024-02-13T03:27:29.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[27879]: audit 2024-02-13T03:27:28.215551+0000 mon.a (mon.0) 980 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-13T03:27:29.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[27879]: audit 2024-02-13T03:27:28.215610+0000 mon.a (mon.0) 981 : audit [INF] from='client.36692 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:29.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[27879]: audit 2024-02-13T03:27:28.215671+0000 mon.a (mon.0) 982 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-9"}]': finished 2024-02-13T03:27:29.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[27879]: cluster 2024-02-13T03:27:28.219958+0000 mon.a (mon.0) 983 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-13T03:27:29.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[27879]: audit 2024-02-13T03:27:28.228665+0000 mon.a (mon.0) 984 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-9", "mode": "writeback"}]: dispatch 2024-02-13T03:27:29.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[27879]: audit 2024-02-13T03:27:28.229605+0000 mon.b (mon.1) 94 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-9", "mode": "writeback"}]: dispatch 2024-02-13T03:27:29.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[27879]: audit 2024-02-13T03:27:28.229834+0000 mon.c (mon.2) 190 : audit [INF] from='client.? v1:172.21.15.161:0/1996406190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi161-61719-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:29.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[27879]: audit 2024-02-13T03:27:28.232887+0000 mon.a (mon.0) 985 : audit [INF] from='client.38024 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi161-61719-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:29.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[27879]: cluster 2024-02-13T03:27:28.266921+0000 mgr.y (mgr.24473) 254 : cluster [DBG] pgmap v214: 460 pgs: 20 creating+peering, 84 unknown, 356 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 op/s 2024-02-13T03:27:29.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:29 smithi161 bash[27879]: audit 2024-02-13T03:27:28.486439+0000 mon.c (mon.2) 191 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:29.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:27:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:27:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:27:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:29 smithi183 bash[20824]: audit 2024-02-13T03:27:28.215346+0000 mon.a (mon.0) 979 : audit [INF] from='client.35305 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi161-61936-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]': finished 2024-02-13T03:27:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:29 smithi183 bash[20824]: audit 2024-02-13T03:27:28.215551+0000 mon.a (mon.0) 980 : audit [INF] from='client.25223 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi161-62096-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-13T03:27:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:29 smithi183 bash[20824]: audit 2024-02-13T03:27:28.215610+0000 mon.a (mon.0) 981 : audit [INF] from='client.36692 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:29 smithi183 bash[20824]: audit 2024-02-13T03:27:28.215671+0000 mon.a (mon.0) 982 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-9"}]': finished 2024-02-13T03:27:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:29 smithi183 bash[20824]: cluster 2024-02-13T03:27:28.219958+0000 mon.a (mon.0) 983 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-13T03:27:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:29 smithi183 bash[20824]: audit 2024-02-13T03:27:28.228665+0000 mon.a (mon.0) 984 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-9", "mode": "writeback"}]: dispatch 2024-02-13T03:27:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:29 smithi183 bash[20824]: audit 2024-02-13T03:27:28.229605+0000 mon.b (mon.1) 94 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-9", "mode": "writeback"}]: dispatch 2024-02-13T03:27:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:29 smithi183 bash[20824]: audit 2024-02-13T03:27:28.229834+0000 mon.c (mon.2) 190 : audit [INF] from='client.? v1:172.21.15.161:0/1996406190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi161-61719-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:29 smithi183 bash[20824]: audit 2024-02-13T03:27:28.232887+0000 mon.a (mon.0) 985 : audit [INF] from='client.38024 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi161-61719-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:29 smithi183 bash[20824]: cluster 2024-02-13T03:27:28.266921+0000 mgr.y (mgr.24473) 254 : cluster [DBG] pgmap v214: 460 pgs: 20 creating+peering, 84 unknown, 356 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 op/s 2024-02-13T03:27:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:29 smithi183 bash[20824]: audit 2024-02-13T03:27:28.486439+0000 mon.c (mon.2) 191 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[27879]: cluster 2024-02-13T03:27:29.272752+0000 mon.a (mon.0) 986 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[27879]: audit 2024-02-13T03:27:29.287642+0000 mon.a (mon.0) 987 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-9", "mode": "writeback"}]': finished 2024-02-13T03:27:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[27879]: audit 2024-02-13T03:27:29.287756+0000 mon.a (mon.0) 988 : audit [INF] from='client.38024 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi161-61719-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[27879]: cluster 2024-02-13T03:27:29.287951+0000 mon.a (mon.0) 989 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-13T03:27:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[27879]: audit 2024-02-13T03:27:29.329119+0000 mon.c (mon.2) 192 : audit [INF] from='client.? v1:172.21.15.161:0/1342681366' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi161-61726-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[27879]: audit 2024-02-13T03:27:29.360021+0000 mon.a (mon.0) 990 : audit [INF] from='client.38570 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi161-61726-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[27879]: audit 2024-02-13T03:27:29.431529+0000 mon.a (mon.0) 991 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[27879]: audit 2024-02-13T03:27:29.431586+0000 mon.b (mon.1) 95 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[27879]: audit 2024-02-13T03:27:29.488270+0000 mon.c (mon.2) 193 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:30.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[27879]: cluster 2024-02-13T03:27:30.267482+0000 mgr.y (mgr.24473) 255 : cluster [DBG] pgmap v216: 492 pgs: 20 creating+peering, 148 unknown, 324 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:30.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[19769]: cluster 2024-02-13T03:27:29.272752+0000 mon.a (mon.0) 986 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:30.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[19769]: audit 2024-02-13T03:27:29.287642+0000 mon.a (mon.0) 987 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-9", "mode": "writeback"}]': finished 2024-02-13T03:27:30.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[19769]: audit 2024-02-13T03:27:29.287756+0000 mon.a (mon.0) 988 : audit [INF] from='client.38024 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi161-61719-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:30.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[19769]: cluster 2024-02-13T03:27:29.287951+0000 mon.a (mon.0) 989 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-13T03:27:30.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[19769]: audit 2024-02-13T03:27:29.329119+0000 mon.c (mon.2) 192 : audit [INF] from='client.? v1:172.21.15.161:0/1342681366' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi161-61726-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:30.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[19769]: audit 2024-02-13T03:27:29.360021+0000 mon.a (mon.0) 990 : audit [INF] from='client.38570 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi161-61726-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:30.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[19769]: audit 2024-02-13T03:27:29.431529+0000 mon.a (mon.0) 991 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:30.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[19769]: audit 2024-02-13T03:27:29.431586+0000 mon.b (mon.1) 95 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:30.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[19769]: audit 2024-02-13T03:27:29.488270+0000 mon.c (mon.2) 193 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:30.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:30 smithi161 bash[19769]: cluster 2024-02-13T03:27:30.267482+0000 mgr.y (mgr.24473) 255 : cluster [DBG] pgmap v216: 492 pgs: 20 creating+peering, 148 unknown, 324 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:30.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:30 smithi183 bash[20824]: cluster 2024-02-13T03:27:29.272752+0000 mon.a (mon.0) 986 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:30 smithi183 bash[20824]: audit 2024-02-13T03:27:29.287642+0000 mon.a (mon.0) 987 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-9", "mode": "writeback"}]': finished 2024-02-13T03:27:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:30 smithi183 bash[20824]: audit 2024-02-13T03:27:29.287756+0000 mon.a (mon.0) 988 : audit [INF] from='client.38024 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi161-61719-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:30 smithi183 bash[20824]: cluster 2024-02-13T03:27:29.287951+0000 mon.a (mon.0) 989 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-13T03:27:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:30 smithi183 bash[20824]: audit 2024-02-13T03:27:29.329119+0000 mon.c (mon.2) 192 : audit [INF] from='client.? v1:172.21.15.161:0/1342681366' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi161-61726-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:30 smithi183 bash[20824]: audit 2024-02-13T03:27:29.360021+0000 mon.a (mon.0) 990 : audit [INF] from='client.38570 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi161-61726-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:30 smithi183 bash[20824]: audit 2024-02-13T03:27:29.431529+0000 mon.a (mon.0) 991 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:30 smithi183 bash[20824]: audit 2024-02-13T03:27:29.431586+0000 mon.b (mon.1) 95 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:30 smithi183 bash[20824]: audit 2024-02-13T03:27:29.488270+0000 mon.c (mon.2) 193 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:30 smithi183 bash[20824]: cluster 2024-02-13T03:27:30.267482+0000 mgr.y (mgr.24473) 255 : cluster [DBG] pgmap v216: 492 pgs: 20 creating+peering, 148 unknown, 324 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.290331+0000 mon.a (mon.0) 992 : audit [INF] from='client.38570 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi161-61726-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.290498+0000 mon.a (mon.0) 993 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:27:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: cluster 2024-02-13T03:27:30.290593+0000 mon.a (mon.0) 994 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-13T03:27:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.303020+0000 mon.b (mon.1) 96 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9"}]: dispatch 2024-02-13T03:27:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.309672+0000 mon.a (mon.0) 995 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9"}]: dispatch 2024-02-13T03:27:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.491007+0000 mon.c (mon.2) 194 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.872865+0000 mon.c (mon.2) 195 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-13T03:27:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.873302+0000 mgr.y (mgr.24473) 256 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-13T03:27:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.875615+0000 mon.c (mon.2) 196 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-13T03:27:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.876159+0000 mgr.y (mgr.24473) 257 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-13T03:27:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.879787+0000 mon.c (mon.2) 197 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-13T03:27:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.881724+0000 mgr.y (mgr.24473) 258 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-13T03:27:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.887002+0000 mon.c (mon.2) 198 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-13T03:27:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.887756+0000 mgr.y (mgr.24473) 259 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-13T03:27:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.891642+0000 mon.c (mon.2) 199 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-13T03:27:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.897675+0000 mgr.y (mgr.24473) 260 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.901266+0000 mon.c (mon.2) 200 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.901605+0000 mgr.y (mgr.24473) 261 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.902758+0000 mon.c (mon.2) 201 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.906928+0000 mgr.y (mgr.24473) 262 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.909710+0000 mon.c (mon.2) 202 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.912675+0000 mgr.y (mgr.24473) 263 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.913194+0000 mon.c (mon.2) 203 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.913428+0000 mgr.y (mgr.24473) 264 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.913987+0000 mon.c (mon.2) 204 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:30.915162+0000 mgr.y (mgr.24473) 265 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: cluster 2024-02-13T03:27:31.287662+0000 mon.a (mon.0) 996 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:31.295953+0000 mon.a (mon.0) 997 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9"}]': finished 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: cluster 2024-02-13T03:27:31.296112+0000 mon.a (mon.0) 998 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-13T03:27:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:31.320943+0000 mon.c (mon.2) 205 : audit [INF] from='client.? v1:172.21.15.161:0/3373224231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi161-61719-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:31.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[19769]: audit 2024-02-13T03:27:31.323123+0000 mon.a (mon.0) 999 : audit [INF] from='client.40307 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi161-61719-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:31.430 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:27:31] "GET /metrics HTTP/1.1" 200 201936 "" "Prometheus/2.33.4" 2024-02-13T03:27:31.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.290331+0000 mon.a (mon.0) 992 : audit [INF] from='client.38570 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi161-61726-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:31.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.290498+0000 mon.a (mon.0) 993 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:27:31.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: cluster 2024-02-13T03:27:30.290593+0000 mon.a (mon.0) 994 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-13T03:27:31.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.303020+0000 mon.b (mon.1) 96 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9"}]: dispatch 2024-02-13T03:27:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.309672+0000 mon.a (mon.0) 995 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9"}]: dispatch 2024-02-13T03:27:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.491007+0000 mon.c (mon.2) 194 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.872865+0000 mon.c (mon.2) 195 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-13T03:27:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.873302+0000 mgr.y (mgr.24473) 256 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-13T03:27:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.875615+0000 mon.c (mon.2) 196 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-13T03:27:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.876159+0000 mgr.y (mgr.24473) 257 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-13T03:27:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.879787+0000 mon.c (mon.2) 197 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-13T03:27:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.881724+0000 mgr.y (mgr.24473) 258 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-13T03:27:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.887002+0000 mon.c (mon.2) 198 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-13T03:27:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.887756+0000 mgr.y (mgr.24473) 259 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-13T03:27:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.891642+0000 mon.c (mon.2) 199 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-13T03:27:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.897675+0000 mgr.y (mgr.24473) 260 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-13T03:27:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.901266+0000 mon.c (mon.2) 200 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-13T03:27:31.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.901605+0000 mgr.y (mgr.24473) 261 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-13T03:27:31.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.902758+0000 mon.c (mon.2) 201 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-13T03:27:31.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.906928+0000 mgr.y (mgr.24473) 262 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-13T03:27:31.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.909710+0000 mon.c (mon.2) 202 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-13T03:27:31.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.912675+0000 mgr.y (mgr.24473) 263 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-13T03:27:31.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.913194+0000 mon.c (mon.2) 203 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-13T03:27:31.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.913428+0000 mgr.y (mgr.24473) 264 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-13T03:27:31.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.913987+0000 mon.c (mon.2) 204 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-13T03:27:31.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:30.915162+0000 mgr.y (mgr.24473) 265 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-13T03:27:31.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: cluster 2024-02-13T03:27:31.287662+0000 mon.a (mon.0) 996 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:27:31.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:31.295953+0000 mon.a (mon.0) 997 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9"}]': finished 2024-02-13T03:27:31.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: cluster 2024-02-13T03:27:31.296112+0000 mon.a (mon.0) 998 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-13T03:27:31.432 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:31.320943+0000 mon.c (mon.2) 205 : audit [INF] from='client.? v1:172.21.15.161:0/3373224231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi161-61719-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:31.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:31 smithi161 bash[27879]: audit 2024-02-13T03:27:31.323123+0000 mon.a (mon.0) 999 : audit [INF] from='client.40307 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi161-61719-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.290331+0000 mon.a (mon.0) 992 : audit [INF] from='client.38570 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi161-61726-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.290498+0000 mon.a (mon.0) 993 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:27:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: cluster 2024-02-13T03:27:30.290593+0000 mon.a (mon.0) 994 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-13T03:27:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.303020+0000 mon.b (mon.1) 96 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9"}]: dispatch 2024-02-13T03:27:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.309672+0000 mon.a (mon.0) 995 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9"}]: dispatch 2024-02-13T03:27:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.491007+0000 mon.c (mon.2) 194 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.872865+0000 mon.c (mon.2) 195 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-13T03:27:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.873302+0000 mgr.y (mgr.24473) 256 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.875615+0000 mon.c (mon.2) 196 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.876159+0000 mgr.y (mgr.24473) 257 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.879787+0000 mon.c (mon.2) 197 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.881724+0000 mgr.y (mgr.24473) 258 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.887002+0000 mon.c (mon.2) 198 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.887756+0000 mgr.y (mgr.24473) 259 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.891642+0000 mon.c (mon.2) 199 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.897675+0000 mgr.y (mgr.24473) 260 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.901266+0000 mon.c (mon.2) 200 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.901605+0000 mgr.y (mgr.24473) 261 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.902758+0000 mon.c (mon.2) 201 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.906928+0000 mgr.y (mgr.24473) 262 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.909710+0000 mon.c (mon.2) 202 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-13T03:27:31.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.912675+0000 mgr.y (mgr.24473) 263 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-13T03:27:31.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.913194+0000 mon.c (mon.2) 203 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-13T03:27:31.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.913428+0000 mgr.y (mgr.24473) 264 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-13T03:27:31.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.913987+0000 mon.c (mon.2) 204 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-13T03:27:31.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:30.915162+0000 mgr.y (mgr.24473) 265 : audit [DBG] from='mon.? v1:172.21.15.161:0/1169400930' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-13T03:27:31.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: cluster 2024-02-13T03:27:31.287662+0000 mon.a (mon.0) 996 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:27:31.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:31.295953+0000 mon.a (mon.0) 997 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-9"}]': finished 2024-02-13T03:27:31.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: cluster 2024-02-13T03:27:31.296112+0000 mon.a (mon.0) 998 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-13T03:27:31.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:31.320943+0000 mon.c (mon.2) 205 : audit [INF] from='client.? v1:172.21.15.161:0/3373224231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi161-61719-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:31.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:31 smithi183 bash[20824]: audit 2024-02-13T03:27:31.323123+0000 mon.a (mon.0) 999 : audit [INF] from='client.40307 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi161-61719-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:32.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:32 smithi161 bash[19769]: audit 2024-02-13T03:27:31.496198+0000 mon.c (mon.2) 206 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:32.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:32 smithi161 bash[27879]: audit 2024-02-13T03:27:31.496198+0000 mon.c (mon.2) 206 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:32.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:32 smithi183 bash[20824]: audit 2024-02-13T03:27:31.496198+0000 mon.c (mon.2) 206 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:34.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: cluster 2024-02-13T03:27:31.425375+0000 osd.7 (osd.7) 7 : cluster [DBG] 18.2 deep-scrub starts 2024-02-13T03:27:34.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: cluster 2024-02-13T03:27:31.434675+0000 osd.7 (osd.7) 8 : cluster [DBG] 18.2 deep-scrub ok 2024-02-13T03:27:34.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: cluster 2024-02-13T03:27:32.268174+0000 mgr.y (mgr.24473) 266 : cluster [DBG] pgmap v219: 396 pgs: 32 unknown, 364 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-02-13T03:27:34.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: cluster 2024-02-13T03:27:32.336404+0000 osd.1 (osd.1) 7 : cluster [DBG] 18.3 deep-scrub starts 2024-02-13T03:27:34.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: cluster 2024-02-13T03:27:32.339995+0000 osd.1 (osd.1) 8 : cluster [DBG] 18.3 deep-scrub ok 2024-02-13T03:27:34.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: audit 2024-02-13T03:27:32.408322+0000 mon.a (mon.0) 1000 : audit [INF] from='client.40307 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi161-61719-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:34.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: cluster 2024-02-13T03:27:32.408556+0000 mon.a (mon.0) 1001 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-13T03:27:34.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: audit 2024-02-13T03:27:32.422718+0000 mon.b (mon.1) 97 : audit [INF] from='client.? v1:172.21.15.161:0/2801967034' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:34.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: audit 2024-02-13T03:27:32.423061+0000 mon.a (mon.0) 1002 : audit [INF] from='client.41644 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:34.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: audit 2024-02-13T03:27:32.425921+0000 mon.a (mon.0) 1003 : audit [INF] from='client.41620 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi161-61726-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:34.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: audit 2024-02-13T03:27:32.427690+0000 mon.b (mon.1) 98 : audit [INF] from='client.? v1:172.21.15.161:0/242367329' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi161-61726-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:34.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: cluster 2024-02-13T03:27:32.451533+0000 mon.a (mon.0) 1004 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:27:34.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: audit 2024-02-13T03:27:32.500822+0000 mon.c (mon.2) 207 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:34.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: audit 2024-02-13T03:27:33.192665+0000 mon.a (mon.0) 1005 : audit [INF] from='client.41644 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:34.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: audit 2024-02-13T03:27:33.192867+0000 mon.a (mon.0) 1006 : audit [INF] from='client.41620 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi161-61726-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:34.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: cluster 2024-02-13T03:27:33.193090+0000 mon.a (mon.0) 1007 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-13T03:27:34.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[19769]: audit 2024-02-13T03:27:33.503482+0000 mon.c (mon.2) 208 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:34.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: cluster 2024-02-13T03:27:31.425375+0000 osd.7 (osd.7) 7 : cluster [DBG] 18.2 deep-scrub starts 2024-02-13T03:27:34.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: cluster 2024-02-13T03:27:31.434675+0000 osd.7 (osd.7) 8 : cluster [DBG] 18.2 deep-scrub ok 2024-02-13T03:27:34.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: cluster 2024-02-13T03:27:32.268174+0000 mgr.y (mgr.24473) 266 : cluster [DBG] pgmap v219: 396 pgs: 32 unknown, 364 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-02-13T03:27:34.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: cluster 2024-02-13T03:27:32.336404+0000 osd.1 (osd.1) 7 : cluster [DBG] 18.3 deep-scrub starts 2024-02-13T03:27:34.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: cluster 2024-02-13T03:27:32.339995+0000 osd.1 (osd.1) 8 : cluster [DBG] 18.3 deep-scrub ok 2024-02-13T03:27:34.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: audit 2024-02-13T03:27:32.408322+0000 mon.a (mon.0) 1000 : audit [INF] from='client.40307 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi161-61719-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:34.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: cluster 2024-02-13T03:27:32.408556+0000 mon.a (mon.0) 1001 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-13T03:27:34.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: audit 2024-02-13T03:27:32.422718+0000 mon.b (mon.1) 97 : audit [INF] from='client.? v1:172.21.15.161:0/2801967034' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:34.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: audit 2024-02-13T03:27:32.423061+0000 mon.a (mon.0) 1002 : audit [INF] from='client.41644 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:34.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: audit 2024-02-13T03:27:32.425921+0000 mon.a (mon.0) 1003 : audit [INF] from='client.41620 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi161-61726-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:34.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: audit 2024-02-13T03:27:32.427690+0000 mon.b (mon.1) 98 : audit [INF] from='client.? v1:172.21.15.161:0/242367329' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi161-61726-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:34.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: cluster 2024-02-13T03:27:32.451533+0000 mon.a (mon.0) 1004 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:27:34.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: audit 2024-02-13T03:27:32.500822+0000 mon.c (mon.2) 207 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:34.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: audit 2024-02-13T03:27:33.192665+0000 mon.a (mon.0) 1005 : audit [INF] from='client.41644 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:34.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: audit 2024-02-13T03:27:33.192867+0000 mon.a (mon.0) 1006 : audit [INF] from='client.41620 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi161-61726-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:34.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: cluster 2024-02-13T03:27:33.193090+0000 mon.a (mon.0) 1007 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-13T03:27:34.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:34 smithi161 bash[27879]: audit 2024-02-13T03:27:33.503482+0000 mon.c (mon.2) 208 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: cluster 2024-02-13T03:27:31.425375+0000 osd.7 (osd.7) 7 : cluster [DBG] 18.2 deep-scrub starts 2024-02-13T03:27:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: cluster 2024-02-13T03:27:31.434675+0000 osd.7 (osd.7) 8 : cluster [DBG] 18.2 deep-scrub ok 2024-02-13T03:27:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: cluster 2024-02-13T03:27:32.268174+0000 mgr.y (mgr.24473) 266 : cluster [DBG] pgmap v219: 396 pgs: 32 unknown, 364 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-02-13T03:27:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: cluster 2024-02-13T03:27:32.336404+0000 osd.1 (osd.1) 7 : cluster [DBG] 18.3 deep-scrub starts 2024-02-13T03:27:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: cluster 2024-02-13T03:27:32.339995+0000 osd.1 (osd.1) 8 : cluster [DBG] 18.3 deep-scrub ok 2024-02-13T03:27:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: audit 2024-02-13T03:27:32.408322+0000 mon.a (mon.0) 1000 : audit [INF] from='client.40307 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi161-61719-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: cluster 2024-02-13T03:27:32.408556+0000 mon.a (mon.0) 1001 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-13T03:27:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: audit 2024-02-13T03:27:32.422718+0000 mon.b (mon.1) 97 : audit [INF] from='client.? v1:172.21.15.161:0/2801967034' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: audit 2024-02-13T03:27:32.423061+0000 mon.a (mon.0) 1002 : audit [INF] from='client.41644 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:34.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: audit 2024-02-13T03:27:32.425921+0000 mon.a (mon.0) 1003 : audit [INF] from='client.41620 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi161-61726-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:34.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: audit 2024-02-13T03:27:32.427690+0000 mon.b (mon.1) 98 : audit [INF] from='client.? v1:172.21.15.161:0/242367329' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi161-61726-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:34.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: cluster 2024-02-13T03:27:32.451533+0000 mon.a (mon.0) 1004 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:27:34.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: audit 2024-02-13T03:27:32.500822+0000 mon.c (mon.2) 207 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:34.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: audit 2024-02-13T03:27:33.192665+0000 mon.a (mon.0) 1005 : audit [INF] from='client.41644 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61925-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:34.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: audit 2024-02-13T03:27:33.192867+0000 mon.a (mon.0) 1006 : audit [INF] from='client.41620 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi161-61726-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:34.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: cluster 2024-02-13T03:27:33.193090+0000 mon.a (mon.0) 1007 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-13T03:27:34.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:34 smithi183 bash[20824]: audit 2024-02-13T03:27:33.503482+0000 mon.c (mon.2) 208 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: cluster 2024-02-13T03:27:33.152125+0000 osd.0 (osd.0) 9 : cluster [DBG] 18.0 deep-scrub starts 2024-02-13T03:27:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: cluster 2024-02-13T03:27:33.155369+0000 osd.0 (osd.0) 10 : cluster [DBG] 18.0 deep-scrub ok 2024-02-13T03:27:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: cluster 2024-02-13T03:27:33.394184+0000 osd.7 (osd.7) 9 : cluster [DBG] 18.6 deep-scrub starts 2024-02-13T03:27:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: cluster 2024-02-13T03:27:33.398030+0000 osd.7 (osd.7) 10 : cluster [DBG] 18.6 deep-scrub ok 2024-02-13T03:27:35.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: cluster 2024-02-13T03:27:33.578235+0000 osd.3 (osd.3) 7 : cluster [DBG] 18.1 deep-scrub starts 2024-02-13T03:27:35.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: cluster 2024-02-13T03:27:33.631894+0000 osd.3 (osd.3) 8 : cluster [DBG] 18.1 deep-scrub ok 2024-02-13T03:27:35.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: cluster 2024-02-13T03:27:34.268772+0000 mgr.y (mgr.24473) 267 : cluster [DBG] pgmap v222: 396 pgs: 64 unknown, 332 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-02-13T03:27:35.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: audit 2024-02-13T03:27:34.286274+0000 mon.b (mon.1) 99 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:35.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: audit 2024-02-13T03:27:34.289643+0000 mon.b (mon.1) 100 : audit [INF] from='client.? v1:172.21.15.161:0/2801967034' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache", "force_nonempty":""}]: dispatch 2024-02-13T03:27:35.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: cluster 2024-02-13T03:27:34.394046+0000 osd.1 (osd.1) 9 : cluster [DBG] 18.5 deep-scrub starts 2024-02-13T03:27:35.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: cluster 2024-02-13T03:27:34.400142+0000 osd.1 (osd.1) 10 : cluster [DBG] 18.5 deep-scrub ok 2024-02-13T03:27:35.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: audit 2024-02-13T03:27:34.504614+0000 mon.c (mon.2) 209 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:35.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: cluster 2024-02-13T03:27:34.798475+0000 mon.a (mon.0) 1008 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-13T03:27:35.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: audit 2024-02-13T03:27:34.805117+0000 mon.a (mon.0) 1009 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:35.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: audit 2024-02-13T03:27:34.806920+0000 mon.a (mon.0) 1010 : audit [INF] from='client.41644 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache", "force_nonempty":""}]: dispatch 2024-02-13T03:27:35.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[27879]: audit 2024-02-13T03:27:34.811988+0000 mon.a (mon.0) 1011 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:35.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: cluster 2024-02-13T03:27:33.152125+0000 osd.0 (osd.0) 9 : cluster [DBG] 18.0 deep-scrub starts 2024-02-13T03:27:35.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: cluster 2024-02-13T03:27:33.155369+0000 osd.0 (osd.0) 10 : cluster [DBG] 18.0 deep-scrub ok 2024-02-13T03:27:35.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: cluster 2024-02-13T03:27:33.394184+0000 osd.7 (osd.7) 9 : cluster [DBG] 18.6 deep-scrub starts 2024-02-13T03:27:35.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: cluster 2024-02-13T03:27:33.398030+0000 osd.7 (osd.7) 10 : cluster [DBG] 18.6 deep-scrub ok 2024-02-13T03:27:35.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: cluster 2024-02-13T03:27:33.578235+0000 osd.3 (osd.3) 7 : cluster [DBG] 18.1 deep-scrub starts 2024-02-13T03:27:35.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: cluster 2024-02-13T03:27:33.631894+0000 osd.3 (osd.3) 8 : cluster [DBG] 18.1 deep-scrub ok 2024-02-13T03:27:35.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: cluster 2024-02-13T03:27:34.268772+0000 mgr.y (mgr.24473) 267 : cluster [DBG] pgmap v222: 396 pgs: 64 unknown, 332 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-02-13T03:27:35.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: audit 2024-02-13T03:27:34.286274+0000 mon.b (mon.1) 99 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:35.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: audit 2024-02-13T03:27:34.289643+0000 mon.b (mon.1) 100 : audit [INF] from='client.? v1:172.21.15.161:0/2801967034' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache", "force_nonempty":""}]: dispatch 2024-02-13T03:27:35.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: cluster 2024-02-13T03:27:34.394046+0000 osd.1 (osd.1) 9 : cluster [DBG] 18.5 deep-scrub starts 2024-02-13T03:27:35.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: cluster 2024-02-13T03:27:34.400142+0000 osd.1 (osd.1) 10 : cluster [DBG] 18.5 deep-scrub ok 2024-02-13T03:27:35.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: audit 2024-02-13T03:27:34.504614+0000 mon.c (mon.2) 209 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:35.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: cluster 2024-02-13T03:27:34.798475+0000 mon.a (mon.0) 1008 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-13T03:27:35.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: audit 2024-02-13T03:27:34.805117+0000 mon.a (mon.0) 1009 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:35.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: audit 2024-02-13T03:27:34.806920+0000 mon.a (mon.0) 1010 : audit [INF] from='client.41644 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache", "force_nonempty":""}]: dispatch 2024-02-13T03:27:35.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:35 smithi161 bash[19769]: audit 2024-02-13T03:27:34.811988+0000 mon.a (mon.0) 1011 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: cluster 2024-02-13T03:27:33.152125+0000 osd.0 (osd.0) 9 : cluster [DBG] 18.0 deep-scrub starts 2024-02-13T03:27:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: cluster 2024-02-13T03:27:33.155369+0000 osd.0 (osd.0) 10 : cluster [DBG] 18.0 deep-scrub ok 2024-02-13T03:27:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: cluster 2024-02-13T03:27:33.394184+0000 osd.7 (osd.7) 9 : cluster [DBG] 18.6 deep-scrub starts 2024-02-13T03:27:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: cluster 2024-02-13T03:27:33.398030+0000 osd.7 (osd.7) 10 : cluster [DBG] 18.6 deep-scrub ok 2024-02-13T03:27:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: cluster 2024-02-13T03:27:33.578235+0000 osd.3 (osd.3) 7 : cluster [DBG] 18.1 deep-scrub starts 2024-02-13T03:27:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: cluster 2024-02-13T03:27:33.631894+0000 osd.3 (osd.3) 8 : cluster [DBG] 18.1 deep-scrub ok 2024-02-13T03:27:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: cluster 2024-02-13T03:27:34.268772+0000 mgr.y (mgr.24473) 267 : cluster [DBG] pgmap v222: 396 pgs: 64 unknown, 332 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-02-13T03:27:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: audit 2024-02-13T03:27:34.286274+0000 mon.b (mon.1) 99 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:35.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: audit 2024-02-13T03:27:34.289643+0000 mon.b (mon.1) 100 : audit [INF] from='client.? v1:172.21.15.161:0/2801967034' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache", "force_nonempty":""}]: dispatch 2024-02-13T03:27:35.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: cluster 2024-02-13T03:27:34.394046+0000 osd.1 (osd.1) 9 : cluster [DBG] 18.5 deep-scrub starts 2024-02-13T03:27:35.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: cluster 2024-02-13T03:27:34.400142+0000 osd.1 (osd.1) 10 : cluster [DBG] 18.5 deep-scrub ok 2024-02-13T03:27:35.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: audit 2024-02-13T03:27:34.504614+0000 mon.c (mon.2) 209 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:35.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: cluster 2024-02-13T03:27:34.798475+0000 mon.a (mon.0) 1008 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-13T03:27:35.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: audit 2024-02-13T03:27:34.805117+0000 mon.a (mon.0) 1009 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:35.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: audit 2024-02-13T03:27:34.806920+0000 mon.a (mon.0) 1010 : audit [INF] from='client.41644 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache", "force_nonempty":""}]: dispatch 2024-02-13T03:27:35.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[20824]: audit 2024-02-13T03:27:34.811988+0000 mon.a (mon.0) 1011 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:27:36.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:27:35 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:27:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: cluster 2024-02-13T03:27:34.608797+0000 osd.3 (osd.3) 9 : cluster [DBG] 18.9 deep-scrub starts 2024-02-13T03:27:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: cluster 2024-02-13T03:27:34.612680+0000 osd.3 (osd.3) 10 : cluster [DBG] 18.9 deep-scrub ok 2024-02-13T03:27:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: cluster 2024-02-13T03:27:35.126466+0000 osd.2 (osd.2) 5 : cluster [DBG] 18.7 deep-scrub starts 2024-02-13T03:27:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: cluster 2024-02-13T03:27:35.160625+0000 osd.0 (osd.0) 11 : cluster [DBG] 18.4 deep-scrub starts 2024-02-13T03:27:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: cluster 2024-02-13T03:27:35.165944+0000 osd.0 (osd.0) 12 : cluster [DBG] 18.4 deep-scrub ok 2024-02-13T03:27:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: cluster 2024-02-13T03:27:35.214560+0000 osd.2 (osd.2) 6 : cluster [DBG] 18.7 deep-scrub ok 2024-02-13T03:27:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: audit 2024-02-13T03:27:35.276038+0000 mon.a (mon.0) 1012 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: audit 2024-02-13T03:27:35.276127+0000 mon.a (mon.0) 1013 : audit [INF] from='client.41644 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache", "force_nonempty":""}]': finished 2024-02-13T03:27:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: cluster 2024-02-13T03:27:35.276232+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-13T03:27:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: audit 2024-02-13T03:27:35.291301+0000 mon.b (mon.1) 101 : audit [INF] from='client.? v1:172.21.15.161:0/2801967034' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61925-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: audit 2024-02-13T03:27:35.291471+0000 mon.b (mon.1) 102 : audit [INF] from='client.? v1:172.21.15.161:0/2983378725' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi161-61719-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: audit 2024-02-13T03:27:35.299755+0000 mon.a (mon.0) 1015 : audit [INF] from='client.41644 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61925-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:36.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: audit 2024-02-13T03:27:35.302756+0000 mon.a (mon.0) 1016 : audit [INF] from='client.43357 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi161-61719-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:36.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[19769]: audit 2024-02-13T03:27:35.507366+0000 mon.c (mon.2) 210 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:36.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: cluster 2024-02-13T03:27:34.608797+0000 osd.3 (osd.3) 9 : cluster [DBG] 18.9 deep-scrub starts 2024-02-13T03:27:36.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: cluster 2024-02-13T03:27:34.612680+0000 osd.3 (osd.3) 10 : cluster [DBG] 18.9 deep-scrub ok 2024-02-13T03:27:36.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: cluster 2024-02-13T03:27:35.126466+0000 osd.2 (osd.2) 5 : cluster [DBG] 18.7 deep-scrub starts 2024-02-13T03:27:36.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: cluster 2024-02-13T03:27:35.160625+0000 osd.0 (osd.0) 11 : cluster [DBG] 18.4 deep-scrub starts 2024-02-13T03:27:36.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: cluster 2024-02-13T03:27:35.165944+0000 osd.0 (osd.0) 12 : cluster [DBG] 18.4 deep-scrub ok 2024-02-13T03:27:36.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: cluster 2024-02-13T03:27:35.214560+0000 osd.2 (osd.2) 6 : cluster [DBG] 18.7 deep-scrub ok 2024-02-13T03:27:36.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: audit 2024-02-13T03:27:35.276038+0000 mon.a (mon.0) 1012 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:36.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: audit 2024-02-13T03:27:35.276127+0000 mon.a (mon.0) 1013 : audit [INF] from='client.41644 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache", "force_nonempty":""}]': finished 2024-02-13T03:27:36.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: cluster 2024-02-13T03:27:35.276232+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-13T03:27:36.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: audit 2024-02-13T03:27:35.291301+0000 mon.b (mon.1) 101 : audit [INF] from='client.? v1:172.21.15.161:0/2801967034' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61925-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:36.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: audit 2024-02-13T03:27:35.291471+0000 mon.b (mon.1) 102 : audit [INF] from='client.? v1:172.21.15.161:0/2983378725' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi161-61719-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:36.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: audit 2024-02-13T03:27:35.299755+0000 mon.a (mon.0) 1015 : audit [INF] from='client.41644 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61925-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:36.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: audit 2024-02-13T03:27:35.302756+0000 mon.a (mon.0) 1016 : audit [INF] from='client.43357 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi161-61719-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:36.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:36 smithi161 bash[27879]: audit 2024-02-13T03:27:35.507366+0000 mon.c (mon.2) 210 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: cluster 2024-02-13T03:27:34.608797+0000 osd.3 (osd.3) 9 : cluster [DBG] 18.9 deep-scrub starts 2024-02-13T03:27:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: cluster 2024-02-13T03:27:34.612680+0000 osd.3 (osd.3) 10 : cluster [DBG] 18.9 deep-scrub ok 2024-02-13T03:27:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: cluster 2024-02-13T03:27:35.126466+0000 osd.2 (osd.2) 5 : cluster [DBG] 18.7 deep-scrub starts 2024-02-13T03:27:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: cluster 2024-02-13T03:27:35.160625+0000 osd.0 (osd.0) 11 : cluster [DBG] 18.4 deep-scrub starts 2024-02-13T03:27:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: cluster 2024-02-13T03:27:35.165944+0000 osd.0 (osd.0) 12 : cluster [DBG] 18.4 deep-scrub ok 2024-02-13T03:27:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: cluster 2024-02-13T03:27:35.214560+0000 osd.2 (osd.2) 6 : cluster [DBG] 18.7 deep-scrub ok 2024-02-13T03:27:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: audit 2024-02-13T03:27:35.276038+0000 mon.a (mon.0) 1012 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: audit 2024-02-13T03:27:35.276127+0000 mon.a (mon.0) 1013 : audit [INF] from='client.41644 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache", "force_nonempty":""}]': finished 2024-02-13T03:27:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: cluster 2024-02-13T03:27:35.276232+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-13T03:27:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: audit 2024-02-13T03:27:35.291301+0000 mon.b (mon.1) 101 : audit [INF] from='client.? v1:172.21.15.161:0/2801967034' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61925-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:36.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: audit 2024-02-13T03:27:35.291471+0000 mon.b (mon.1) 102 : audit [INF] from='client.? v1:172.21.15.161:0/2983378725' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi161-61719-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:36.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: audit 2024-02-13T03:27:35.299755+0000 mon.a (mon.0) 1015 : audit [INF] from='client.41644 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61925-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:36.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: audit 2024-02-13T03:27:35.302756+0000 mon.a (mon.0) 1016 : audit [INF] from='client.43357 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi161-61719-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:36.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:36 smithi183 bash[20824]: audit 2024-02-13T03:27:35.507366+0000 mon.c (mon.2) 210 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:37.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:35.833305+0000 mgr.y (mgr.24473) 268 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:37.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: cluster 2024-02-13T03:27:36.270595+0000 mgr.y (mgr.24473) 269 : cluster [DBG] pgmap v225: 460 pgs: 32 creating+peering, 428 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 3 op/s 2024-02-13T03:27:37.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:36.279481+0000 mon.a (mon.0) 1017 : audit [INF] from='client.41644 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61925-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:37.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:36.279561+0000 mon.a (mon.0) 1018 : audit [INF] from='client.43357 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi161-61719-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:37.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: cluster 2024-02-13T03:27:36.279774+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-13T03:27:37.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:36.303282+0000 mon.b (mon.1) 103 : audit [INF] from='client.? v1:172.21.15.161:0/2297664183' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi161-61726-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:37.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:36.316420+0000 mon.a (mon.0) 1020 : audit [INF] from='client.45094 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi161-61726-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:37.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:36.317947+0000 mon.b (mon.1) 104 : audit [INF] from='client.? v1:172.21.15.161:0/2801967034' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache"}]: dispatch 2024-02-13T03:27:37.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:36.323522+0000 mon.a (mon.0) 1021 : audit [INF] from='client.41644 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache"}]: dispatch 2024-02-13T03:27:37.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:36.358360+0000 mon.a (mon.0) 1022 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:37.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:36.361245+0000 mon.b (mon.1) 105 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:37.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:36.508648+0000 mon.c (mon.2) 211 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:37.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:37.282485+0000 mon.a (mon.0) 1023 : audit [INF] from='client.45094 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi161-61726-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:37.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:37.282604+0000 mon.a (mon.0) 1024 : audit [INF] from='client.41644 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache"}]': finished 2024-02-13T03:27:37.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:37.282719+0000 mon.a (mon.0) 1025 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:37.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: cluster 2024-02-13T03:27:37.282779+0000 mon.a (mon.0) 1026 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-13T03:27:37.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:37.292841+0000 mon.b (mon.1) 106 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-11"}]: dispatch 2024-02-13T03:27:37.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[19769]: audit 2024-02-13T03:27:37.295062+0000 mon.a (mon.0) 1027 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-11"}]: dispatch 2024-02-13T03:27:37.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:35.833305+0000 mgr.y (mgr.24473) 268 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:37.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: cluster 2024-02-13T03:27:36.270595+0000 mgr.y (mgr.24473) 269 : cluster [DBG] pgmap v225: 460 pgs: 32 creating+peering, 428 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 3 op/s 2024-02-13T03:27:37.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:36.279481+0000 mon.a (mon.0) 1017 : audit [INF] from='client.41644 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61925-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:36.279561+0000 mon.a (mon.0) 1018 : audit [INF] from='client.43357 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi161-61719-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: cluster 2024-02-13T03:27:36.279774+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:36.303282+0000 mon.b (mon.1) 103 : audit [INF] from='client.? v1:172.21.15.161:0/2297664183' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi161-61726-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:36.316420+0000 mon.a (mon.0) 1020 : audit [INF] from='client.45094 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi161-61726-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:36.317947+0000 mon.b (mon.1) 104 : audit [INF] from='client.? v1:172.21.15.161:0/2801967034' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache"}]: dispatch 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:36.323522+0000 mon.a (mon.0) 1021 : audit [INF] from='client.41644 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache"}]: dispatch 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:36.358360+0000 mon.a (mon.0) 1022 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:36.361245+0000 mon.b (mon.1) 105 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:36.508648+0000 mon.c (mon.2) 211 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:37.282485+0000 mon.a (mon.0) 1023 : audit [INF] from='client.45094 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi161-61726-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:37.282604+0000 mon.a (mon.0) 1024 : audit [INF] from='client.41644 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache"}]': finished 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:37.282719+0000 mon.a (mon.0) 1025 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: cluster 2024-02-13T03:27:37.282779+0000 mon.a (mon.0) 1026 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-13T03:27:37.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:37.292841+0000 mon.b (mon.1) 106 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-11"}]: dispatch 2024-02-13T03:27:37.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:37 smithi161 bash[27879]: audit 2024-02-13T03:27:37.295062+0000 mon.a (mon.0) 1027 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-11"}]: dispatch 2024-02-13T03:27:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:35.833305+0000 mgr.y (mgr.24473) 268 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: cluster 2024-02-13T03:27:36.270595+0000 mgr.y (mgr.24473) 269 : cluster [DBG] pgmap v225: 460 pgs: 32 creating+peering, 428 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 3 op/s 2024-02-13T03:27:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:36.279481+0000 mon.a (mon.0) 1017 : audit [INF] from='client.41644 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61925-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:36.279561+0000 mon.a (mon.0) 1018 : audit [INF] from='client.43357 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi161-61719-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: cluster 2024-02-13T03:27:36.279774+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-13T03:27:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:36.303282+0000 mon.b (mon.1) 103 : audit [INF] from='client.? v1:172.21.15.161:0/2297664183' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi161-61726-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:36.316420+0000 mon.a (mon.0) 1020 : audit [INF] from='client.45094 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi161-61726-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:36.317947+0000 mon.b (mon.1) 104 : audit [INF] from='client.? v1:172.21.15.161:0/2801967034' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache"}]: dispatch 2024-02-13T03:27:37.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:36.323522+0000 mon.a (mon.0) 1021 : audit [INF] from='client.41644 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache"}]: dispatch 2024-02-13T03:27:37.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:36.358360+0000 mon.a (mon.0) 1022 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:37.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:36.361245+0000 mon.b (mon.1) 105 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:37.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:36.508648+0000 mon.c (mon.2) 211 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:37.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:37.282485+0000 mon.a (mon.0) 1023 : audit [INF] from='client.45094 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi161-61726-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:37.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:37.282604+0000 mon.a (mon.0) 1024 : audit [INF] from='client.41644 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61925-10", "tierpool":"test-rados-api-smithi161-61925-10-cache"}]': finished 2024-02-13T03:27:37.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:37.282719+0000 mon.a (mon.0) 1025 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:37.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: cluster 2024-02-13T03:27:37.282779+0000 mon.a (mon.0) 1026 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-13T03:27:37.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:37.292841+0000 mon.b (mon.1) 106 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-11"}]: dispatch 2024-02-13T03:27:37.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:37 smithi183 bash[20824]: audit 2024-02-13T03:27:37.295062+0000 mon.a (mon.0) 1027 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-11"}]: dispatch 2024-02-13T03:27:39.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[27879]: audit 2024-02-13T03:27:37.515274+0000 mon.c (mon.2) 212 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:39.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[27879]: audit 2024-02-13T03:27:38.195299+0000 mon.a (mon.0) 1028 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-11"}]': finished 2024-02-13T03:27:39.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[27879]: cluster 2024-02-13T03:27:38.195455+0000 mon.a (mon.0) 1029 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-13T03:27:39.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[27879]: audit 2024-02-13T03:27:38.203779+0000 mon.b (mon.1) 107 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-11", "mode": "writeback"}]: dispatch 2024-02-13T03:27:39.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[27879]: audit 2024-02-13T03:27:38.213079+0000 mon.c (mon.2) 213 : audit [INF] from='client.? v1:172.21.15.161:0/3439885083' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi161-61719-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:39.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[27879]: audit 2024-02-13T03:27:38.220250+0000 mon.a (mon.0) 1030 : audit [INF] from='client.46361 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi161-61719-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:39.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[27879]: audit 2024-02-13T03:27:38.228584+0000 mon.a (mon.0) 1031 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-11", "mode": "writeback"}]: dispatch 2024-02-13T03:27:39.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[27879]: cluster 2024-02-13T03:27:38.274155+0000 mgr.y (mgr.24473) 270 : cluster [DBG] pgmap v229: 428 pgs: 32 unknown, 396 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 4 op/s 2024-02-13T03:27:39.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[27879]: audit 2024-02-13T03:27:38.517956+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:39.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[19769]: audit 2024-02-13T03:27:37.515274+0000 mon.c (mon.2) 212 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:39.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[19769]: audit 2024-02-13T03:27:38.195299+0000 mon.a (mon.0) 1028 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-11"}]': finished 2024-02-13T03:27:39.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[19769]: cluster 2024-02-13T03:27:38.195455+0000 mon.a (mon.0) 1029 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-13T03:27:39.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[19769]: audit 2024-02-13T03:27:38.203779+0000 mon.b (mon.1) 107 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-11", "mode": "writeback"}]: dispatch 2024-02-13T03:27:39.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[19769]: audit 2024-02-13T03:27:38.213079+0000 mon.c (mon.2) 213 : audit [INF] from='client.? v1:172.21.15.161:0/3439885083' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi161-61719-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:39.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[19769]: audit 2024-02-13T03:27:38.220250+0000 mon.a (mon.0) 1030 : audit [INF] from='client.46361 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi161-61719-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:39.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[19769]: audit 2024-02-13T03:27:38.228584+0000 mon.a (mon.0) 1031 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-11", "mode": "writeback"}]: dispatch 2024-02-13T03:27:39.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[19769]: cluster 2024-02-13T03:27:38.274155+0000 mgr.y (mgr.24473) 270 : cluster [DBG] pgmap v229: 428 pgs: 32 unknown, 396 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 4 op/s 2024-02-13T03:27:39.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:38 smithi161 bash[19769]: audit 2024-02-13T03:27:38.517956+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:39.221 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: Running main() from gmock_main.cc 2024-02-13T03:27:39.221 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-02-13T03:27:39.221 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [----------] Global test environment set-up. 2024-02-13T03:27:39.221 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-02-13T03:27:39.221 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-02-13T03:27:39.221 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ OK ] LibRadosPools.PoolList (12867 ms) 2024-02-13T03:27:39.221 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-02-13T03:27:39.221 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (4716 ms) 2024-02-13T03:27:39.221 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-02-13T03:27:39.221 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (4612 ms) 2024-02-13T03:27:39.221 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3048 ms) 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3017 ms) 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (6400 ms) 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5144 ms) 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5204 ms) 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7914 ms) 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [----------] 9 tests from LibRadosPools (52924 ms total) 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: 2024-02-13T03:27:39.222 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [----------] Global test environment tear-down 2024-02-13T03:27:39.223 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (52924 ms total) 2024-02-13T03:27:39.223 INFO:tasks.workunit.client.0.smithi161.stdout: api_pool: [ PASSED ] 9 tests. 2024-02-13T03:27:39.235 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-02-13T03:27:39.235 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [----------] Global test environment set-up. 2024-02-13T03:27:39.235 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-02-13T03:27:39.235 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-02-13T03:27:39.235 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-02-13T03:27:39.235 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-13T03:27:39.235 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: 2024-02-13T03:27:39.235 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-02-13T03:27:39.235 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-02-13T03:27:39.235 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-02-13T03:27:39.235 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: 2024-02-13T03:26:46.319+0000 7f8332d3e580 -1 failed for service _ceph-mon._tcp 2024-02-13T03:27:39.235 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: 2024-02-13T03:26:46.319+0000 7f8332d3e580 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-02-13T03:27:39.235 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (50 ms) 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2007 ms) 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2057 ms total) 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: started 0x55bd538986d0 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: started 0x55bd538a1bb0 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: started 2 aios 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: waiting 0x55bd538986d0 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: waiting 0x55bd538a1bb0 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: done. 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (14542 ms) 2024-02-13T03:27:39.236 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (14542 ms total) 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ OK ] LibRadosMisc.Exec (3 ms) 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (8 ms) 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (12 ms) 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4479 ms) 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-02-13T03:27:39.237 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-02-13T03:27:39.238 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-02-13T03:27:39.238 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-02-13T03:27:39.238 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (27370 ms) 2024-02-13T03:27:39.238 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [----------] 8 tests from LibRadosMisc (31872 ms total) 2024-02-13T03:27:39.238 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: 2024-02-13T03:27:39.238 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [----------] Global test environment tear-down 2024-02-13T03:27:39.238 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (52948 ms total) 2024-02-13T03:27:39.238 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc: [ PASSED ] 12 tests. 2024-02-13T03:27:39.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:38 smithi183 bash[20824]: audit 2024-02-13T03:27:37.515274+0000 mon.c (mon.2) 212 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:39.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:38 smithi183 bash[20824]: audit 2024-02-13T03:27:38.195299+0000 mon.a (mon.0) 1028 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-11"}]': finished 2024-02-13T03:27:39.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:38 smithi183 bash[20824]: cluster 2024-02-13T03:27:38.195455+0000 mon.a (mon.0) 1029 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-13T03:27:39.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:38 smithi183 bash[20824]: audit 2024-02-13T03:27:38.203779+0000 mon.b (mon.1) 107 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-11", "mode": "writeback"}]: dispatch 2024-02-13T03:27:39.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:38 smithi183 bash[20824]: audit 2024-02-13T03:27:38.213079+0000 mon.c (mon.2) 213 : audit [INF] from='client.? v1:172.21.15.161:0/3439885083' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi161-61719-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:39.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:38 smithi183 bash[20824]: audit 2024-02-13T03:27:38.220250+0000 mon.a (mon.0) 1030 : audit [INF] from='client.46361 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi161-61719-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:39.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:38 smithi183 bash[20824]: audit 2024-02-13T03:27:38.228584+0000 mon.a (mon.0) 1031 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-11", "mode": "writeback"}]: dispatch 2024-02-13T03:27:39.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:38 smithi183 bash[20824]: cluster 2024-02-13T03:27:38.274155+0000 mgr.y (mgr.24473) 270 : cluster [DBG] pgmap v229: 428 pgs: 32 unknown, 396 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 4 op/s 2024-02-13T03:27:39.240 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:38 smithi183 bash[20824]: audit 2024-02-13T03:27:38.517956+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:39.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:27:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:27:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:27:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[27879]: cluster 2024-02-13T03:27:39.192252+0000 mon.a (mon.0) 1032 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[27879]: audit 2024-02-13T03:27:39.209642+0000 mon.a (mon.0) 1033 : audit [INF] from='client.46361 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi161-61719-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[27879]: audit 2024-02-13T03:27:39.212502+0000 mon.a (mon.0) 1034 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-11", "mode": "writeback"}]': finished 2024-02-13T03:27:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[27879]: cluster 2024-02-13T03:27:39.212839+0000 mon.a (mon.0) 1035 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-13T03:27:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[27879]: audit 2024-02-13T03:27:39.218805+0000 mon.b (mon.1) 108 : audit [INF] from='client.? v1:172.21.15.161:0/3812394696' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[27879]: audit 2024-02-13T03:27:39.226147+0000 mon.b (mon.1) 109 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[27879]: audit 2024-02-13T03:27:39.238803+0000 mon.a (mon.0) 1036 : audit [INF] from='client.47815 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[27879]: audit 2024-02-13T03:27:39.240348+0000 mon.a (mon.0) 1037 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:40.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[27879]: audit 2024-02-13T03:27:39.519265+0000 mon.c (mon.2) 215 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:40.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[19769]: cluster 2024-02-13T03:27:39.192252+0000 mon.a (mon.0) 1032 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:40.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[19769]: audit 2024-02-13T03:27:39.209642+0000 mon.a (mon.0) 1033 : audit [INF] from='client.46361 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi161-61719-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:40.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[19769]: audit 2024-02-13T03:27:39.212502+0000 mon.a (mon.0) 1034 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-11", "mode": "writeback"}]': finished 2024-02-13T03:27:40.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[19769]: cluster 2024-02-13T03:27:39.212839+0000 mon.a (mon.0) 1035 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-13T03:27:40.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[19769]: audit 2024-02-13T03:27:39.218805+0000 mon.b (mon.1) 108 : audit [INF] from='client.? v1:172.21.15.161:0/3812394696' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:40.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[19769]: audit 2024-02-13T03:27:39.226147+0000 mon.b (mon.1) 109 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:40.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[19769]: audit 2024-02-13T03:27:39.238803+0000 mon.a (mon.0) 1036 : audit [INF] from='client.47815 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:40.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[19769]: audit 2024-02-13T03:27:39.240348+0000 mon.a (mon.0) 1037 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:40.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:39 smithi161 bash[19769]: audit 2024-02-13T03:27:39.519265+0000 mon.c (mon.2) 215 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:39 smithi183 bash[20824]: cluster 2024-02-13T03:27:39.192252+0000 mon.a (mon.0) 1032 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:39 smithi183 bash[20824]: audit 2024-02-13T03:27:39.209642+0000 mon.a (mon.0) 1033 : audit [INF] from='client.46361 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi161-61719-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:39 smithi183 bash[20824]: audit 2024-02-13T03:27:39.212502+0000 mon.a (mon.0) 1034 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-11", "mode": "writeback"}]': finished 2024-02-13T03:27:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:39 smithi183 bash[20824]: cluster 2024-02-13T03:27:39.212839+0000 mon.a (mon.0) 1035 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-13T03:27:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:39 smithi183 bash[20824]: audit 2024-02-13T03:27:39.218805+0000 mon.b (mon.1) 108 : audit [INF] from='client.? v1:172.21.15.161:0/3812394696' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:39 smithi183 bash[20824]: audit 2024-02-13T03:27:39.226147+0000 mon.b (mon.1) 109 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:39 smithi183 bash[20824]: audit 2024-02-13T03:27:39.238803+0000 mon.a (mon.0) 1036 : audit [INF] from='client.47815 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:39 smithi183 bash[20824]: audit 2024-02-13T03:27:39.240348+0000 mon.a (mon.0) 1037 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:39 smithi183 bash[20824]: audit 2024-02-13T03:27:39.519265+0000 mon.c (mon.2) 215 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:41.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[19769]: cluster 2024-02-13T03:27:39.171571+0000 osd.0 (osd.0) 13 : cluster [DBG] 18.8 deep-scrub starts 2024-02-13T03:27:41.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[19769]: cluster 2024-02-13T03:27:39.178583+0000 osd.0 (osd.0) 14 : cluster [DBG] 18.8 deep-scrub ok 2024-02-13T03:27:41.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[19769]: audit 2024-02-13T03:27:40.202324+0000 mon.a (mon.0) 1038 : audit [INF] from='client.47815 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:41.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[19769]: audit 2024-02-13T03:27:40.202429+0000 mon.a (mon.0) 1039 : audit [INF] from='client.35305 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]': finished 2024-02-13T03:27:41.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[19769]: cluster 2024-02-13T03:27:40.203177+0000 mon.a (mon.0) 1040 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-13T03:27:41.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[19769]: audit 2024-02-13T03:27:40.210104+0000 mon.b (mon.1) 110 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:41.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[19769]: audit 2024-02-13T03:27:40.210431+0000 mon.a (mon.0) 1041 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:41.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[19769]: audit 2024-02-13T03:27:40.270589+0000 mon.b (mon.1) 111 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:41.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[19769]: cluster 2024-02-13T03:27:40.275101+0000 mgr.y (mgr.24473) 271 : cluster [DBG] pgmap v232: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:41.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[19769]: audit 2024-02-13T03:27:40.520839+0000 mon.c (mon.2) 216 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:41.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[19769]: audit 2024-02-13T03:27:40.672695+0000 mon.a (mon.0) 1042 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[27879]: cluster 2024-02-13T03:27:39.171571+0000 osd.0 (osd.0) 13 : cluster [DBG] 18.8 deep-scrub starts 2024-02-13T03:27:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[27879]: cluster 2024-02-13T03:27:39.178583+0000 osd.0 (osd.0) 14 : cluster [DBG] 18.8 deep-scrub ok 2024-02-13T03:27:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[27879]: audit 2024-02-13T03:27:40.202324+0000 mon.a (mon.0) 1038 : audit [INF] from='client.47815 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[27879]: audit 2024-02-13T03:27:40.202429+0000 mon.a (mon.0) 1039 : audit [INF] from='client.35305 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]': finished 2024-02-13T03:27:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[27879]: cluster 2024-02-13T03:27:40.203177+0000 mon.a (mon.0) 1040 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-13T03:27:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[27879]: audit 2024-02-13T03:27:40.210104+0000 mon.b (mon.1) 110 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:41.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[27879]: audit 2024-02-13T03:27:40.210431+0000 mon.a (mon.0) 1041 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:41.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[27879]: audit 2024-02-13T03:27:40.270589+0000 mon.b (mon.1) 111 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:41.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[27879]: cluster 2024-02-13T03:27:40.275101+0000 mgr.y (mgr.24473) 271 : cluster [DBG] pgmap v232: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:41.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[27879]: audit 2024-02-13T03:27:40.520839+0000 mon.c (mon.2) 216 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:41.101 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:40 smithi161 bash[27879]: audit 2024-02-13T03:27:40.672695+0000 mon.a (mon.0) 1042 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:41.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:40 smithi183 bash[20824]: cluster 2024-02-13T03:27:39.171571+0000 osd.0 (osd.0) 13 : cluster [DBG] 18.8 deep-scrub starts 2024-02-13T03:27:41.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:40 smithi183 bash[20824]: cluster 2024-02-13T03:27:39.178583+0000 osd.0 (osd.0) 14 : cluster [DBG] 18.8 deep-scrub ok 2024-02-13T03:27:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:40 smithi183 bash[20824]: audit 2024-02-13T03:27:40.202324+0000 mon.a (mon.0) 1038 : audit [INF] from='client.47815 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:40 smithi183 bash[20824]: audit 2024-02-13T03:27:40.202429+0000 mon.a (mon.0) 1039 : audit [INF] from='client.35305 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi161-61936-9"}]': finished 2024-02-13T03:27:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:40 smithi183 bash[20824]: cluster 2024-02-13T03:27:40.203177+0000 mon.a (mon.0) 1040 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-13T03:27:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:40 smithi183 bash[20824]: audit 2024-02-13T03:27:40.210104+0000 mon.b (mon.1) 110 : audit [INF] from='client.? v1:172.21.15.161:0/3176611375' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:40 smithi183 bash[20824]: audit 2024-02-13T03:27:40.210431+0000 mon.a (mon.0) 1041 : audit [INF] from='client.35305 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]: dispatch 2024-02-13T03:27:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:40 smithi183 bash[20824]: audit 2024-02-13T03:27:40.270589+0000 mon.b (mon.1) 111 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:40 smithi183 bash[20824]: cluster 2024-02-13T03:27:40.275101+0000 mgr.y (mgr.24473) 271 : cluster [DBG] pgmap v232: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:40 smithi183 bash[20824]: audit 2024-02-13T03:27:40.520839+0000 mon.c (mon.2) 216 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:40 smithi183 bash[20824]: audit 2024-02-13T03:27:40.672695+0000 mon.a (mon.0) 1042 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:27:41.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:27:41] "GET /metrics HTTP/1.1" 200 202473 "" "Prometheus/2.33.4" 2024-02-13T03:27:42.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[19769]: cluster 2024-02-13T03:27:41.199870+0000 mon.a (mon.0) 1043 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:27:42.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[19769]: audit 2024-02-13T03:27:41.210715+0000 mon.a (mon.0) 1044 : audit [INF] from='client.35305 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]': finished 2024-02-13T03:27:42.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[19769]: audit 2024-02-13T03:27:41.211491+0000 mon.a (mon.0) 1045 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:27:42.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[19769]: cluster 2024-02-13T03:27:41.211574+0000 mon.a (mon.0) 1046 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-13T03:27:42.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[19769]: audit 2024-02-13T03:27:41.216793+0000 mon.b (mon.1) 112 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11"}]: dispatch 2024-02-13T03:27:42.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[19769]: audit 2024-02-13T03:27:41.218386+0000 mon.b (mon.1) 113 : audit [INF] from='client.? v1:172.21.15.161:0/2594815875' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi161-61719-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[19769]: audit 2024-02-13T03:27:41.219003+0000 mon.a (mon.0) 1047 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11"}]: dispatch 2024-02-13T03:27:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[19769]: audit 2024-02-13T03:27:41.231833+0000 mon.a (mon.0) 1048 : audit [INF] from='client.48478 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi161-61719-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[19769]: audit 2024-02-13T03:27:41.247822+0000 mon.a (mon.0) 1049 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[19769]: audit 2024-02-13T03:27:41.249085+0000 mon.a (mon.0) 1050 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[19769]: audit 2024-02-13T03:27:41.522443+0000 mon.c (mon.2) 217 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[19769]: audit 2024-02-13T03:27:41.680763+0000 mon.a (mon.0) 1051 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:42.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[27879]: cluster 2024-02-13T03:27:41.199870+0000 mon.a (mon.0) 1043 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:27:42.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[27879]: audit 2024-02-13T03:27:41.210715+0000 mon.a (mon.0) 1044 : audit [INF] from='client.35305 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]': finished 2024-02-13T03:27:42.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[27879]: audit 2024-02-13T03:27:41.211491+0000 mon.a (mon.0) 1045 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:27:42.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[27879]: cluster 2024-02-13T03:27:41.211574+0000 mon.a (mon.0) 1046 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-13T03:27:42.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[27879]: audit 2024-02-13T03:27:41.216793+0000 mon.b (mon.1) 112 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11"}]: dispatch 2024-02-13T03:27:42.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[27879]: audit 2024-02-13T03:27:41.218386+0000 mon.b (mon.1) 113 : audit [INF] from='client.? v1:172.21.15.161:0/2594815875' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi161-61719-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:42.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[27879]: audit 2024-02-13T03:27:41.219003+0000 mon.a (mon.0) 1047 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11"}]: dispatch 2024-02-13T03:27:42.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[27879]: audit 2024-02-13T03:27:41.231833+0000 mon.a (mon.0) 1048 : audit [INF] from='client.48478 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi161-61719-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:42.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[27879]: audit 2024-02-13T03:27:41.247822+0000 mon.a (mon.0) 1049 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:42.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[27879]: audit 2024-02-13T03:27:41.249085+0000 mon.a (mon.0) 1050 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:42.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[27879]: audit 2024-02-13T03:27:41.522443+0000 mon.c (mon.2) 217 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:42.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:41 smithi161 bash[27879]: audit 2024-02-13T03:27:41.680763+0000 mon.a (mon.0) 1051 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:41 smithi183 bash[20824]: cluster 2024-02-13T03:27:41.199870+0000 mon.a (mon.0) 1043 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:27:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:41 smithi183 bash[20824]: audit 2024-02-13T03:27:41.210715+0000 mon.a (mon.0) 1044 : audit [INF] from='client.35305 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi161-61936-9"}]': finished 2024-02-13T03:27:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:41 smithi183 bash[20824]: audit 2024-02-13T03:27:41.211491+0000 mon.a (mon.0) 1045 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:27:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:41 smithi183 bash[20824]: cluster 2024-02-13T03:27:41.211574+0000 mon.a (mon.0) 1046 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-13T03:27:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:41 smithi183 bash[20824]: audit 2024-02-13T03:27:41.216793+0000 mon.b (mon.1) 112 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11"}]: dispatch 2024-02-13T03:27:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:41 smithi183 bash[20824]: audit 2024-02-13T03:27:41.218386+0000 mon.b (mon.1) 113 : audit [INF] from='client.? v1:172.21.15.161:0/2594815875' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi161-61719-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:41 smithi183 bash[20824]: audit 2024-02-13T03:27:41.219003+0000 mon.a (mon.0) 1047 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11"}]: dispatch 2024-02-13T03:27:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:41 smithi183 bash[20824]: audit 2024-02-13T03:27:41.231833+0000 mon.a (mon.0) 1048 : audit [INF] from='client.48478 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi161-61719-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:41 smithi183 bash[20824]: audit 2024-02-13T03:27:41.247822+0000 mon.a (mon.0) 1049 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:42.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:41 smithi183 bash[20824]: audit 2024-02-13T03:27:41.249085+0000 mon.a (mon.0) 1050 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:42.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:41 smithi183 bash[20824]: audit 2024-02-13T03:27:41.522443+0000 mon.c (mon.2) 217 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:42.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:41 smithi183 bash[20824]: audit 2024-02-13T03:27:41.680763+0000 mon.a (mon.0) 1051 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:27:43.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[27879]: cluster 2024-02-13T03:27:42.202854+0000 mon.a (mon.0) 1052 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:27:43.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[27879]: audit 2024-02-13T03:27:42.213204+0000 mon.a (mon.0) 1053 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11"}]': finished 2024-02-13T03:27:43.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[27879]: audit 2024-02-13T03:27:42.213432+0000 mon.a (mon.0) 1054 : audit [INF] from='client.48478 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi161-61719-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:43.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[27879]: audit 2024-02-13T03:27:42.213722+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:43.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[27879]: cluster 2024-02-13T03:27:42.213839+0000 mon.a (mon.0) 1056 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-13T03:27:43.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[27879]: audit 2024-02-13T03:27:42.217347+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi161-61936-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:43.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[27879]: audit 2024-02-13T03:27:42.222025+0000 mon.a (mon.0) 1058 : audit [INF] from='client.48487 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi161-61726-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:43.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[27879]: audit 2024-02-13T03:27:42.224348+0000 mon.b (mon.1) 114 : audit [INF] from='client.? v1:172.21.15.161:0/2083439515' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi161-61726-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:43.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[27879]: cluster 2024-02-13T03:27:42.275886+0000 mgr.y (mgr.24473) 272 : cluster [DBG] pgmap v235: 388 pgs: 64 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-13T03:27:43.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[27879]: audit 2024-02-13T03:27:42.524027+0000 mon.c (mon.2) 218 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:43.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[19769]: cluster 2024-02-13T03:27:42.202854+0000 mon.a (mon.0) 1052 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:27:43.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[19769]: audit 2024-02-13T03:27:42.213204+0000 mon.a (mon.0) 1053 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11"}]': finished 2024-02-13T03:27:43.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[19769]: audit 2024-02-13T03:27:42.213432+0000 mon.a (mon.0) 1054 : audit [INF] from='client.48478 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi161-61719-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:43.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[19769]: audit 2024-02-13T03:27:42.213722+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:43.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[19769]: cluster 2024-02-13T03:27:42.213839+0000 mon.a (mon.0) 1056 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-13T03:27:43.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[19769]: audit 2024-02-13T03:27:42.217347+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi161-61936-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:43.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[19769]: audit 2024-02-13T03:27:42.222025+0000 mon.a (mon.0) 1058 : audit [INF] from='client.48487 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi161-61726-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:43.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[19769]: audit 2024-02-13T03:27:42.224348+0000 mon.b (mon.1) 114 : audit [INF] from='client.? v1:172.21.15.161:0/2083439515' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi161-61726-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:43.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[19769]: cluster 2024-02-13T03:27:42.275886+0000 mgr.y (mgr.24473) 272 : cluster [DBG] pgmap v235: 388 pgs: 64 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-13T03:27:43.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:42 smithi161 bash[19769]: audit 2024-02-13T03:27:42.524027+0000 mon.c (mon.2) 218 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:43.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:42 smithi183 bash[20824]: cluster 2024-02-13T03:27:42.202854+0000 mon.a (mon.0) 1052 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:27:43.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:42 smithi183 bash[20824]: audit 2024-02-13T03:27:42.213204+0000 mon.a (mon.0) 1053 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-11"}]': finished 2024-02-13T03:27:43.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:42 smithi183 bash[20824]: audit 2024-02-13T03:27:42.213432+0000 mon.a (mon.0) 1054 : audit [INF] from='client.48478 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi161-61719-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:42 smithi183 bash[20824]: audit 2024-02-13T03:27:42.213722+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:27:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:42 smithi183 bash[20824]: cluster 2024-02-13T03:27:42.213839+0000 mon.a (mon.0) 1056 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-13T03:27:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:42 smithi183 bash[20824]: audit 2024-02-13T03:27:42.217347+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi161-61936-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:42 smithi183 bash[20824]: audit 2024-02-13T03:27:42.222025+0000 mon.a (mon.0) 1058 : audit [INF] from='client.48487 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi161-61726-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:42 smithi183 bash[20824]: audit 2024-02-13T03:27:42.224348+0000 mon.b (mon.1) 114 : audit [INF] from='client.? v1:172.21.15.161:0/2083439515' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi161-61726-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:42 smithi183 bash[20824]: cluster 2024-02-13T03:27:42.275886+0000 mgr.y (mgr.24473) 272 : cluster [DBG] pgmap v235: 388 pgs: 64 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-13T03:27:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:42 smithi183 bash[20824]: audit 2024-02-13T03:27:42.524027+0000 mon.c (mon.2) 218 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:44.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:44 smithi161 bash[27879]: audit 2024-02-13T03:27:43.525551+0000 mon.c (mon.2) 219 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:44.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:44 smithi161 bash[27879]: audit 2024-02-13T03:27:43.661042+0000 mon.a (mon.0) 1059 : audit [INF] from='client.48487 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi161-61726-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:44.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:44 smithi161 bash[27879]: cluster 2024-02-13T03:27:43.661440+0000 mon.a (mon.0) 1060 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-13T03:27:44.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:44 smithi161 bash[19769]: audit 2024-02-13T03:27:43.525551+0000 mon.c (mon.2) 219 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:44.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:44 smithi161 bash[19769]: audit 2024-02-13T03:27:43.661042+0000 mon.a (mon.0) 1059 : audit [INF] from='client.48487 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi161-61726-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:44.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:44 smithi161 bash[19769]: cluster 2024-02-13T03:27:43.661440+0000 mon.a (mon.0) 1060 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-13T03:27:44.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:44 smithi183 bash[20824]: audit 2024-02-13T03:27:43.525551+0000 mon.c (mon.2) 219 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:44.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:44 smithi183 bash[20824]: audit 2024-02-13T03:27:43.661042+0000 mon.a (mon.0) 1059 : audit [INF] from='client.48487 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi161-61726-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:44.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:44 smithi183 bash[20824]: cluster 2024-02-13T03:27:43.661440+0000 mon.a (mon.0) 1060 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-13T03:27:45.837 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:45 smithi183 bash[20824]: cluster 2024-02-13T03:27:44.276789+0000 mgr.y (mgr.24473) 273 : cluster [DBG] pgmap v237: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-13T03:27:45.837 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:45 smithi183 bash[20824]: audit 2024-02-13T03:27:44.527087+0000 mon.c (mon.2) 220 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:45.837 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:45 smithi183 bash[20824]: audit 2024-02-13T03:27:44.545454+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi161-61936-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]': finished 2024-02-13T03:27:45.837 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:45 smithi183 bash[20824]: cluster 2024-02-13T03:27:44.546140+0000 mon.a (mon.0) 1062 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-13T03:27:45.837 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:45 smithi183 bash[20824]: audit 2024-02-13T03:27:44.550231+0000 mon.c (mon.2) 221 : audit [INF] from='client.? v1:172.21.15.161:0/4100702288' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi161-61719-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:45.838 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:45 smithi183 bash[20824]: audit 2024-02-13T03:27:44.551266+0000 mon.a (mon.0) 1063 : audit [INF] from='client.47840 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi161-61719-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:45.838 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:45 smithi183 bash[20824]: audit 2024-02-13T03:27:44.554007+0000 mon.b (mon.1) 115 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:45.838 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:45 smithi183 bash[20824]: audit 2024-02-13T03:27:44.558510+0000 mon.a (mon.0) 1064 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:45.838 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:45 smithi183 bash[20824]: audit 2024-02-13T03:27:45.528680+0000 mon.c (mon.2) 222 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:45.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[19769]: cluster 2024-02-13T03:27:44.276789+0000 mgr.y (mgr.24473) 273 : cluster [DBG] pgmap v237: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-13T03:27:45.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[19769]: audit 2024-02-13T03:27:44.527087+0000 mon.c (mon.2) 220 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:45.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[19769]: audit 2024-02-13T03:27:44.545454+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi161-61936-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]': finished 2024-02-13T03:27:45.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[19769]: cluster 2024-02-13T03:27:44.546140+0000 mon.a (mon.0) 1062 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-13T03:27:45.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[19769]: audit 2024-02-13T03:27:44.550231+0000 mon.c (mon.2) 221 : audit [INF] from='client.? v1:172.21.15.161:0/4100702288' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi161-61719-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:45.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[19769]: audit 2024-02-13T03:27:44.551266+0000 mon.a (mon.0) 1063 : audit [INF] from='client.47840 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi161-61719-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:45.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[19769]: audit 2024-02-13T03:27:44.554007+0000 mon.b (mon.1) 115 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:45.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[19769]: audit 2024-02-13T03:27:44.558510+0000 mon.a (mon.0) 1064 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:45.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[19769]: audit 2024-02-13T03:27:45.528680+0000 mon.c (mon.2) 222 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:45.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[27879]: cluster 2024-02-13T03:27:44.276789+0000 mgr.y (mgr.24473) 273 : cluster [DBG] pgmap v237: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-13T03:27:45.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[27879]: audit 2024-02-13T03:27:44.527087+0000 mon.c (mon.2) 220 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:45.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[27879]: audit 2024-02-13T03:27:44.545454+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi161-61936-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]': finished 2024-02-13T03:27:45.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[27879]: cluster 2024-02-13T03:27:44.546140+0000 mon.a (mon.0) 1062 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-13T03:27:45.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[27879]: audit 2024-02-13T03:27:44.550231+0000 mon.c (mon.2) 221 : audit [INF] from='client.? v1:172.21.15.161:0/4100702288' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi161-61719-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:45.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[27879]: audit 2024-02-13T03:27:44.551266+0000 mon.a (mon.0) 1063 : audit [INF] from='client.47840 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi161-61719-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:45.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[27879]: audit 2024-02-13T03:27:44.554007+0000 mon.b (mon.1) 115 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:45.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[27879]: audit 2024-02-13T03:27:44.558510+0000 mon.a (mon.0) 1064 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:45.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:45 smithi161 bash[27879]: audit 2024-02-13T03:27:45.528680+0000 mon.c (mon.2) 222 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:46.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:27:45 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:27:46.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:46 smithi161 bash[27879]: audit 2024-02-13T03:27:45.555688+0000 mon.a (mon.0) 1065 : audit [INF] from='client.47840 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi161-61719-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:46.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:46 smithi161 bash[27879]: audit 2024-02-13T03:27:45.555950+0000 mon.a (mon.0) 1066 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:46.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:46 smithi161 bash[27879]: cluster 2024-02-13T03:27:45.556088+0000 mon.a (mon.0) 1067 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-13T03:27:46.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:46 smithi161 bash[27879]: audit 2024-02-13T03:27:45.834769+0000 mgr.y (mgr.24473) 274 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:46.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:46 smithi161 bash[27879]: cluster 2024-02-13T03:27:46.279929+0000 mgr.y (mgr.24473) 275 : cluster [DBG] pgmap v240: 364 pgs: 2 active, 31 creating+activating, 331 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:46.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:46 smithi161 bash[27879]: audit 2024-02-13T03:27:46.530438+0000 mon.c (mon.2) 223 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:46.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:46 smithi161 bash[19769]: audit 2024-02-13T03:27:45.555688+0000 mon.a (mon.0) 1065 : audit [INF] from='client.47840 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi161-61719-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:46.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:46 smithi161 bash[19769]: audit 2024-02-13T03:27:45.555950+0000 mon.a (mon.0) 1066 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:46.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:46 smithi161 bash[19769]: cluster 2024-02-13T03:27:45.556088+0000 mon.a (mon.0) 1067 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-13T03:27:46.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:46 smithi161 bash[19769]: audit 2024-02-13T03:27:45.834769+0000 mgr.y (mgr.24473) 274 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:46.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:46 smithi161 bash[19769]: cluster 2024-02-13T03:27:46.279929+0000 mgr.y (mgr.24473) 275 : cluster [DBG] pgmap v240: 364 pgs: 2 active, 31 creating+activating, 331 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:46.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:46 smithi161 bash[19769]: audit 2024-02-13T03:27:46.530438+0000 mon.c (mon.2) 223 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:46.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:46 smithi183 bash[20824]: audit 2024-02-13T03:27:45.555688+0000 mon.a (mon.0) 1065 : audit [INF] from='client.47840 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi161-61719-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:46.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:46 smithi183 bash[20824]: audit 2024-02-13T03:27:45.555950+0000 mon.a (mon.0) 1066 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:46.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:46 smithi183 bash[20824]: cluster 2024-02-13T03:27:45.556088+0000 mon.a (mon.0) 1067 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-13T03:27:46.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:46 smithi183 bash[20824]: audit 2024-02-13T03:27:45.834769+0000 mgr.y (mgr.24473) 274 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:46.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:46 smithi183 bash[20824]: cluster 2024-02-13T03:27:46.279929+0000 mgr.y (mgr.24473) 275 : cluster [DBG] pgmap v240: 364 pgs: 2 active, 31 creating+activating, 331 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:46.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:46 smithi183 bash[20824]: audit 2024-02-13T03:27:46.530438+0000 mon.c (mon.2) 223 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:47.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[27879]: cluster 2024-02-13T03:27:46.580866+0000 mon.a (mon.0) 1068 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-13T03:27:47.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[27879]: audit 2024-02-13T03:27:46.584144+0000 mon.c (mon.2) 224 : audit [INF] from='client.? v1:172.21.15.161:0/1405390758' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi161-61726-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:47.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[27879]: audit 2024-02-13T03:27:46.587935+0000 mon.a (mon.0) 1069 : audit [INF] from='client.47843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi161-61726-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:47.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[27879]: audit 2024-02-13T03:27:47.532157+0000 mon.c (mon.2) 225 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:47.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[27879]: audit 2024-02-13T03:27:47.584469+0000 mon.a (mon.0) 1070 : audit [INF] from='client.47843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi161-61726-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:47.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[27879]: cluster 2024-02-13T03:27:47.584749+0000 mon.a (mon.0) 1071 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-13T03:27:47.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[27879]: audit 2024-02-13T03:27:47.587774+0000 mon.c (mon.2) 226 : audit [INF] from='client.? v1:172.21.15.161:0/3710435080' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi161-61719-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:47.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[27879]: audit 2024-02-13T03:27:47.598118+0000 mon.a (mon.0) 1072 : audit [INF] from='client.47849 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi161-61719-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:47.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[19769]: cluster 2024-02-13T03:27:46.580866+0000 mon.a (mon.0) 1068 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-13T03:27:47.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[19769]: audit 2024-02-13T03:27:46.584144+0000 mon.c (mon.2) 224 : audit [INF] from='client.? v1:172.21.15.161:0/1405390758' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi161-61726-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:47.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[19769]: audit 2024-02-13T03:27:46.587935+0000 mon.a (mon.0) 1069 : audit [INF] from='client.47843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi161-61726-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:47.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[19769]: audit 2024-02-13T03:27:47.532157+0000 mon.c (mon.2) 225 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:47.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[19769]: audit 2024-02-13T03:27:47.584469+0000 mon.a (mon.0) 1070 : audit [INF] from='client.47843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi161-61726-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:47.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[19769]: cluster 2024-02-13T03:27:47.584749+0000 mon.a (mon.0) 1071 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-13T03:27:47.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[19769]: audit 2024-02-13T03:27:47.587774+0000 mon.c (mon.2) 226 : audit [INF] from='client.? v1:172.21.15.161:0/3710435080' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi161-61719-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:47.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:47 smithi161 bash[19769]: audit 2024-02-13T03:27:47.598118+0000 mon.a (mon.0) 1072 : audit [INF] from='client.47849 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi161-61719-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:47.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:47 smithi183 bash[20824]: cluster 2024-02-13T03:27:46.580866+0000 mon.a (mon.0) 1068 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-13T03:27:47.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:47 smithi183 bash[20824]: audit 2024-02-13T03:27:46.584144+0000 mon.c (mon.2) 224 : audit [INF] from='client.? v1:172.21.15.161:0/1405390758' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi161-61726-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:47.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:47 smithi183 bash[20824]: audit 2024-02-13T03:27:46.587935+0000 mon.a (mon.0) 1069 : audit [INF] from='client.47843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi161-61726-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:47.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:47 smithi183 bash[20824]: audit 2024-02-13T03:27:47.532157+0000 mon.c (mon.2) 225 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:47.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:47 smithi183 bash[20824]: audit 2024-02-13T03:27:47.584469+0000 mon.a (mon.0) 1070 : audit [INF] from='client.47843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi161-61726-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:47.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:47 smithi183 bash[20824]: cluster 2024-02-13T03:27:47.584749+0000 mon.a (mon.0) 1071 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-13T03:27:47.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:47 smithi183 bash[20824]: audit 2024-02-13T03:27:47.587774+0000 mon.c (mon.2) 226 : audit [INF] from='client.? v1:172.21.15.161:0/3710435080' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi161-61719-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:47.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:47 smithi183 bash[20824]: audit 2024-02-13T03:27:47.598118+0000 mon.a (mon.0) 1072 : audit [INF] from='client.47849 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi161-61719-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:49 smithi183 bash[20824]: audit 2024-02-13T03:27:48.203960+0000 mon.a (mon.0) 1073 : audit [INF] from='client.47849 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi161-61719-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:49 smithi183 bash[20824]: cluster 2024-02-13T03:27:48.204578+0000 mon.a (mon.0) 1074 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-13T03:27:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:49 smithi183 bash[20824]: cluster 2024-02-13T03:27:48.280944+0000 mgr.y (mgr.24473) 276 : cluster [DBG] pgmap v244: 364 pgs: 32 unknown, 1 active, 16 creating+activating, 315 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 547 B/s wr, 1 op/s 2024-02-13T03:27:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:49 smithi183 bash[20824]: audit 2024-02-13T03:27:48.533740+0000 mon.c (mon.2) 227 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:49.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:27:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:27:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:27:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:49 smithi161 bash[27879]: audit 2024-02-13T03:27:48.203960+0000 mon.a (mon.0) 1073 : audit [INF] from='client.47849 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi161-61719-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:49 smithi161 bash[27879]: cluster 2024-02-13T03:27:48.204578+0000 mon.a (mon.0) 1074 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-13T03:27:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:49 smithi161 bash[27879]: cluster 2024-02-13T03:27:48.280944+0000 mgr.y (mgr.24473) 276 : cluster [DBG] pgmap v244: 364 pgs: 32 unknown, 1 active, 16 creating+activating, 315 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 547 B/s wr, 1 op/s 2024-02-13T03:27:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:49 smithi161 bash[27879]: audit 2024-02-13T03:27:48.533740+0000 mon.c (mon.2) 227 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:49.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:49 smithi161 bash[19769]: audit 2024-02-13T03:27:48.203960+0000 mon.a (mon.0) 1073 : audit [INF] from='client.47849 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi161-61719-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:49.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:49 smithi161 bash[19769]: cluster 2024-02-13T03:27:48.204578+0000 mon.a (mon.0) 1074 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-13T03:27:49.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:49 smithi161 bash[19769]: cluster 2024-02-13T03:27:48.280944+0000 mgr.y (mgr.24473) 276 : cluster [DBG] pgmap v244: 364 pgs: 32 unknown, 1 active, 16 creating+activating, 315 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 547 B/s wr, 1 op/s 2024-02-13T03:27:49.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:49 smithi161 bash[19769]: audit 2024-02-13T03:27:48.533740+0000 mon.c (mon.2) 227 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:50.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:50 smithi161 bash[27879]: cluster 2024-02-13T03:27:49.224613+0000 mon.a (mon.0) 1075 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-13T03:27:50.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:50 smithi161 bash[27879]: audit 2024-02-13T03:27:49.229325+0000 mon.c (mon.2) 228 : audit [INF] from='client.? v1:172.21.15.161:0/403341621' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi161-61726-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:50.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:50 smithi161 bash[27879]: audit 2024-02-13T03:27:49.231737+0000 mon.a (mon.0) 1076 : audit [INF] from='client.47855 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi161-61726-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:50.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:50 smithi161 bash[27879]: audit 2024-02-13T03:27:49.397325+0000 mon.b (mon.1) 116 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:50.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:50 smithi161 bash[27879]: audit 2024-02-13T03:27:49.535259+0000 mon.c (mon.2) 229 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:50.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:50 smithi161 bash[27879]: audit 2024-02-13T03:27:49.701625+0000 mon.a (mon.0) 1077 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:50.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:50 smithi161 bash[19769]: cluster 2024-02-13T03:27:49.224613+0000 mon.a (mon.0) 1075 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-13T03:27:50.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:50 smithi161 bash[19769]: audit 2024-02-13T03:27:49.229325+0000 mon.c (mon.2) 228 : audit [INF] from='client.? v1:172.21.15.161:0/403341621' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi161-61726-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:50.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:50 smithi161 bash[19769]: audit 2024-02-13T03:27:49.231737+0000 mon.a (mon.0) 1076 : audit [INF] from='client.47855 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi161-61726-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:50.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:50 smithi161 bash[19769]: audit 2024-02-13T03:27:49.397325+0000 mon.b (mon.1) 116 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:50.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:50 smithi161 bash[19769]: audit 2024-02-13T03:27:49.535259+0000 mon.c (mon.2) 229 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:50.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:50 smithi161 bash[19769]: audit 2024-02-13T03:27:49.701625+0000 mon.a (mon.0) 1077 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:50.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:50 smithi183 bash[20824]: cluster 2024-02-13T03:27:49.224613+0000 mon.a (mon.0) 1075 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-13T03:27:50.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:50 smithi183 bash[20824]: audit 2024-02-13T03:27:49.229325+0000 mon.c (mon.2) 228 : audit [INF] from='client.? v1:172.21.15.161:0/403341621' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi161-61726-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:50.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:50 smithi183 bash[20824]: audit 2024-02-13T03:27:49.231737+0000 mon.a (mon.0) 1076 : audit [INF] from='client.47855 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi161-61726-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:50.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:50 smithi183 bash[20824]: audit 2024-02-13T03:27:49.397325+0000 mon.b (mon.1) 116 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:50.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:50 smithi183 bash[20824]: audit 2024-02-13T03:27:49.535259+0000 mon.c (mon.2) 229 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:50.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:50 smithi183 bash[20824]: audit 2024-02-13T03:27:49.701625+0000 mon.a (mon.0) 1077 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:27:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:27:51] "GET /metrics HTTP/1.1" 200 195775 "" "Prometheus/2.33.4" 2024-02-13T03:27:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[27879]: audit 2024-02-13T03:27:50.231737+0000 mon.a (mon.0) 1078 : audit [INF] from='client.47855 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi161-61726-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[27879]: audit 2024-02-13T03:27:50.232208+0000 mon.a (mon.0) 1079 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[27879]: cluster 2024-02-13T03:27:50.232314+0000 mon.a (mon.0) 1080 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-13T03:27:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[27879]: audit 2024-02-13T03:27:50.241091+0000 mon.b (mon.1) 117 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-13"}]: dispatch 2024-02-13T03:27:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[27879]: audit 2024-02-13T03:27:50.246383+0000 mon.a (mon.0) 1081 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-13"}]: dispatch 2024-02-13T03:27:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[27879]: audit 2024-02-13T03:27:50.246806+0000 mon.b (mon.1) 118 : audit [INF] from='client.? v1:172.21.15.161:0/1622670916' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi161-61719-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[27879]: audit 2024-02-13T03:27:50.247024+0000 mon.a (mon.0) 1082 : audit [INF] from='client.48520 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi161-61719-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[27879]: cluster 2024-02-13T03:27:50.281584+0000 mgr.y (mgr.24473) 277 : cluster [DBG] pgmap v247: 396 pgs: 64 unknown, 1 active, 16 creating+activating, 315 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[27879]: audit 2024-02-13T03:27:50.536767+0000 mon.c (mon.2) 230 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:51.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[19769]: audit 2024-02-13T03:27:50.231737+0000 mon.a (mon.0) 1078 : audit [INF] from='client.47855 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi161-61726-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:51.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[19769]: audit 2024-02-13T03:27:50.232208+0000 mon.a (mon.0) 1079 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:51.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[19769]: cluster 2024-02-13T03:27:50.232314+0000 mon.a (mon.0) 1080 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-13T03:27:51.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[19769]: audit 2024-02-13T03:27:50.241091+0000 mon.b (mon.1) 117 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-13"}]: dispatch 2024-02-13T03:27:51.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[19769]: audit 2024-02-13T03:27:50.246383+0000 mon.a (mon.0) 1081 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-13"}]: dispatch 2024-02-13T03:27:51.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[19769]: audit 2024-02-13T03:27:50.246806+0000 mon.b (mon.1) 118 : audit [INF] from='client.? v1:172.21.15.161:0/1622670916' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi161-61719-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:51.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[19769]: audit 2024-02-13T03:27:50.247024+0000 mon.a (mon.0) 1082 : audit [INF] from='client.48520 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi161-61719-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:51.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[19769]: cluster 2024-02-13T03:27:50.281584+0000 mgr.y (mgr.24473) 277 : cluster [DBG] pgmap v247: 396 pgs: 64 unknown, 1 active, 16 creating+activating, 315 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:51.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:51 smithi161 bash[19769]: audit 2024-02-13T03:27:50.536767+0000 mon.c (mon.2) 230 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:51 smithi183 bash[20824]: audit 2024-02-13T03:27:50.231737+0000 mon.a (mon.0) 1078 : audit [INF] from='client.47855 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi161-61726-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:51 smithi183 bash[20824]: audit 2024-02-13T03:27:50.232208+0000 mon.a (mon.0) 1079 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:27:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:51 smithi183 bash[20824]: cluster 2024-02-13T03:27:50.232314+0000 mon.a (mon.0) 1080 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-13T03:27:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:51 smithi183 bash[20824]: audit 2024-02-13T03:27:50.241091+0000 mon.b (mon.1) 117 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-13"}]: dispatch 2024-02-13T03:27:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:51 smithi183 bash[20824]: audit 2024-02-13T03:27:50.246383+0000 mon.a (mon.0) 1081 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-13"}]: dispatch 2024-02-13T03:27:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:51 smithi183 bash[20824]: audit 2024-02-13T03:27:50.246806+0000 mon.b (mon.1) 118 : audit [INF] from='client.? v1:172.21.15.161:0/1622670916' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi161-61719-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:51 smithi183 bash[20824]: audit 2024-02-13T03:27:50.247024+0000 mon.a (mon.0) 1082 : audit [INF] from='client.48520 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi161-61719-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:51 smithi183 bash[20824]: cluster 2024-02-13T03:27:50.281584+0000 mgr.y (mgr.24473) 277 : cluster [DBG] pgmap v247: 396 pgs: 64 unknown, 1 active, 16 creating+activating, 315 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:27:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:51 smithi183 bash[20824]: audit 2024-02-13T03:27:50.536767+0000 mon.c (mon.2) 230 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:52.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[27879]: audit 2024-02-13T03:27:51.234182+0000 mon.a (mon.0) 1083 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-13"}]': finished 2024-02-13T03:27:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[27879]: audit 2024-02-13T03:27:51.234607+0000 mon.a (mon.0) 1084 : audit [INF] from='client.48520 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi161-61719-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[27879]: cluster 2024-02-13T03:27:51.234899+0000 mon.a (mon.0) 1085 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-13T03:27:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[27879]: audit 2024-02-13T03:27:51.239261+0000 mon.b (mon.1) 119 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-13", "mode": "writeback"}]: dispatch 2024-02-13T03:27:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[27879]: audit 2024-02-13T03:27:51.241214+0000 mon.a (mon.0) 1086 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-13", "mode": "writeback"}]: dispatch 2024-02-13T03:27:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[27879]: audit 2024-02-13T03:27:51.538414+0000 mon.c (mon.2) 231 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[27879]: cluster 2024-02-13T03:27:52.227130+0000 mon.a (mon.0) 1087 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[27879]: audit 2024-02-13T03:27:52.240198+0000 mon.a (mon.0) 1088 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-13", "mode": "writeback"}]': finished 2024-02-13T03:27:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[27879]: cluster 2024-02-13T03:27:52.240557+0000 mon.a (mon.0) 1089 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-13T03:27:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[27879]: audit 2024-02-13T03:27:52.248134+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? v1:172.21.15.161:0/644123935' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi161-61726-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[19769]: audit 2024-02-13T03:27:51.234182+0000 mon.a (mon.0) 1083 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-13"}]': finished 2024-02-13T03:27:52.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[19769]: audit 2024-02-13T03:27:51.234607+0000 mon.a (mon.0) 1084 : audit [INF] from='client.48520 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi161-61719-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:52.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[19769]: cluster 2024-02-13T03:27:51.234899+0000 mon.a (mon.0) 1085 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-13T03:27:52.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[19769]: audit 2024-02-13T03:27:51.239261+0000 mon.b (mon.1) 119 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-13", "mode": "writeback"}]: dispatch 2024-02-13T03:27:52.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[19769]: audit 2024-02-13T03:27:51.241214+0000 mon.a (mon.0) 1086 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-13", "mode": "writeback"}]: dispatch 2024-02-13T03:27:52.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[19769]: audit 2024-02-13T03:27:51.538414+0000 mon.c (mon.2) 231 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:52.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[19769]: cluster 2024-02-13T03:27:52.227130+0000 mon.a (mon.0) 1087 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:52.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[19769]: audit 2024-02-13T03:27:52.240198+0000 mon.a (mon.0) 1088 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-13", "mode": "writeback"}]': finished 2024-02-13T03:27:52.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[19769]: cluster 2024-02-13T03:27:52.240557+0000 mon.a (mon.0) 1089 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-13T03:27:52.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:52 smithi161 bash[19769]: audit 2024-02-13T03:27:52.248134+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? v1:172.21.15.161:0/644123935' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi161-61726-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:52.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:52 smithi183 bash[20824]: audit 2024-02-13T03:27:51.234182+0000 mon.a (mon.0) 1083 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-13"}]': finished 2024-02-13T03:27:52.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:52 smithi183 bash[20824]: audit 2024-02-13T03:27:51.234607+0000 mon.a (mon.0) 1084 : audit [INF] from='client.48520 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi161-61719-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:52.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:52 smithi183 bash[20824]: cluster 2024-02-13T03:27:51.234899+0000 mon.a (mon.0) 1085 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-13T03:27:52.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:52 smithi183 bash[20824]: audit 2024-02-13T03:27:51.239261+0000 mon.b (mon.1) 119 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-13", "mode": "writeback"}]: dispatch 2024-02-13T03:27:52.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:52 smithi183 bash[20824]: audit 2024-02-13T03:27:51.241214+0000 mon.a (mon.0) 1086 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-13", "mode": "writeback"}]: dispatch 2024-02-13T03:27:52.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:52 smithi183 bash[20824]: audit 2024-02-13T03:27:51.538414+0000 mon.c (mon.2) 231 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:52.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:52 smithi183 bash[20824]: cluster 2024-02-13T03:27:52.227130+0000 mon.a (mon.0) 1087 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:27:52.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:52 smithi183 bash[20824]: audit 2024-02-13T03:27:52.240198+0000 mon.a (mon.0) 1088 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-13", "mode": "writeback"}]': finished 2024-02-13T03:27:52.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:52 smithi183 bash[20824]: cluster 2024-02-13T03:27:52.240557+0000 mon.a (mon.0) 1089 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-13T03:27:52.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:52 smithi183 bash[20824]: audit 2024-02-13T03:27:52.248134+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? v1:172.21.15.161:0/644123935' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi161-61726-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:53.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: cluster 2024-02-13T03:27:52.282111+0000 mgr.y (mgr.24473) 278 : cluster [DBG] pgmap v250: 364 pgs: 32 unknown, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 133 KiB/s wr, 127 op/s 2024-02-13T03:27:53.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:52.539713+0000 mon.c (mon.2) 232 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:53.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:53.058000+0000 mon.b (mon.1) 120 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.0"}]: dispatch 2024-02-13T03:27:53.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:53.058784+0000 mon.b (mon.1) 121 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.1"}]: dispatch 2024-02-13T03:27:53.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:53.059380+0000 mon.b (mon.1) 122 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.2"}]: dispatch 2024-02-13T03:27:53.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:53.059929+0000 mon.b (mon.1) 123 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.3"}]: dispatch 2024-02-13T03:27:53.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:53.060514+0000 mon.b (mon.1) 124 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.4"}]: dispatch 2024-02-13T03:27:53.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:53.061110+0000 mon.b (mon.1) 125 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.5"}]: dispatch 2024-02-13T03:27:53.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:53.061669+0000 mon.b (mon.1) 126 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.6"}]: dispatch 2024-02-13T03:27:53.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:53.062207+0000 mon.b (mon.1) 127 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.7"}]: dispatch 2024-02-13T03:27:53.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:53.062741+0000 mon.b (mon.1) 128 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.8"}]: dispatch 2024-02-13T03:27:53.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:53.063286+0000 mon.b (mon.1) 129 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.9"}]: dispatch 2024-02-13T03:27:53.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:53.239796+0000 mon.a (mon.0) 1091 : audit [INF] from='client.? v1:172.21.15.161:0/644123935' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi161-61726-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:53.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: cluster 2024-02-13T03:27:53.240199+0000 mon.a (mon.0) 1092 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-13T03:27:53.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:53.243126+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:53.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[19769]: audit 2024-02-13T03:27:53.247026+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? v1:172.21.15.161:0/2766101720' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi161-61719-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: cluster 2024-02-13T03:27:52.282111+0000 mgr.y (mgr.24473) 278 : cluster [DBG] pgmap v250: 364 pgs: 32 unknown, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 133 KiB/s wr, 127 op/s 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:52.539713+0000 mon.c (mon.2) 232 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:53.058000+0000 mon.b (mon.1) 120 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.0"}]: dispatch 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:53.058784+0000 mon.b (mon.1) 121 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.1"}]: dispatch 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:53.059380+0000 mon.b (mon.1) 122 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.2"}]: dispatch 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:53.059929+0000 mon.b (mon.1) 123 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.3"}]: dispatch 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:53.060514+0000 mon.b (mon.1) 124 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.4"}]: dispatch 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:53.061110+0000 mon.b (mon.1) 125 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.5"}]: dispatch 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:53.061669+0000 mon.b (mon.1) 126 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.6"}]: dispatch 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:53.062207+0000 mon.b (mon.1) 127 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.7"}]: dispatch 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:53.062741+0000 mon.b (mon.1) 128 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.8"}]: dispatch 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:53.063286+0000 mon.b (mon.1) 129 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.9"}]: dispatch 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:53.239796+0000 mon.a (mon.0) 1091 : audit [INF] from='client.? v1:172.21.15.161:0/644123935' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi161-61726-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: cluster 2024-02-13T03:27:53.240199+0000 mon.a (mon.0) 1092 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-13T03:27:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:53.243126+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:53 smithi161 bash[27879]: audit 2024-02-13T03:27:53.247026+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? v1:172.21.15.161:0/2766101720' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi161-61719-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: cluster 2024-02-13T03:27:52.282111+0000 mgr.y (mgr.24473) 278 : cluster [DBG] pgmap v250: 364 pgs: 32 unknown, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 133 KiB/s wr, 127 op/s 2024-02-13T03:27:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:52.539713+0000 mon.c (mon.2) 232 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:53.058000+0000 mon.b (mon.1) 120 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.0"}]: dispatch 2024-02-13T03:27:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:53.058784+0000 mon.b (mon.1) 121 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.1"}]: dispatch 2024-02-13T03:27:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:53.059380+0000 mon.b (mon.1) 122 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.2"}]: dispatch 2024-02-13T03:27:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:53.059929+0000 mon.b (mon.1) 123 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.3"}]: dispatch 2024-02-13T03:27:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:53.060514+0000 mon.b (mon.1) 124 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.4"}]: dispatch 2024-02-13T03:27:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:53.061110+0000 mon.b (mon.1) 125 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.5"}]: dispatch 2024-02-13T03:27:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:53.061669+0000 mon.b (mon.1) 126 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.6"}]: dispatch 2024-02-13T03:27:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:53.062207+0000 mon.b (mon.1) 127 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.7"}]: dispatch 2024-02-13T03:27:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:53.062741+0000 mon.b (mon.1) 128 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.8"}]: dispatch 2024-02-13T03:27:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:53.063286+0000 mon.b (mon.1) 129 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "94.9"}]: dispatch 2024-02-13T03:27:53.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:53.239796+0000 mon.a (mon.0) 1091 : audit [INF] from='client.? v1:172.21.15.161:0/644123935' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi161-61726-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:53.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: cluster 2024-02-13T03:27:53.240199+0000 mon.a (mon.0) 1092 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-13T03:27:53.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:53.243126+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:53.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:53 smithi183 bash[20824]: audit 2024-02-13T03:27:53.247026+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? v1:172.21.15.161:0/2766101720' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi161-61719-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:53.055026+0000 mgr.y (mgr.24473) 279 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.0"}]: dispatch 2024-02-13T03:27:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:53.055693+0000 mgr.y (mgr.24473) 280 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.1"}]: dispatch 2024-02-13T03:27:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:53.056274+0000 mgr.y (mgr.24473) 281 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.2"}]: dispatch 2024-02-13T03:27:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:53.056831+0000 mgr.y (mgr.24473) 282 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.3"}]: dispatch 2024-02-13T03:27:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:53.057417+0000 mgr.y (mgr.24473) 283 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.4"}]: dispatch 2024-02-13T03:27:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:53.058016+0000 mgr.y (mgr.24473) 284 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.5"}]: dispatch 2024-02-13T03:27:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:53.058565+0000 mgr.y (mgr.24473) 285 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.6"}]: dispatch 2024-02-13T03:27:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:53.059101+0000 mgr.y (mgr.24473) 286 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.7"}]: dispatch 2024-02-13T03:27:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:53.059632+0000 mgr.y (mgr.24473) 287 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.8"}]: dispatch 2024-02-13T03:27:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:53.060182+0000 mgr.y (mgr.24473) 288 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.9"}]: dispatch 2024-02-13T03:27:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: cluster 2024-02-13T03:27:53.260573+0000 mon.a (mon.0) 1095 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:27:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: cluster 2024-02-13T03:27:53.411243+0000 osd.0 (osd.0) 15 : cluster [DBG] 94.5 scrub starts 2024-02-13T03:27:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: cluster 2024-02-13T03:27:53.415633+0000 osd.0 (osd.0) 16 : cluster [DBG] 94.5 scrub ok 2024-02-13T03:27:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:53.540846+0000 mon.c (mon.2) 233 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:54.245987+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]': finished 2024-02-13T03:27:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:54.246476+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? v1:172.21.15.161:0/2766101720' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi161-61719-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: cluster 2024-02-13T03:27:54.247341+0000 mon.a (mon.0) 1098 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-13T03:27:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[27879]: audit 2024-02-13T03:27:54.249890+0000 mon.a (mon.0) 1099 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:53.055026+0000 mgr.y (mgr.24473) 279 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.0"}]: dispatch 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:53.055693+0000 mgr.y (mgr.24473) 280 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.1"}]: dispatch 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:53.056274+0000 mgr.y (mgr.24473) 281 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.2"}]: dispatch 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:53.056831+0000 mgr.y (mgr.24473) 282 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.3"}]: dispatch 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:53.057417+0000 mgr.y (mgr.24473) 283 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.4"}]: dispatch 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:53.058016+0000 mgr.y (mgr.24473) 284 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.5"}]: dispatch 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:53.058565+0000 mgr.y (mgr.24473) 285 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.6"}]: dispatch 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:53.059101+0000 mgr.y (mgr.24473) 286 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.7"}]: dispatch 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:53.059632+0000 mgr.y (mgr.24473) 287 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.8"}]: dispatch 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:53.060182+0000 mgr.y (mgr.24473) 288 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.9"}]: dispatch 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: cluster 2024-02-13T03:27:53.260573+0000 mon.a (mon.0) 1095 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: cluster 2024-02-13T03:27:53.411243+0000 osd.0 (osd.0) 15 : cluster [DBG] 94.5 scrub starts 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: cluster 2024-02-13T03:27:53.415633+0000 osd.0 (osd.0) 16 : cluster [DBG] 94.5 scrub ok 2024-02-13T03:27:54.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:53.540846+0000 mon.c (mon.2) 233 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:54.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:54.245987+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]': finished 2024-02-13T03:27:54.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:54.246476+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? v1:172.21.15.161:0/2766101720' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi161-61719-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:54.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: cluster 2024-02-13T03:27:54.247341+0000 mon.a (mon.0) 1098 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-13T03:27:54.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:54 smithi161 bash[19769]: audit 2024-02-13T03:27:54.249890+0000 mon.a (mon.0) 1099 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:53.055026+0000 mgr.y (mgr.24473) 279 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.0"}]: dispatch 2024-02-13T03:27:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:53.055693+0000 mgr.y (mgr.24473) 280 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.1"}]: dispatch 2024-02-13T03:27:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:53.056274+0000 mgr.y (mgr.24473) 281 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.2"}]: dispatch 2024-02-13T03:27:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:53.056831+0000 mgr.y (mgr.24473) 282 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.3"}]: dispatch 2024-02-13T03:27:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:53.057417+0000 mgr.y (mgr.24473) 283 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.4"}]: dispatch 2024-02-13T03:27:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:53.058016+0000 mgr.y (mgr.24473) 284 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.5"}]: dispatch 2024-02-13T03:27:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:53.058565+0000 mgr.y (mgr.24473) 285 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.6"}]: dispatch 2024-02-13T03:27:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:53.059101+0000 mgr.y (mgr.24473) 286 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.7"}]: dispatch 2024-02-13T03:27:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:53.059632+0000 mgr.y (mgr.24473) 287 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.8"}]: dispatch 2024-02-13T03:27:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:53.060182+0000 mgr.y (mgr.24473) 288 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "94.9"}]: dispatch 2024-02-13T03:27:54.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: cluster 2024-02-13T03:27:53.260573+0000 mon.a (mon.0) 1095 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:27:54.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: cluster 2024-02-13T03:27:53.411243+0000 osd.0 (osd.0) 15 : cluster [DBG] 94.5 scrub starts 2024-02-13T03:27:54.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: cluster 2024-02-13T03:27:53.415633+0000 osd.0 (osd.0) 16 : cluster [DBG] 94.5 scrub ok 2024-02-13T03:27:54.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:53.540846+0000 mon.c (mon.2) 233 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:54.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:54.245987+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]': finished 2024-02-13T03:27:54.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:54.246476+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? v1:172.21.15.161:0/2766101720' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi161-61719-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:54.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: cluster 2024-02-13T03:27:54.247341+0000 mon.a (mon.0) 1098 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-13T03:27:54.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:54 smithi183 bash[20824]: audit 2024-02-13T03:27:54.249890+0000 mon.a (mon.0) 1099 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]: dispatch 2024-02-13T03:27:55.266 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: Running main() from gmock_main.cc 2024-02-13T03:27:55.267 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [==========] Running 12 tests from 4 test suites. 2024-02-13T03:27:55.267 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [----------] Global test environment set-up. 2024-02-13T03:27:55.267 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-02-13T03:27:55.267 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-02-13T03:27:55.267 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (2495 ms) 2024-02-13T03:27:55.267 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-02-13T03:27:55.267 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (3710 ms) 2024-02-13T03:27:55.267 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-02-13T03:27:55.267 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (3611 ms) 2024-02-13T03:27:55.267 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-02-13T03:27:55.267 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (1999 ms) 2024-02-13T03:27:55.267 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (11815 ms total) 2024-02-13T03:27:55.267 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (5352 ms) 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4203 ms) 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged (9555 ms total) 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3071 ms) 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (1900 ms) 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-02-13T03:27:55.268 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (3095 ms) 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1911 ms) 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9977 ms total) 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (3696 ms) 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4017 ms) 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (7713 ms total) 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [----------] Global test environment tear-down 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [==========] 12 tests from 4 test suites ran. (68964 ms total) 2024-02-13T03:27:55.269 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots: [ PASSED ] 12 tests. 2024-02-13T03:27:55.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:27:55.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62179 2024-02-13T03:27:55.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 62179 2024-02-13T03:27:55.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:27:55.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61746 2024-02-13T03:27:55.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61746 2024-02-13T03:27:55.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:27:55.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62081 2024-02-13T03:27:55.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 62081 2024-02-13T03:27:55.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:27:55.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61964 2024-02-13T03:27:55.270 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61964 2024-02-13T03:27:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:55 smithi161 bash[19769]: cluster 2024-02-13T03:27:54.282816+0000 mgr.y (mgr.24473) 289 : cluster [DBG] pgmap v253: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 125 KiB/s wr, 125 op/s 2024-02-13T03:27:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:55 smithi161 bash[19769]: audit 2024-02-13T03:27:54.542303+0000 mon.c (mon.2) 234 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:55 smithi161 bash[19769]: audit 2024-02-13T03:27:55.246682+0000 mon.a (mon.0) 1100 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]': finished 2024-02-13T03:27:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:55 smithi161 bash[19769]: cluster 2024-02-13T03:27:55.246945+0000 mon.a (mon.0) 1101 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-13T03:27:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:55 smithi161 bash[19769]: audit 2024-02-13T03:27:55.254846+0000 mon.a (mon.0) 1102 : audit [INF] from='client.? v1:172.21.15.161:0/1570701927' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi161-61726-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:55 smithi161 bash[27879]: cluster 2024-02-13T03:27:54.282816+0000 mgr.y (mgr.24473) 289 : cluster [DBG] pgmap v253: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 125 KiB/s wr, 125 op/s 2024-02-13T03:27:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:55 smithi161 bash[27879]: audit 2024-02-13T03:27:54.542303+0000 mon.c (mon.2) 234 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:55 smithi161 bash[27879]: audit 2024-02-13T03:27:55.246682+0000 mon.a (mon.0) 1100 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]': finished 2024-02-13T03:27:55.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:55 smithi161 bash[27879]: cluster 2024-02-13T03:27:55.246945+0000 mon.a (mon.0) 1101 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-13T03:27:55.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:55 smithi161 bash[27879]: audit 2024-02-13T03:27:55.254846+0000 mon.a (mon.0) 1102 : audit [INF] from='client.? v1:172.21.15.161:0/1570701927' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi161-61726-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:55.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:55 smithi183 bash[20824]: cluster 2024-02-13T03:27:54.282816+0000 mgr.y (mgr.24473) 289 : cluster [DBG] pgmap v253: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 125 KiB/s wr, 125 op/s 2024-02-13T03:27:55.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:55 smithi183 bash[20824]: audit 2024-02-13T03:27:54.542303+0000 mon.c (mon.2) 234 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:55 smithi183 bash[20824]: audit 2024-02-13T03:27:55.246682+0000 mon.a (mon.0) 1100 : audit [INF] from='client.? v1:172.21.15.161:0/693736105' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi161-61936-14"}]': finished 2024-02-13T03:27:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:55 smithi183 bash[20824]: cluster 2024-02-13T03:27:55.246945+0000 mon.a (mon.0) 1101 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-13T03:27:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:55 smithi183 bash[20824]: audit 2024-02-13T03:27:55.254846+0000 mon.a (mon.0) 1102 : audit [INF] from='client.? v1:172.21.15.161:0/1570701927' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi161-61726-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:56.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:27:55 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:27:56.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:56 smithi161 bash[27879]: cluster 2024-02-13T03:27:55.425714+0000 osd.0 (osd.0) 17 : cluster [DBG] 94.8 scrub starts 2024-02-13T03:27:56.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:56 smithi161 bash[27879]: cluster 2024-02-13T03:27:55.431302+0000 osd.0 (osd.0) 18 : cluster [DBG] 94.8 scrub ok 2024-02-13T03:27:56.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:56 smithi161 bash[27879]: audit 2024-02-13T03:27:55.543772+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:56.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:56 smithi161 bash[27879]: audit 2024-02-13T03:27:56.254781+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? v1:172.21.15.161:0/1570701927' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi161-61726-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:56.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:56 smithi161 bash[27879]: cluster 2024-02-13T03:27:56.255773+0000 mon.a (mon.0) 1104 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-13T03:27:56.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:56 smithi161 bash[27879]: audit 2024-02-13T03:27:56.266033+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? v1:172.21.15.161:0/3495122856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi161-61719-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:56.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:56 smithi161 bash[19769]: cluster 2024-02-13T03:27:55.425714+0000 osd.0 (osd.0) 17 : cluster [DBG] 94.8 scrub starts 2024-02-13T03:27:56.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:56 smithi161 bash[19769]: cluster 2024-02-13T03:27:55.431302+0000 osd.0 (osd.0) 18 : cluster [DBG] 94.8 scrub ok 2024-02-13T03:27:56.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:56 smithi161 bash[19769]: audit 2024-02-13T03:27:55.543772+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:56.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:56 smithi161 bash[19769]: audit 2024-02-13T03:27:56.254781+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? v1:172.21.15.161:0/1570701927' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi161-61726-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:56.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:56 smithi161 bash[19769]: cluster 2024-02-13T03:27:56.255773+0000 mon.a (mon.0) 1104 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-13T03:27:56.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:56 smithi161 bash[19769]: audit 2024-02-13T03:27:56.266033+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? v1:172.21.15.161:0/3495122856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi161-61719-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:56 smithi183 bash[20824]: cluster 2024-02-13T03:27:55.425714+0000 osd.0 (osd.0) 17 : cluster [DBG] 94.8 scrub starts 2024-02-13T03:27:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:56 smithi183 bash[20824]: cluster 2024-02-13T03:27:55.431302+0000 osd.0 (osd.0) 18 : cluster [DBG] 94.8 scrub ok 2024-02-13T03:27:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:56 smithi183 bash[20824]: audit 2024-02-13T03:27:55.543772+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:56 smithi183 bash[20824]: audit 2024-02-13T03:27:56.254781+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? v1:172.21.15.161:0/1570701927' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi161-61726-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:56 smithi183 bash[20824]: cluster 2024-02-13T03:27:56.255773+0000 mon.a (mon.0) 1104 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-13T03:27:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:56 smithi183 bash[20824]: audit 2024-02-13T03:27:56.266033+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? v1:172.21.15.161:0/3495122856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi161-61719-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[27879]: audit 2024-02-13T03:27:55.838202+0000 mgr.y (mgr.24473) 290 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[27879]: cluster 2024-02-13T03:27:56.283560+0000 mgr.y (mgr.24473) 291 : cluster [DBG] pgmap v256: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 53 KiB/s rd, 76 op/s 2024-02-13T03:27:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[27879]: cluster 2024-02-13T03:27:56.408004+0000 osd.0 (osd.0) 19 : cluster [DBG] 94.9 scrub starts 2024-02-13T03:27:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[27879]: cluster 2024-02-13T03:27:56.412726+0000 osd.0 (osd.0) 20 : cluster [DBG] 94.9 scrub ok 2024-02-13T03:27:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[27879]: audit 2024-02-13T03:27:56.545416+0000 mon.c (mon.2) 236 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[27879]: audit 2024-02-13T03:27:57.263351+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? v1:172.21.15.161:0/3495122856' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi161-61719-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[27879]: cluster 2024-02-13T03:27:57.263887+0000 mon.a (mon.0) 1107 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-13T03:27:57.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[19769]: audit 2024-02-13T03:27:55.838202+0000 mgr.y (mgr.24473) 290 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:57.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[19769]: cluster 2024-02-13T03:27:56.283560+0000 mgr.y (mgr.24473) 291 : cluster [DBG] pgmap v256: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 53 KiB/s rd, 76 op/s 2024-02-13T03:27:57.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[19769]: cluster 2024-02-13T03:27:56.408004+0000 osd.0 (osd.0) 19 : cluster [DBG] 94.9 scrub starts 2024-02-13T03:27:57.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[19769]: cluster 2024-02-13T03:27:56.412726+0000 osd.0 (osd.0) 20 : cluster [DBG] 94.9 scrub ok 2024-02-13T03:27:57.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[19769]: audit 2024-02-13T03:27:56.545416+0000 mon.c (mon.2) 236 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:57.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[19769]: audit 2024-02-13T03:27:57.263351+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? v1:172.21.15.161:0/3495122856' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi161-61719-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:57.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:57 smithi161 bash[19769]: cluster 2024-02-13T03:27:57.263887+0000 mon.a (mon.0) 1107 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-13T03:27:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:57 smithi183 bash[20824]: audit 2024-02-13T03:27:55.838202+0000 mgr.y (mgr.24473) 290 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:27:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:57 smithi183 bash[20824]: cluster 2024-02-13T03:27:56.283560+0000 mgr.y (mgr.24473) 291 : cluster [DBG] pgmap v256: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 53 KiB/s rd, 76 op/s 2024-02-13T03:27:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:57 smithi183 bash[20824]: cluster 2024-02-13T03:27:56.408004+0000 osd.0 (osd.0) 19 : cluster [DBG] 94.9 scrub starts 2024-02-13T03:27:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:57 smithi183 bash[20824]: cluster 2024-02-13T03:27:56.412726+0000 osd.0 (osd.0) 20 : cluster [DBG] 94.9 scrub ok 2024-02-13T03:27:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:57 smithi183 bash[20824]: audit 2024-02-13T03:27:56.545416+0000 mon.c (mon.2) 236 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:57 smithi183 bash[20824]: audit 2024-02-13T03:27:57.263351+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? v1:172.21.15.161:0/3495122856' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi161-61719-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:57 smithi183 bash[20824]: cluster 2024-02-13T03:27:57.263887+0000 mon.a (mon.0) 1107 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-13T03:27:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:58 smithi161 bash[27879]: audit 2024-02-13T03:27:57.547014+0000 mon.c (mon.2) 237 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:58 smithi161 bash[27879]: cluster 2024-02-13T03:27:58.046099+0000 osd.2 (osd.2) 7 : cluster [DBG] 94.2 scrub starts 2024-02-13T03:27:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:58 smithi161 bash[27879]: cluster 2024-02-13T03:27:58.053051+0000 osd.2 (osd.2) 8 : cluster [DBG] 94.2 scrub ok 2024-02-13T03:27:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:58 smithi161 bash[27879]: cluster 2024-02-13T03:27:58.269814+0000 mon.a (mon.0) 1108 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-13T03:27:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:58 smithi161 bash[27879]: audit 2024-02-13T03:27:58.272568+0000 mon.c (mon.2) 238 : audit [INF] from='client.? v1:172.21.15.161:0/1728912528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi161-61726-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:58 smithi161 bash[27879]: audit 2024-02-13T03:27:58.275405+0000 mon.a (mon.0) 1109 : audit [INF] from='client.47879 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi161-61726-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:58 smithi161 bash[19769]: audit 2024-02-13T03:27:57.547014+0000 mon.c (mon.2) 237 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:58 smithi161 bash[19769]: cluster 2024-02-13T03:27:58.046099+0000 osd.2 (osd.2) 7 : cluster [DBG] 94.2 scrub starts 2024-02-13T03:27:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:58 smithi161 bash[19769]: cluster 2024-02-13T03:27:58.053051+0000 osd.2 (osd.2) 8 : cluster [DBG] 94.2 scrub ok 2024-02-13T03:27:58.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:58 smithi161 bash[19769]: cluster 2024-02-13T03:27:58.269814+0000 mon.a (mon.0) 1108 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-13T03:27:58.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:58 smithi161 bash[19769]: audit 2024-02-13T03:27:58.272568+0000 mon.c (mon.2) 238 : audit [INF] from='client.? v1:172.21.15.161:0/1728912528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi161-61726-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:58.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:58 smithi161 bash[19769]: audit 2024-02-13T03:27:58.275405+0000 mon.a (mon.0) 1109 : audit [INF] from='client.47879 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi161-61726-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:58 smithi183 bash[20824]: audit 2024-02-13T03:27:57.547014+0000 mon.c (mon.2) 237 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:58 smithi183 bash[20824]: cluster 2024-02-13T03:27:58.046099+0000 osd.2 (osd.2) 7 : cluster [DBG] 94.2 scrub starts 2024-02-13T03:27:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:58 smithi183 bash[20824]: cluster 2024-02-13T03:27:58.053051+0000 osd.2 (osd.2) 8 : cluster [DBG] 94.2 scrub ok 2024-02-13T03:27:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:58 smithi183 bash[20824]: cluster 2024-02-13T03:27:58.269814+0000 mon.a (mon.0) 1108 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-13T03:27:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:58 smithi183 bash[20824]: audit 2024-02-13T03:27:58.272568+0000 mon.c (mon.2) 238 : audit [INF] from='client.? v1:172.21.15.161:0/1728912528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi161-61726-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:58 smithi183 bash[20824]: audit 2024-02-13T03:27:58.275405+0000 mon.a (mon.0) 1109 : audit [INF] from='client.47879 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi161-61726-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:27:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[19769]: cluster 2024-02-13T03:27:58.098539+0000 osd.6 (osd.6) 3 : cluster [DBG] 94.3 scrub starts 2024-02-13T03:27:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[19769]: cluster 2024-02-13T03:27:58.103004+0000 osd.6 (osd.6) 4 : cluster [DBG] 94.3 scrub ok 2024-02-13T03:27:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[19769]: cluster 2024-02-13T03:27:58.143523+0000 osd.7 (osd.7) 11 : cluster [DBG] 94.1 scrub starts 2024-02-13T03:27:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[19769]: cluster 2024-02-13T03:27:58.146236+0000 osd.7 (osd.7) 12 : cluster [DBG] 94.1 scrub ok 2024-02-13T03:27:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[19769]: cluster 2024-02-13T03:27:58.284248+0000 mgr.y (mgr.24473) 292 : cluster [DBG] pgmap v259: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 76 op/s 2024-02-13T03:27:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[19769]: audit 2024-02-13T03:27:58.548503+0000 mon.c (mon.2) 239 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[19769]: audit 2024-02-13T03:27:59.277530+0000 mon.a (mon.0) 1110 : audit [INF] from='client.47879 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi161-61726-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[19769]: cluster 2024-02-13T03:27:59.277855+0000 mon.a (mon.0) 1111 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-13T03:27:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[27879]: cluster 2024-02-13T03:27:58.098539+0000 osd.6 (osd.6) 3 : cluster [DBG] 94.3 scrub starts 2024-02-13T03:27:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[27879]: cluster 2024-02-13T03:27:58.103004+0000 osd.6 (osd.6) 4 : cluster [DBG] 94.3 scrub ok 2024-02-13T03:27:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[27879]: cluster 2024-02-13T03:27:58.143523+0000 osd.7 (osd.7) 11 : cluster [DBG] 94.1 scrub starts 2024-02-13T03:27:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[27879]: cluster 2024-02-13T03:27:58.146236+0000 osd.7 (osd.7) 12 : cluster [DBG] 94.1 scrub ok 2024-02-13T03:27:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[27879]: cluster 2024-02-13T03:27:58.284248+0000 mgr.y (mgr.24473) 292 : cluster [DBG] pgmap v259: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 76 op/s 2024-02-13T03:27:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[27879]: audit 2024-02-13T03:27:58.548503+0000 mon.c (mon.2) 239 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[27879]: audit 2024-02-13T03:27:59.277530+0000 mon.a (mon.0) 1110 : audit [INF] from='client.47879 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi161-61726-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:27:59 smithi161 bash[27879]: cluster 2024-02-13T03:27:59.277855+0000 mon.a (mon.0) 1111 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-13T03:27:59.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:27:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:27:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:27:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:59 smithi183 bash[20824]: cluster 2024-02-13T03:27:58.098539+0000 osd.6 (osd.6) 3 : cluster [DBG] 94.3 scrub starts 2024-02-13T03:27:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:59 smithi183 bash[20824]: cluster 2024-02-13T03:27:58.103004+0000 osd.6 (osd.6) 4 : cluster [DBG] 94.3 scrub ok 2024-02-13T03:27:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:59 smithi183 bash[20824]: cluster 2024-02-13T03:27:58.143523+0000 osd.7 (osd.7) 11 : cluster [DBG] 94.1 scrub starts 2024-02-13T03:27:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:59 smithi183 bash[20824]: cluster 2024-02-13T03:27:58.146236+0000 osd.7 (osd.7) 12 : cluster [DBG] 94.1 scrub ok 2024-02-13T03:27:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:59 smithi183 bash[20824]: cluster 2024-02-13T03:27:58.284248+0000 mgr.y (mgr.24473) 292 : cluster [DBG] pgmap v259: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 76 op/s 2024-02-13T03:27:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:59 smithi183 bash[20824]: audit 2024-02-13T03:27:58.548503+0000 mon.c (mon.2) 239 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:27:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:59 smithi183 bash[20824]: audit 2024-02-13T03:27:59.277530+0000 mon.a (mon.0) 1110 : audit [INF] from='client.47879 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi161-61726-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:27:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:27:59 smithi183 bash[20824]: cluster 2024-02-13T03:27:59.277855+0000 mon.a (mon.0) 1111 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-13T03:28:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[27879]: cluster 2024-02-13T03:27:59.070267+0000 osd.2 (osd.2) 9 : cluster [DBG] 94.6 scrub starts 2024-02-13T03:28:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[27879]: cluster 2024-02-13T03:27:59.073634+0000 osd.2 (osd.2) 10 : cluster [DBG] 94.6 scrub ok 2024-02-13T03:28:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[27879]: audit 2024-02-13T03:27:59.303380+0000 mon.c (mon.2) 240 : audit [INF] from='client.? v1:172.21.15.161:0/820518063' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi161-61719-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[27879]: audit 2024-02-13T03:27:59.310357+0000 mon.a (mon.0) 1112 : audit [INF] from='client.47885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi161-61719-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[27879]: audit 2024-02-13T03:27:59.550063+0000 mon.c (mon.2) 241 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[27879]: audit 2024-02-13T03:28:00.283713+0000 mon.a (mon.0) 1113 : audit [INF] from='client.47885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi161-61719-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[27879]: cluster 2024-02-13T03:28:00.284097+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-13T03:28:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[27879]: audit 2024-02-13T03:28:00.298619+0000 mon.a (mon.0) 1115 : audit [INF] from='client.? v1:172.21.15.161:0/2581290098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61993-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:00.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[19769]: cluster 2024-02-13T03:27:59.070267+0000 osd.2 (osd.2) 9 : cluster [DBG] 94.6 scrub starts 2024-02-13T03:28:00.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[19769]: cluster 2024-02-13T03:27:59.073634+0000 osd.2 (osd.2) 10 : cluster [DBG] 94.6 scrub ok 2024-02-13T03:28:00.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[19769]: audit 2024-02-13T03:27:59.303380+0000 mon.c (mon.2) 240 : audit [INF] from='client.? v1:172.21.15.161:0/820518063' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi161-61719-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:00.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[19769]: audit 2024-02-13T03:27:59.310357+0000 mon.a (mon.0) 1112 : audit [INF] from='client.47885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi161-61719-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:00.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[19769]: audit 2024-02-13T03:27:59.550063+0000 mon.c (mon.2) 241 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:00.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[19769]: audit 2024-02-13T03:28:00.283713+0000 mon.a (mon.0) 1113 : audit [INF] from='client.47885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi161-61719-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:00.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[19769]: cluster 2024-02-13T03:28:00.284097+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-13T03:28:00.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:00 smithi161 bash[19769]: audit 2024-02-13T03:28:00.298619+0000 mon.a (mon.0) 1115 : audit [INF] from='client.? v1:172.21.15.161:0/2581290098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61993-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:00 smithi183 bash[20824]: cluster 2024-02-13T03:27:59.070267+0000 osd.2 (osd.2) 9 : cluster [DBG] 94.6 scrub starts 2024-02-13T03:28:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:00 smithi183 bash[20824]: cluster 2024-02-13T03:27:59.073634+0000 osd.2 (osd.2) 10 : cluster [DBG] 94.6 scrub ok 2024-02-13T03:28:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:00 smithi183 bash[20824]: audit 2024-02-13T03:27:59.303380+0000 mon.c (mon.2) 240 : audit [INF] from='client.? v1:172.21.15.161:0/820518063' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi161-61719-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:00 smithi183 bash[20824]: audit 2024-02-13T03:27:59.310357+0000 mon.a (mon.0) 1112 : audit [INF] from='client.47885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi161-61719-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:00 smithi183 bash[20824]: audit 2024-02-13T03:27:59.550063+0000 mon.c (mon.2) 241 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:00 smithi183 bash[20824]: audit 2024-02-13T03:28:00.283713+0000 mon.a (mon.0) 1113 : audit [INF] from='client.47885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi161-61719-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:00 smithi183 bash[20824]: cluster 2024-02-13T03:28:00.284097+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-13T03:28:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:00 smithi183 bash[20824]: audit 2024-02-13T03:28:00.298619+0000 mon.a (mon.0) 1115 : audit [INF] from='client.? v1:172.21.15.161:0/2581290098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61993-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:28:01] "GET /metrics HTTP/1.1" 200 195775 "" "Prometheus/2.33.4" 2024-02-13T03:28:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[19769]: cluster 2024-02-13T03:28:00.286125+0000 mgr.y (mgr.24473) 293 : cluster [DBG] pgmap v262: 356 pgs: 64 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[19769]: audit 2024-02-13T03:28:00.551557+0000 mon.c (mon.2) 242 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[19769]: audit 2024-02-13T03:28:00.927582+0000 mon.c (mon.2) 243 : audit [DBG] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-13T03:28:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[19769]: audit 2024-02-13T03:28:00.928134+0000 mon.c (mon.2) 244 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[19769]: audit 2024-02-13T03:28:00.928481+0000 mon.a (mon.0) 1116 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[19769]: audit 2024-02-13T03:28:01.292969+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? v1:172.21.15.161:0/2581290098' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61993-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[19769]: audit 2024-02-13T03:28:01.293267+0000 mon.a (mon.0) 1118 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:01.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[19769]: cluster 2024-02-13T03:28:01.293386+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-13T03:28:01.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[19769]: audit 2024-02-13T03:28:01.294971+0000 mon.c (mon.2) 245 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app2"}]: dispatch 2024-02-13T03:28:01.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[19769]: audit 2024-02-13T03:28:01.295626+0000 mon.c (mon.2) 246 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[19769]: audit 2024-02-13T03:28:01.300575+0000 mon.a (mon.0) 1120 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[19769]: audit 2024-02-13T03:28:01.306756+0000 mon.b (mon.1) 130 : audit [INF] from='client.? v1:172.21.15.161:0/4134772214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi161-61726-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[27879]: cluster 2024-02-13T03:28:00.286125+0000 mgr.y (mgr.24473) 293 : cluster [DBG] pgmap v262: 356 pgs: 64 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:01.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[27879]: audit 2024-02-13T03:28:00.551557+0000 mon.c (mon.2) 242 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:01.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[27879]: audit 2024-02-13T03:28:00.927582+0000 mon.c (mon.2) 243 : audit [DBG] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-13T03:28:01.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[27879]: audit 2024-02-13T03:28:00.928134+0000 mon.c (mon.2) 244 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[27879]: audit 2024-02-13T03:28:00.928481+0000 mon.a (mon.0) 1116 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[27879]: audit 2024-02-13T03:28:01.292969+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? v1:172.21.15.161:0/2581290098' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61993-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:01.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[27879]: audit 2024-02-13T03:28:01.293267+0000 mon.a (mon.0) 1118 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:01.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[27879]: cluster 2024-02-13T03:28:01.293386+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-13T03:28:01.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[27879]: audit 2024-02-13T03:28:01.294971+0000 mon.c (mon.2) 245 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app2"}]: dispatch 2024-02-13T03:28:01.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[27879]: audit 2024-02-13T03:28:01.295626+0000 mon.c (mon.2) 246 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[27879]: audit 2024-02-13T03:28:01.300575+0000 mon.a (mon.0) 1120 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:01 smithi161 bash[27879]: audit 2024-02-13T03:28:01.306756+0000 mon.b (mon.1) 130 : audit [INF] from='client.? v1:172.21.15.161:0/4134772214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi161-61726-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:01 smithi183 bash[20824]: cluster 2024-02-13T03:28:00.286125+0000 mgr.y (mgr.24473) 293 : cluster [DBG] pgmap v262: 356 pgs: 64 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:01 smithi183 bash[20824]: audit 2024-02-13T03:28:00.551557+0000 mon.c (mon.2) 242 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:01 smithi183 bash[20824]: audit 2024-02-13T03:28:00.927582+0000 mon.c (mon.2) 243 : audit [DBG] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-13T03:28:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:01 smithi183 bash[20824]: audit 2024-02-13T03:28:00.928134+0000 mon.c (mon.2) 244 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:01 smithi183 bash[20824]: audit 2024-02-13T03:28:00.928481+0000 mon.a (mon.0) 1116 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:01 smithi183 bash[20824]: audit 2024-02-13T03:28:01.292969+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? v1:172.21.15.161:0/2581290098' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61993-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:01 smithi183 bash[20824]: audit 2024-02-13T03:28:01.293267+0000 mon.a (mon.0) 1118 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:01 smithi183 bash[20824]: cluster 2024-02-13T03:28:01.293386+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-13T03:28:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:01 smithi183 bash[20824]: audit 2024-02-13T03:28:01.294971+0000 mon.c (mon.2) 245 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app2"}]: dispatch 2024-02-13T03:28:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:01 smithi183 bash[20824]: audit 2024-02-13T03:28:01.295626+0000 mon.c (mon.2) 246 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:01 smithi183 bash[20824]: audit 2024-02-13T03:28:01.300575+0000 mon.a (mon.0) 1120 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:01 smithi183 bash[20824]: audit 2024-02-13T03:28:01.306756+0000 mon.b (mon.1) 130 : audit [INF] from='client.? v1:172.21.15.161:0/4134772214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi161-61726-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:02.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[19769]: cluster 2024-02-13T03:28:01.077333+0000 osd.2 (osd.2) 11 : cluster [DBG] 94.7 scrub starts 2024-02-13T03:28:02.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[19769]: cluster 2024-02-13T03:28:01.082837+0000 osd.2 (osd.2) 12 : cluster [DBG] 94.7 scrub ok 2024-02-13T03:28:02.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[19769]: audit 2024-02-13T03:28:01.311981+0000 mon.a (mon.0) 1121 : audit [INF] from='client.48559 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi161-61726-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:02.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[19769]: audit 2024-02-13T03:28:01.553157+0000 mon.c (mon.2) 247 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:02.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[19769]: cluster 2024-02-13T03:28:02.287129+0000 mgr.y (mgr.24473) 294 : cluster [DBG] pgmap v264: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:02.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[19769]: audit 2024-02-13T03:28:02.302219+0000 mon.a (mon.0) 1122 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[19769]: audit 2024-02-13T03:28:02.303313+0000 mon.a (mon.0) 1123 : audit [INF] from='client.48559 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi161-61726-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[19769]: cluster 2024-02-13T03:28:02.303899+0000 mon.a (mon.0) 1124 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-13T03:28:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[19769]: audit 2024-02-13T03:28:02.305372+0000 mon.c (mon.2) 248 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:28:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[19769]: audit 2024-02-13T03:28:02.306090+0000 mon.c (mon.2) 249 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:28:02.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[19769]: audit 2024-02-13T03:28:02.311045+0000 mon.a (mon.0) 1125 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:28:02.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[27879]: cluster 2024-02-13T03:28:01.077333+0000 osd.2 (osd.2) 11 : cluster [DBG] 94.7 scrub starts 2024-02-13T03:28:02.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[27879]: cluster 2024-02-13T03:28:01.082837+0000 osd.2 (osd.2) 12 : cluster [DBG] 94.7 scrub ok 2024-02-13T03:28:02.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[27879]: audit 2024-02-13T03:28:01.311981+0000 mon.a (mon.0) 1121 : audit [INF] from='client.48559 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi161-61726-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:02.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[27879]: audit 2024-02-13T03:28:01.553157+0000 mon.c (mon.2) 247 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:02.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[27879]: cluster 2024-02-13T03:28:02.287129+0000 mgr.y (mgr.24473) 294 : cluster [DBG] pgmap v264: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:02.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[27879]: audit 2024-02-13T03:28:02.302219+0000 mon.a (mon.0) 1122 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:02.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[27879]: audit 2024-02-13T03:28:02.303313+0000 mon.a (mon.0) 1123 : audit [INF] from='client.48559 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi161-61726-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[27879]: cluster 2024-02-13T03:28:02.303899+0000 mon.a (mon.0) 1124 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-13T03:28:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[27879]: audit 2024-02-13T03:28:02.305372+0000 mon.c (mon.2) 248 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:28:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[27879]: audit 2024-02-13T03:28:02.306090+0000 mon.c (mon.2) 249 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:28:02.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:02 smithi161 bash[27879]: audit 2024-02-13T03:28:02.311045+0000 mon.a (mon.0) 1125 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:28:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:02 smithi183 bash[20824]: cluster 2024-02-13T03:28:01.077333+0000 osd.2 (osd.2) 11 : cluster [DBG] 94.7 scrub starts 2024-02-13T03:28:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:02 smithi183 bash[20824]: cluster 2024-02-13T03:28:01.082837+0000 osd.2 (osd.2) 12 : cluster [DBG] 94.7 scrub ok 2024-02-13T03:28:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:02 smithi183 bash[20824]: audit 2024-02-13T03:28:01.311981+0000 mon.a (mon.0) 1121 : audit [INF] from='client.48559 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi161-61726-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:02 smithi183 bash[20824]: audit 2024-02-13T03:28:01.553157+0000 mon.c (mon.2) 247 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:02 smithi183 bash[20824]: cluster 2024-02-13T03:28:02.287129+0000 mgr.y (mgr.24473) 294 : cluster [DBG] pgmap v264: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:02 smithi183 bash[20824]: audit 2024-02-13T03:28:02.302219+0000 mon.a (mon.0) 1122 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi161-61893-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:02 smithi183 bash[20824]: audit 2024-02-13T03:28:02.303313+0000 mon.a (mon.0) 1123 : audit [INF] from='client.48559 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi161-61726-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:02 smithi183 bash[20824]: cluster 2024-02-13T03:28:02.303899+0000 mon.a (mon.0) 1124 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-13T03:28:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:02 smithi183 bash[20824]: audit 2024-02-13T03:28:02.305372+0000 mon.c (mon.2) 248 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:28:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:02 smithi183 bash[20824]: audit 2024-02-13T03:28:02.306090+0000 mon.c (mon.2) 249 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:28:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:02 smithi183 bash[20824]: audit 2024-02-13T03:28:02.311045+0000 mon.a (mon.0) 1125 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-13T03:28:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[19769]: audit 2024-02-13T03:28:02.314322+0000 mon.c (mon.2) 250 : audit [INF] from='client.? v1:172.21.15.161:0/3137157164' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61719-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[19769]: audit 2024-02-13T03:28:02.315489+0000 mon.a (mon.0) 1126 : audit [INF] from='client.47903 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61719-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[19769]: audit 2024-02-13T03:28:02.554714+0000 mon.c (mon.2) 251 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[19769]: cluster 2024-02-13T03:28:03.201391+0000 mon.a (mon.0) 1127 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[19769]: audit 2024-02-13T03:28:03.304630+0000 mon.a (mon.0) 1128 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-13T03:28:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[19769]: audit 2024-02-13T03:28:03.305224+0000 mon.a (mon.0) 1129 : audit [INF] from='client.47903 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61719-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[19769]: cluster 2024-02-13T03:28:03.305735+0000 mon.a (mon.0) 1130 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-13T03:28:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[27879]: audit 2024-02-13T03:28:02.314322+0000 mon.c (mon.2) 250 : audit [INF] from='client.? v1:172.21.15.161:0/3137157164' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61719-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[27879]: audit 2024-02-13T03:28:02.315489+0000 mon.a (mon.0) 1126 : audit [INF] from='client.47903 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61719-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[27879]: audit 2024-02-13T03:28:02.554714+0000 mon.c (mon.2) 251 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[27879]: cluster 2024-02-13T03:28:03.201391+0000 mon.a (mon.0) 1127 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[27879]: audit 2024-02-13T03:28:03.304630+0000 mon.a (mon.0) 1128 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-13T03:28:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[27879]: audit 2024-02-13T03:28:03.305224+0000 mon.a (mon.0) 1129 : audit [INF] from='client.47903 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61719-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:03 smithi161 bash[27879]: cluster 2024-02-13T03:28:03.305735+0000 mon.a (mon.0) 1130 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-13T03:28:03.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:03 smithi183 bash[20824]: audit 2024-02-13T03:28:02.314322+0000 mon.c (mon.2) 250 : audit [INF] from='client.? v1:172.21.15.161:0/3137157164' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61719-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:03 smithi183 bash[20824]: audit 2024-02-13T03:28:02.315489+0000 mon.a (mon.0) 1126 : audit [INF] from='client.47903 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61719-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:03 smithi183 bash[20824]: audit 2024-02-13T03:28:02.554714+0000 mon.c (mon.2) 251 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:03 smithi183 bash[20824]: cluster 2024-02-13T03:28:03.201391+0000 mon.a (mon.0) 1127 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:03 smithi183 bash[20824]: audit 2024-02-13T03:28:03.304630+0000 mon.a (mon.0) 1128 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-13T03:28:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:03 smithi183 bash[20824]: audit 2024-02-13T03:28:03.305224+0000 mon.a (mon.0) 1129 : audit [INF] from='client.47903 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61719-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:03 smithi183 bash[20824]: cluster 2024-02-13T03:28:03.305735+0000 mon.a (mon.0) 1130 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-13T03:28:04.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:04 smithi161 bash[27879]: audit 2024-02-13T03:28:03.331493+0000 mon.c (mon.2) 252 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-13T03:28:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:04 smithi161 bash[27879]: audit 2024-02-13T03:28:03.342588+0000 mon.a (mon.0) 1131 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-13T03:28:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:04 smithi161 bash[27879]: cluster 2024-02-13T03:28:03.384862+0000 osd.1 (osd.1) 11 : cluster [DBG] 94.0 scrub starts 2024-02-13T03:28:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:04 smithi161 bash[27879]: cluster 2024-02-13T03:28:03.386075+0000 osd.1 (osd.1) 12 : cluster [DBG] 94.0 scrub ok 2024-02-13T03:28:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:04 smithi161 bash[27879]: audit 2024-02-13T03:28:03.556254+0000 mon.c (mon.2) 253 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:04 smithi161 bash[27879]: cluster 2024-02-13T03:28:04.288103+0000 mgr.y (mgr.24473) 295 : cluster [DBG] pgmap v267: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:04.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:04 smithi161 bash[19769]: audit 2024-02-13T03:28:03.331493+0000 mon.c (mon.2) 252 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-13T03:28:04.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:04 smithi161 bash[19769]: audit 2024-02-13T03:28:03.342588+0000 mon.a (mon.0) 1131 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-13T03:28:04.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:04 smithi161 bash[19769]: cluster 2024-02-13T03:28:03.384862+0000 osd.1 (osd.1) 11 : cluster [DBG] 94.0 scrub starts 2024-02-13T03:28:04.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:04 smithi161 bash[19769]: cluster 2024-02-13T03:28:03.386075+0000 osd.1 (osd.1) 12 : cluster [DBG] 94.0 scrub ok 2024-02-13T03:28:04.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:04 smithi161 bash[19769]: audit 2024-02-13T03:28:03.556254+0000 mon.c (mon.2) 253 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:04.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:04 smithi161 bash[19769]: cluster 2024-02-13T03:28:04.288103+0000 mgr.y (mgr.24473) 295 : cluster [DBG] pgmap v267: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:04 smithi183 bash[20824]: audit 2024-02-13T03:28:03.331493+0000 mon.c (mon.2) 252 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-13T03:28:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:04 smithi183 bash[20824]: audit 2024-02-13T03:28:03.342588+0000 mon.a (mon.0) 1131 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-13T03:28:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:04 smithi183 bash[20824]: cluster 2024-02-13T03:28:03.384862+0000 osd.1 (osd.1) 11 : cluster [DBG] 94.0 scrub starts 2024-02-13T03:28:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:04 smithi183 bash[20824]: cluster 2024-02-13T03:28:03.386075+0000 osd.1 (osd.1) 12 : cluster [DBG] 94.0 scrub ok 2024-02-13T03:28:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:04 smithi183 bash[20824]: audit 2024-02-13T03:28:03.556254+0000 mon.c (mon.2) 253 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:04 smithi183 bash[20824]: cluster 2024-02-13T03:28:04.288103+0000 mgr.y (mgr.24473) 295 : cluster [DBG] pgmap v267: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.350034+0000 mon.a (mon.0) 1132 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-13T03:28:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: cluster 2024-02-13T03:28:04.350248+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-13T03:28:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.352753+0000 mon.c (mon.2) 254 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1"}]: dispatch 2024-02-13T03:28:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.353427+0000 mon.a (mon.0) 1134 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1"}]: dispatch 2024-02-13T03:28:05.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.357516+0000 mon.a (mon.0) 1135 : audit [INF] from='client.48583 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:05.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.359330+0000 mon.b (mon.1) 131 : audit [INF] from='client.? v1:172.21.15.161:0/1884982059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:05.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.436235+0000 mon.c (mon.2) 255 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:05.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.557784+0000 mon.c (mon.2) 256 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:05.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.829345+0000 mon.a (mon.0) 1136 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:05.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.830535+0000 mon.c (mon.2) 257 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:05.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.831055+0000 mon.a (mon.0) 1137 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:05.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.831685+0000 mon.c (mon.2) 258 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:05.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.838774+0000 mon.c (mon.2) 259 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:28:05.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.839665+0000 mon.a (mon.0) 1138 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:28:05.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:04.840994+0000 mon.a (mon.0) 1139 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:05.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:05.224709+0000 mon.c (mon.2) 260 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:28:05.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:05.225986+0000 mon.c (mon.2) 261 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:28:05.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: cluster 2024-02-13T03:28:05.228133+0000 mgr.y (mgr.24473) 296 : cluster [DBG] pgmap v269: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:05.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[19769]: audit 2024-02-13T03:28:05.236288+0000 mon.a (mon.0) 1140 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:28:05.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.350034+0000 mon.a (mon.0) 1132 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-13T03:28:05.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: cluster 2024-02-13T03:28:04.350248+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-13T03:28:05.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.352753+0000 mon.c (mon.2) 254 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1"}]: dispatch 2024-02-13T03:28:05.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.353427+0000 mon.a (mon.0) 1134 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1"}]: dispatch 2024-02-13T03:28:05.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.357516+0000 mon.a (mon.0) 1135 : audit [INF] from='client.48583 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:05.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.359330+0000 mon.b (mon.1) 131 : audit [INF] from='client.? v1:172.21.15.161:0/1884982059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:05.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.436235+0000 mon.c (mon.2) 255 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:05.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.557784+0000 mon.c (mon.2) 256 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:05.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.829345+0000 mon.a (mon.0) 1136 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:05.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.830535+0000 mon.c (mon.2) 257 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:05.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.831055+0000 mon.a (mon.0) 1137 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:05.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.831685+0000 mon.c (mon.2) 258 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:05.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.838774+0000 mon.c (mon.2) 259 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:28:05.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.839665+0000 mon.a (mon.0) 1138 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:28:05.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:04.840994+0000 mon.a (mon.0) 1139 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:05.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:05.224709+0000 mon.c (mon.2) 260 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:28:05.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:05.225986+0000 mon.c (mon.2) 261 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:28:05.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: cluster 2024-02-13T03:28:05.228133+0000 mgr.y (mgr.24473) 296 : cluster [DBG] pgmap v269: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:05.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:05 smithi161 bash[27879]: audit 2024-02-13T03:28:05.236288+0000 mon.a (mon.0) 1140 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:28:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.350034+0000 mon.a (mon.0) 1132 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-13T03:28:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: cluster 2024-02-13T03:28:04.350248+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-13T03:28:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.352753+0000 mon.c (mon.2) 254 : audit [INF] from='client.? v1:172.21.15.161:0/296018926' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1"}]: dispatch 2024-02-13T03:28:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.353427+0000 mon.a (mon.0) 1134 : audit [INF] from='client.24746 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1"}]: dispatch 2024-02-13T03:28:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.357516+0000 mon.a (mon.0) 1135 : audit [INF] from='client.48583 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.359330+0000 mon.b (mon.1) 131 : audit [INF] from='client.? v1:172.21.15.161:0/1884982059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.436235+0000 mon.c (mon.2) 255 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.557784+0000 mon.c (mon.2) 256 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.829345+0000 mon.a (mon.0) 1136 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.830535+0000 mon.c (mon.2) 257 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:05.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.831055+0000 mon.a (mon.0) 1137 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:05.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.831685+0000 mon.c (mon.2) 258 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:05.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.838774+0000 mon.c (mon.2) 259 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:28:05.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.839665+0000 mon.a (mon.0) 1138 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:28:05.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:04.840994+0000 mon.a (mon.0) 1139 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:05.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:05.224709+0000 mon.c (mon.2) 260 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:28:05.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:05.225986+0000 mon.c (mon.2) 261 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:28:05.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: cluster 2024-02-13T03:28:05.228133+0000 mgr.y (mgr.24473) 296 : cluster [DBG] pgmap v269: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:05.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[20824]: audit 2024-02-13T03:28:05.236288+0000 mon.a (mon.0) 1140 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:28:06.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:28:05 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:28:06.373 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-02-13T03:28:06.373 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-02-13T03:28:06.373 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-02-13T03:28:06.373 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-02-13T03:28:06.373 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-02-13T03:28:06.373 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-13T03:28:06.373 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: seed 61893 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (9 ms) 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (262 ms) 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (14 ms) 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (6 ms) 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (13 ms) 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (4 ms) 2024-02-13T03:28:06.374 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (3 ms) 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (4 ms) 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (12 ms) 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (3 ms) 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (6 ms) 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (9 ms) 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-02-13T03:28:06.375 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: osd_max_attr_size = 0 2024-02-13T03:28:06.376 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-02-13T03:28:06.376 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1599 ms) 2024-02-13T03:28:06.376 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-02-13T03:28:06.376 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (415 ms) 2024-02-13T03:28:06.376 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-02-13T03:28:06.376 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-02-13T03:28:06.376 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: done waiting, doing copies 2024-02-13T03:28:06.376 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: waiting for final deep scrubs... 2024-02-13T03:28:06.376 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: done waiting 2024-02-13T03:28:06.376 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60880 ms) 2024-02-13T03:28:06.376 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-02-13T03:28:06.376 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (7 ms) 2024-02-13T03:28:06.376 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-02-13T03:28:06.377 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (4 ms) 2024-02-13T03:28:06.377 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-02-13T03:28:06.377 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4435 ms) 2024-02-13T03:28:06.377 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-02-13T03:28:06.377 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-02-13T03:28:06.377 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-02-13T03:28:06.377 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-02-13T03:28:06.377 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-02-13T03:28:06.377 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-02-13T03:28:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[27879]: audit 2024-02-13T03:28:05.357545+0000 mon.a (mon.0) 1141 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1"}]': finished 2024-02-13T03:28:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[27879]: audit 2024-02-13T03:28:05.357860+0000 mon.a (mon.0) 1142 : audit [INF] from='client.48583 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[27879]: audit 2024-02-13T03:28:05.358077+0000 mon.a (mon.0) 1143 : audit [INF] from='client.47933 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[27879]: cluster 2024-02-13T03:28:05.358150+0000 mon.a (mon.0) 1144 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-13T03:28:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[27879]: audit 2024-02-13T03:28:05.365837+0000 mon.c (mon.2) 262 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi161-61993-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[27879]: audit 2024-02-13T03:28:05.366770+0000 mon.c (mon.2) 263 : audit [INF] from='client.? v1:172.21.15.161:0/1767781971' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi161-61719-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[27879]: audit 2024-02-13T03:28:05.379028+0000 mon.a (mon.0) 1145 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi161-61993-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:07.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[27879]: audit 2024-02-13T03:28:05.380070+0000 mon.a (mon.0) 1146 : audit [INF] from='client.47927 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi161-61719-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:07.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[27879]: audit 2024-02-13T03:28:05.559296+0000 mon.c (mon.2) 264 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:07.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[27879]: cluster 2024-02-13T03:28:05.834027+0000 mon.a (mon.0) 1147 : cluster [INF] Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-02-13T03:28:07.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[27879]: audit 2024-02-13T03:28:05.847735+0000 mgr.y (mgr.24473) 297 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[19769]: audit 2024-02-13T03:28:05.357545+0000 mon.a (mon.0) 1141 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1"}]': finished 2024-02-13T03:28:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[19769]: audit 2024-02-13T03:28:05.357860+0000 mon.a (mon.0) 1142 : audit [INF] from='client.48583 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[19769]: audit 2024-02-13T03:28:05.358077+0000 mon.a (mon.0) 1143 : audit [INF] from='client.47933 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[19769]: cluster 2024-02-13T03:28:05.358150+0000 mon.a (mon.0) 1144 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-13T03:28:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[19769]: audit 2024-02-13T03:28:05.365837+0000 mon.c (mon.2) 262 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi161-61993-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[19769]: audit 2024-02-13T03:28:05.366770+0000 mon.c (mon.2) 263 : audit [INF] from='client.? v1:172.21.15.161:0/1767781971' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi161-61719-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[19769]: audit 2024-02-13T03:28:05.379028+0000 mon.a (mon.0) 1145 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi161-61993-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:07.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[19769]: audit 2024-02-13T03:28:05.380070+0000 mon.a (mon.0) 1146 : audit [INF] from='client.47927 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi161-61719-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:07.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[19769]: audit 2024-02-13T03:28:05.559296+0000 mon.c (mon.2) 264 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:07.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[19769]: cluster 2024-02-13T03:28:05.834027+0000 mon.a (mon.0) 1147 : cluster [INF] Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-02-13T03:28:07.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:06 smithi161 bash[19769]: audit 2024-02-13T03:28:05.847735+0000 mgr.y (mgr.24473) 297 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:06 smithi183 bash[20824]: audit 2024-02-13T03:28:05.357545+0000 mon.a (mon.0) 1141 : audit [INF] from='client.24746 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi161-61893-1","app":"app1","key":"key1"}]': finished 2024-02-13T03:28:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:06 smithi183 bash[20824]: audit 2024-02-13T03:28:05.357860+0000 mon.a (mon.0) 1142 : audit [INF] from='client.48583 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:06 smithi183 bash[20824]: audit 2024-02-13T03:28:05.358077+0000 mon.a (mon.0) 1143 : audit [INF] from='client.47933 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:06 smithi183 bash[20824]: cluster 2024-02-13T03:28:05.358150+0000 mon.a (mon.0) 1144 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-13T03:28:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:06 smithi183 bash[20824]: audit 2024-02-13T03:28:05.365837+0000 mon.c (mon.2) 262 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi161-61993-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:06 smithi183 bash[20824]: audit 2024-02-13T03:28:05.366770+0000 mon.c (mon.2) 263 : audit [INF] from='client.? v1:172.21.15.161:0/1767781971' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi161-61719-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:06 smithi183 bash[20824]: audit 2024-02-13T03:28:05.379028+0000 mon.a (mon.0) 1145 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi161-61993-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:06 smithi183 bash[20824]: audit 2024-02-13T03:28:05.380070+0000 mon.a (mon.0) 1146 : audit [INF] from='client.47927 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi161-61719-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:06 smithi183 bash[20824]: audit 2024-02-13T03:28:05.559296+0000 mon.c (mon.2) 264 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:06 smithi183 bash[20824]: cluster 2024-02-13T03:28:05.834027+0000 mon.a (mon.0) 1147 : cluster [INF] Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-02-13T03:28:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:06 smithi183 bash[20824]: audit 2024-02-13T03:28:05.847735+0000 mgr.y (mgr.24473) 297 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:08.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:06.418203+0000 mon.b (mon.1) 132 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:06.560956+0000 mon.c (mon.2) 265 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:08.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:06.798203+0000 mon.a (mon.0) 1148 : audit [INF] from='client.47927 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi161-61719-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:08.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: cluster 2024-02-13T03:28:06.798499+0000 mon.a (mon.0) 1149 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-13T03:28:08.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:06.808106+0000 mon.a (mon.0) 1150 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:06.809589+0000 mon.a (mon.0) 1151 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:06.811137+0000 mon.a (mon.0) 1152 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61893-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:08.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:06.812263+0000 mon.b (mon.1) 133 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:06.813848+0000 mon.b (mon.1) 134 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61893-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:08.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: cluster 2024-02-13T03:28:07.228991+0000 mgr.y (mgr.24473) 298 : cluster [DBG] pgmap v272: 292 pgs: 32 creating+peering, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:08.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:07.369101+0000 mon.a (mon.0) 1153 : audit [INF] from='client.47933 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi161-61993-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]': finished 2024-02-13T03:28:08.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:07.369286+0000 mon.a (mon.0) 1154 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61893-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:08.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: cluster 2024-02-13T03:28:07.369743+0000 mon.a (mon.0) 1155 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-13T03:28:08.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:07.373830+0000 mon.c (mon.2) 266 : audit [INF] from='client.? v1:172.21.15.161:0/821337026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi161-61726-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:08.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:07.376090+0000 mon.b (mon.1) 135 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61893-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:07.381313+0000 mon.a (mon.0) 1156 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61893-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[19769]: audit 2024-02-13T03:28:07.382009+0000 mon.a (mon.0) 1157 : audit [INF] from='client.47945 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi161-61726-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:08.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:06.418203+0000 mon.b (mon.1) 132 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:06.560956+0000 mon.c (mon.2) 265 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:08.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:06.798203+0000 mon.a (mon.0) 1148 : audit [INF] from='client.47927 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi161-61719-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:08.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: cluster 2024-02-13T03:28:06.798499+0000 mon.a (mon.0) 1149 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-13T03:28:08.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:06.808106+0000 mon.a (mon.0) 1150 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:06.809589+0000 mon.a (mon.0) 1151 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:06.811137+0000 mon.a (mon.0) 1152 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61893-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:08.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:06.812263+0000 mon.b (mon.1) 133 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:06.813848+0000 mon.b (mon.1) 134 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61893-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:08.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: cluster 2024-02-13T03:28:07.228991+0000 mgr.y (mgr.24473) 298 : cluster [DBG] pgmap v272: 292 pgs: 32 creating+peering, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:08.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:07.369101+0000 mon.a (mon.0) 1153 : audit [INF] from='client.47933 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi161-61993-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]': finished 2024-02-13T03:28:08.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:07.369286+0000 mon.a (mon.0) 1154 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61893-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:08.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: cluster 2024-02-13T03:28:07.369743+0000 mon.a (mon.0) 1155 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-13T03:28:08.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:07.373830+0000 mon.c (mon.2) 266 : audit [INF] from='client.? v1:172.21.15.161:0/821337026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi161-61726-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:08.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:07.376090+0000 mon.b (mon.1) 135 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61893-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:07.381313+0000 mon.a (mon.0) 1156 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61893-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:07 smithi161 bash[27879]: audit 2024-02-13T03:28:07.382009+0000 mon.a (mon.0) 1157 : audit [INF] from='client.47945 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi161-61726-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:06.418203+0000 mon.b (mon.1) 132 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:06.560956+0000 mon.c (mon.2) 265 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:06.798203+0000 mon.a (mon.0) 1148 : audit [INF] from='client.47927 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi161-61719-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: cluster 2024-02-13T03:28:06.798499+0000 mon.a (mon.0) 1149 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-13T03:28:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:06.808106+0000 mon.a (mon.0) 1150 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:06.809589+0000 mon.a (mon.0) 1151 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:06.811137+0000 mon.a (mon.0) 1152 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61893-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:08.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:06.812263+0000 mon.b (mon.1) 133 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:06.813848+0000 mon.b (mon.1) 134 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61893-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:08.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: cluster 2024-02-13T03:28:07.228991+0000 mgr.y (mgr.24473) 298 : cluster [DBG] pgmap v272: 292 pgs: 32 creating+peering, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:08.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:07.369101+0000 mon.a (mon.0) 1153 : audit [INF] from='client.47933 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi161-61993-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]': finished 2024-02-13T03:28:08.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:07.369286+0000 mon.a (mon.0) 1154 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61893-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:08.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: cluster 2024-02-13T03:28:07.369743+0000 mon.a (mon.0) 1155 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-13T03:28:08.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:07.373830+0000 mon.c (mon.2) 266 : audit [INF] from='client.? v1:172.21.15.161:0/821337026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi161-61726-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:08.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:07.376090+0000 mon.b (mon.1) 135 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61893-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:07.381313+0000 mon.a (mon.0) 1156 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61893-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:08.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:07 smithi183 bash[20824]: audit 2024-02-13T03:28:07.382009+0000 mon.a (mon.0) 1157 : audit [INF] from='client.47945 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi161-61726-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:09.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:08 smithi161 bash[27879]: audit 2024-02-13T03:28:07.562528+0000 mon.c (mon.2) 267 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:09.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:08 smithi161 bash[27879]: audit 2024-02-13T03:28:08.564022+0000 mon.c (mon.2) 268 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:09.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:08 smithi161 bash[27879]: audit 2024-02-13T03:28:08.797434+0000 mon.a (mon.0) 1158 : audit [INF] from='client.47945 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi161-61726-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:09.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:08 smithi161 bash[27879]: cluster 2024-02-13T03:28:08.798027+0000 mon.a (mon.0) 1159 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-13T03:28:09.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:08 smithi161 bash[27879]: audit 2024-02-13T03:28:08.799320+0000 mon.c (mon.2) 269 : audit [INF] from='client.? v1:172.21.15.161:0/1154722227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi161-61719-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:09.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:08 smithi161 bash[27879]: audit 2024-02-13T03:28:08.803317+0000 mon.a (mon.0) 1160 : audit [INF] from='client.47951 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi161-61719-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:09.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:08 smithi161 bash[19769]: audit 2024-02-13T03:28:07.562528+0000 mon.c (mon.2) 267 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:09.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:08 smithi161 bash[19769]: audit 2024-02-13T03:28:08.564022+0000 mon.c (mon.2) 268 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:09.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:08 smithi161 bash[19769]: audit 2024-02-13T03:28:08.797434+0000 mon.a (mon.0) 1158 : audit [INF] from='client.47945 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi161-61726-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:09.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:08 smithi161 bash[19769]: cluster 2024-02-13T03:28:08.798027+0000 mon.a (mon.0) 1159 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-13T03:28:09.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:08 smithi161 bash[19769]: audit 2024-02-13T03:28:08.799320+0000 mon.c (mon.2) 269 : audit [INF] from='client.? v1:172.21.15.161:0/1154722227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi161-61719-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:09.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:08 smithi161 bash[19769]: audit 2024-02-13T03:28:08.803317+0000 mon.a (mon.0) 1160 : audit [INF] from='client.47951 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi161-61719-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:09.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:08 smithi183 bash[20824]: audit 2024-02-13T03:28:07.562528+0000 mon.c (mon.2) 267 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:09.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:08 smithi183 bash[20824]: audit 2024-02-13T03:28:08.564022+0000 mon.c (mon.2) 268 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:09.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:08 smithi183 bash[20824]: audit 2024-02-13T03:28:08.797434+0000 mon.a (mon.0) 1158 : audit [INF] from='client.47945 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi161-61726-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:09.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:08 smithi183 bash[20824]: cluster 2024-02-13T03:28:08.798027+0000 mon.a (mon.0) 1159 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-13T03:28:09.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:08 smithi183 bash[20824]: audit 2024-02-13T03:28:08.799320+0000 mon.c (mon.2) 269 : audit [INF] from='client.? v1:172.21.15.161:0/1154722227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi161-61719-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:09.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:08 smithi183 bash[20824]: audit 2024-02-13T03:28:08.803317+0000 mon.a (mon.0) 1160 : audit [INF] from='client.47951 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi161-61719-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:09.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:28:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:28:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:28:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[27879]: cluster 2024-02-13T03:28:08.397854+0000 osd.1 (osd.1) 13 : cluster [DBG] 94.4 scrub starts 2024-02-13T03:28:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[27879]: cluster 2024-02-13T03:28:08.408199+0000 osd.1 (osd.1) 14 : cluster [DBG] 94.4 scrub ok 2024-02-13T03:28:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[27879]: cluster 2024-02-13T03:28:09.230034+0000 mgr.y (mgr.24473) 299 : cluster [DBG] pgmap v275: 332 pgs: 72 unknown, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[27879]: audit 2024-02-13T03:28:09.374239+0000 mon.a (mon.0) 1161 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61893-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61893-24"}]': finished 2024-02-13T03:28:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[27879]: audit 2024-02-13T03:28:09.374509+0000 mon.a (mon.0) 1162 : audit [INF] from='client.47951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi161-61719-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[27879]: cluster 2024-02-13T03:28:09.374597+0000 mon.a (mon.0) 1163 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-13T03:28:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[27879]: audit 2024-02-13T03:28:09.565527+0000 mon.c (mon.2) 270 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[19769]: cluster 2024-02-13T03:28:08.397854+0000 osd.1 (osd.1) 13 : cluster [DBG] 94.4 scrub starts 2024-02-13T03:28:10.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[19769]: cluster 2024-02-13T03:28:08.408199+0000 osd.1 (osd.1) 14 : cluster [DBG] 94.4 scrub ok 2024-02-13T03:28:10.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[19769]: cluster 2024-02-13T03:28:09.230034+0000 mgr.y (mgr.24473) 299 : cluster [DBG] pgmap v275: 332 pgs: 72 unknown, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:10.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[19769]: audit 2024-02-13T03:28:09.374239+0000 mon.a (mon.0) 1161 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61893-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61893-24"}]': finished 2024-02-13T03:28:10.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[19769]: audit 2024-02-13T03:28:09.374509+0000 mon.a (mon.0) 1162 : audit [INF] from='client.47951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi161-61719-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:10.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[19769]: cluster 2024-02-13T03:28:09.374597+0000 mon.a (mon.0) 1163 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-13T03:28:10.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:10 smithi161 bash[19769]: audit 2024-02-13T03:28:09.565527+0000 mon.c (mon.2) 270 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:10.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:10 smithi183 bash[20824]: cluster 2024-02-13T03:28:08.397854+0000 osd.1 (osd.1) 13 : cluster [DBG] 94.4 scrub starts 2024-02-13T03:28:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:10 smithi183 bash[20824]: cluster 2024-02-13T03:28:08.408199+0000 osd.1 (osd.1) 14 : cluster [DBG] 94.4 scrub ok 2024-02-13T03:28:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:10 smithi183 bash[20824]: cluster 2024-02-13T03:28:09.230034+0000 mgr.y (mgr.24473) 299 : cluster [DBG] pgmap v275: 332 pgs: 72 unknown, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-13T03:28:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:10 smithi183 bash[20824]: audit 2024-02-13T03:28:09.374239+0000 mon.a (mon.0) 1161 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61893-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61893-24"}]': finished 2024-02-13T03:28:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:10 smithi183 bash[20824]: audit 2024-02-13T03:28:09.374509+0000 mon.a (mon.0) 1162 : audit [INF] from='client.47951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi161-61719-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:10 smithi183 bash[20824]: cluster 2024-02-13T03:28:09.374597+0000 mon.a (mon.0) 1163 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-13T03:28:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:10 smithi183 bash[20824]: audit 2024-02-13T03:28:09.565527+0000 mon.c (mon.2) 270 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:11.391 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:28:11] "GET /metrics HTTP/1.1" 200 191220 "" "Prometheus/2.33.4" 2024-02-13T03:28:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[27879]: cluster 2024-02-13T03:28:10.378312+0000 mon.a (mon.0) 1164 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-13T03:28:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[27879]: audit 2024-02-13T03:28:10.380825+0000 mon.c (mon.2) 271 : audit [INF] from='client.? v1:172.21.15.161:0/3969977817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi161-61726-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[27879]: audit 2024-02-13T03:28:10.382969+0000 mon.a (mon.0) 1165 : audit [INF] from='client.47957 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi161-61726-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[27879]: audit 2024-02-13T03:28:10.390263+0000 mon.b (mon.1) 136 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[27879]: audit 2024-02-13T03:28:10.394460+0000 mon.a (mon.0) 1166 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[27879]: audit 2024-02-13T03:28:10.567099+0000 mon.c (mon.2) 272 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[27879]: cluster 2024-02-13T03:28:11.232984+0000 mgr.y (mgr.24473) 300 : cluster [DBG] pgmap v278: 340 pgs: 64 creating+peering, 276 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:28:11.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[19769]: cluster 2024-02-13T03:28:10.378312+0000 mon.a (mon.0) 1164 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-13T03:28:11.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[19769]: audit 2024-02-13T03:28:10.380825+0000 mon.c (mon.2) 271 : audit [INF] from='client.? v1:172.21.15.161:0/3969977817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi161-61726-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:11.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[19769]: audit 2024-02-13T03:28:10.382969+0000 mon.a (mon.0) 1165 : audit [INF] from='client.47957 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi161-61726-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:11.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[19769]: audit 2024-02-13T03:28:10.390263+0000 mon.b (mon.1) 136 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:11.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[19769]: audit 2024-02-13T03:28:10.394460+0000 mon.a (mon.0) 1166 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:11.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[19769]: audit 2024-02-13T03:28:10.567099+0000 mon.c (mon.2) 272 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:11.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:11 smithi161 bash[19769]: cluster 2024-02-13T03:28:11.232984+0000 mgr.y (mgr.24473) 300 : cluster [DBG] pgmap v278: 340 pgs: 64 creating+peering, 276 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:28:11.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:11 smithi183 bash[20824]: cluster 2024-02-13T03:28:10.378312+0000 mon.a (mon.0) 1164 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-13T03:28:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:11 smithi183 bash[20824]: audit 2024-02-13T03:28:10.380825+0000 mon.c (mon.2) 271 : audit [INF] from='client.? v1:172.21.15.161:0/3969977817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi161-61726-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:11 smithi183 bash[20824]: audit 2024-02-13T03:28:10.382969+0000 mon.a (mon.0) 1165 : audit [INF] from='client.47957 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi161-61726-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:11 smithi183 bash[20824]: audit 2024-02-13T03:28:10.390263+0000 mon.b (mon.1) 136 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:11 smithi183 bash[20824]: audit 2024-02-13T03:28:10.394460+0000 mon.a (mon.0) 1166 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:11 smithi183 bash[20824]: audit 2024-02-13T03:28:10.567099+0000 mon.c (mon.2) 272 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:11 smithi183 bash[20824]: cluster 2024-02-13T03:28:11.232984+0000 mgr.y (mgr.24473) 300 : cluster [DBG] pgmap v278: 340 pgs: 64 creating+peering, 276 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:28:12.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[27879]: cluster 2024-02-13T03:28:11.372963+0000 mon.a (mon.0) 1167 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:12.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[27879]: audit 2024-02-13T03:28:11.393870+0000 mon.a (mon.0) 1168 : audit [INF] from='client.47957 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi161-61726-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:12.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[27879]: audit 2024-02-13T03:28:11.394061+0000 mon.a (mon.0) 1169 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:12.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[27879]: cluster 2024-02-13T03:28:11.394346+0000 mon.a (mon.0) 1170 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-13T03:28:12.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[27879]: audit 2024-02-13T03:28:11.400258+0000 mon.a (mon.0) 1171 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:12.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[27879]: audit 2024-02-13T03:28:11.402271+0000 mon.c (mon.2) 273 : audit [INF] from='client.? v1:172.21.15.161:0/4177124832' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi161-61719-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:12.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[27879]: audit 2024-02-13T03:28:11.402887+0000 mon.b (mon.1) 137 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:12.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[27879]: audit 2024-02-13T03:28:11.409290+0000 mon.a (mon.0) 1172 : audit [INF] from='client.47963 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi161-61719-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:12.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[27879]: audit 2024-02-13T03:28:11.568606+0000 mon.c (mon.2) 274 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:12.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[19769]: cluster 2024-02-13T03:28:11.372963+0000 mon.a (mon.0) 1167 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:12.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[19769]: audit 2024-02-13T03:28:11.393870+0000 mon.a (mon.0) 1168 : audit [INF] from='client.47957 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi161-61726-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:12.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[19769]: audit 2024-02-13T03:28:11.394061+0000 mon.a (mon.0) 1169 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:12.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[19769]: cluster 2024-02-13T03:28:11.394346+0000 mon.a (mon.0) 1170 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-13T03:28:12.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[19769]: audit 2024-02-13T03:28:11.400258+0000 mon.a (mon.0) 1171 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:12.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[19769]: audit 2024-02-13T03:28:11.402271+0000 mon.c (mon.2) 273 : audit [INF] from='client.? v1:172.21.15.161:0/4177124832' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi161-61719-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:12.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[19769]: audit 2024-02-13T03:28:11.402887+0000 mon.b (mon.1) 137 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:12.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[19769]: audit 2024-02-13T03:28:11.409290+0000 mon.a (mon.0) 1172 : audit [INF] from='client.47963 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi161-61719-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:12.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:12 smithi161 bash[19769]: audit 2024-02-13T03:28:11.568606+0000 mon.c (mon.2) 274 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:12.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:12 smithi183 bash[20824]: cluster 2024-02-13T03:28:11.372963+0000 mon.a (mon.0) 1167 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:12 smithi183 bash[20824]: audit 2024-02-13T03:28:11.393870+0000 mon.a (mon.0) 1168 : audit [INF] from='client.47957 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi161-61726-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:12 smithi183 bash[20824]: audit 2024-02-13T03:28:11.394061+0000 mon.a (mon.0) 1169 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:12 smithi183 bash[20824]: cluster 2024-02-13T03:28:11.394346+0000 mon.a (mon.0) 1170 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-13T03:28:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:12 smithi183 bash[20824]: audit 2024-02-13T03:28:11.400258+0000 mon.a (mon.0) 1171 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:12 smithi183 bash[20824]: audit 2024-02-13T03:28:11.402271+0000 mon.c (mon.2) 273 : audit [INF] from='client.? v1:172.21.15.161:0/4177124832' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi161-61719-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:12 smithi183 bash[20824]: audit 2024-02-13T03:28:11.402887+0000 mon.b (mon.1) 137 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:12 smithi183 bash[20824]: audit 2024-02-13T03:28:11.409290+0000 mon.a (mon.0) 1172 : audit [INF] from='client.47963 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi161-61719-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:12 smithi183 bash[20824]: audit 2024-02-13T03:28:11.568606+0000 mon.c (mon.2) 274 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:13.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:13 smithi183 bash[20824]: audit 2024-02-13T03:28:12.396009+0000 mon.a (mon.0) 1173 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:13 smithi183 bash[20824]: audit 2024-02-13T03:28:12.396413+0000 mon.a (mon.0) 1174 : audit [INF] from='client.47963 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi161-61719-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:13 smithi183 bash[20824]: cluster 2024-02-13T03:28:12.397147+0000 mon.a (mon.0) 1175 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-13T03:28:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:13 smithi183 bash[20824]: audit 2024-02-13T03:28:12.569980+0000 mon.c (mon.2) 275 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:13 smithi183 bash[20824]: cluster 2024-02-13T03:28:13.218727+0000 mon.a (mon.0) 1176 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-13T03:28:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:13 smithi183 bash[20824]: audit 2024-02-13T03:28:13.223248+0000 mon.a (mon.0) 1177 : audit [INF] from='client.? v1:172.21.15.161:0/39028508' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi161-61726-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:14.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:13 smithi161 bash[27879]: audit 2024-02-13T03:28:12.396009+0000 mon.a (mon.0) 1173 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:14.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:13 smithi161 bash[27879]: audit 2024-02-13T03:28:12.396413+0000 mon.a (mon.0) 1174 : audit [INF] from='client.47963 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi161-61719-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:14.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:13 smithi161 bash[27879]: cluster 2024-02-13T03:28:12.397147+0000 mon.a (mon.0) 1175 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-13T03:28:14.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:13 smithi161 bash[27879]: audit 2024-02-13T03:28:12.569980+0000 mon.c (mon.2) 275 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:14.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:13 smithi161 bash[27879]: cluster 2024-02-13T03:28:13.218727+0000 mon.a (mon.0) 1176 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-13T03:28:14.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:13 smithi161 bash[27879]: audit 2024-02-13T03:28:13.223248+0000 mon.a (mon.0) 1177 : audit [INF] from='client.? v1:172.21.15.161:0/39028508' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi161-61726-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:14.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:13 smithi161 bash[19769]: audit 2024-02-13T03:28:12.396009+0000 mon.a (mon.0) 1173 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61893-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:14.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:13 smithi161 bash[19769]: audit 2024-02-13T03:28:12.396413+0000 mon.a (mon.0) 1174 : audit [INF] from='client.47963 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi161-61719-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:14.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:13 smithi161 bash[19769]: cluster 2024-02-13T03:28:12.397147+0000 mon.a (mon.0) 1175 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-13T03:28:14.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:13 smithi161 bash[19769]: audit 2024-02-13T03:28:12.569980+0000 mon.c (mon.2) 275 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:14.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:13 smithi161 bash[19769]: cluster 2024-02-13T03:28:13.218727+0000 mon.a (mon.0) 1176 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-13T03:28:14.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:13 smithi161 bash[19769]: audit 2024-02-13T03:28:13.223248+0000 mon.a (mon.0) 1177 : audit [INF] from='client.? v1:172.21.15.161:0/39028508' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi161-61726-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:14 smithi183 bash[20824]: cluster 2024-02-13T03:28:13.233741+0000 mgr.y (mgr.24473) 301 : cluster [DBG] pgmap v282: 308 pgs: 32 unknown, 276 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 530 B/s wr, 1 op/s 2024-02-13T03:28:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:14 smithi183 bash[20824]: audit 2024-02-13T03:28:13.571148+0000 mon.c (mon.2) 276 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:14 smithi183 bash[20824]: audit 2024-02-13T03:28:14.224764+0000 mon.a (mon.0) 1178 : audit [INF] from='client.? v1:172.21.15.161:0/39028508' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi161-61726-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:14 smithi183 bash[20824]: cluster 2024-02-13T03:28:14.225217+0000 mon.a (mon.0) 1179 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-13T03:28:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:14 smithi183 bash[20824]: audit 2024-02-13T03:28:14.238861+0000 mon.a (mon.0) 1180 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:14 smithi183 bash[20824]: audit 2024-02-13T03:28:14.239487+0000 mon.a (mon.0) 1181 : audit [INF] from='client.48643 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi161-61719-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:14 smithi183 bash[20824]: audit 2024-02-13T03:28:14.241540+0000 mon.b (mon.1) 138 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:14 smithi183 bash[20824]: audit 2024-02-13T03:28:14.241913+0000 mon.b (mon.1) 139 : audit [INF] from='client.? v1:172.21.15.161:0/2690641893' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi161-61719-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:14.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:14 smithi183 bash[20824]: audit 2024-02-13T03:28:14.572676+0000 mon.c (mon.2) 277 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:15.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[27879]: cluster 2024-02-13T03:28:13.233741+0000 mgr.y (mgr.24473) 301 : cluster [DBG] pgmap v282: 308 pgs: 32 unknown, 276 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 530 B/s wr, 1 op/s 2024-02-13T03:28:15.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[27879]: audit 2024-02-13T03:28:13.571148+0000 mon.c (mon.2) 276 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:15.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[27879]: audit 2024-02-13T03:28:14.224764+0000 mon.a (mon.0) 1178 : audit [INF] from='client.? v1:172.21.15.161:0/39028508' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi161-61726-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:15.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[27879]: cluster 2024-02-13T03:28:14.225217+0000 mon.a (mon.0) 1179 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-13T03:28:15.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[27879]: audit 2024-02-13T03:28:14.238861+0000 mon.a (mon.0) 1180 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:15.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[27879]: audit 2024-02-13T03:28:14.239487+0000 mon.a (mon.0) 1181 : audit [INF] from='client.48643 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi161-61719-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:15.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[27879]: audit 2024-02-13T03:28:14.241540+0000 mon.b (mon.1) 138 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:15.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[27879]: audit 2024-02-13T03:28:14.241913+0000 mon.b (mon.1) 139 : audit [INF] from='client.? v1:172.21.15.161:0/2690641893' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi161-61719-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:15.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[27879]: audit 2024-02-13T03:28:14.572676+0000 mon.c (mon.2) 277 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:15.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[19769]: cluster 2024-02-13T03:28:13.233741+0000 mgr.y (mgr.24473) 301 : cluster [DBG] pgmap v282: 308 pgs: 32 unknown, 276 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 530 B/s wr, 1 op/s 2024-02-13T03:28:15.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[19769]: audit 2024-02-13T03:28:13.571148+0000 mon.c (mon.2) 276 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:15.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[19769]: audit 2024-02-13T03:28:14.224764+0000 mon.a (mon.0) 1178 : audit [INF] from='client.? v1:172.21.15.161:0/39028508' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi161-61726-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:15.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[19769]: cluster 2024-02-13T03:28:14.225217+0000 mon.a (mon.0) 1179 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-13T03:28:15.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[19769]: audit 2024-02-13T03:28:14.238861+0000 mon.a (mon.0) 1180 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:15.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[19769]: audit 2024-02-13T03:28:14.239487+0000 mon.a (mon.0) 1181 : audit [INF] from='client.48643 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi161-61719-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:15.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[19769]: audit 2024-02-13T03:28:14.241540+0000 mon.b (mon.1) 138 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:15.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[19769]: audit 2024-02-13T03:28:14.241913+0000 mon.b (mon.1) 139 : audit [INF] from='client.? v1:172.21.15.161:0/2690641893' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi161-61719-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:15.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:14 smithi161 bash[19769]: audit 2024-02-13T03:28:14.572676+0000 mon.c (mon.2) 277 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:16.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:28:15 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:28:16.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[27879]: audit 2024-02-13T03:28:15.229574+0000 mon.a (mon.0) 1182 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]': finished 2024-02-13T03:28:16.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[27879]: audit 2024-02-13T03:28:15.229799+0000 mon.a (mon.0) 1183 : audit [INF] from='client.48643 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi161-61719-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:16.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[27879]: cluster 2024-02-13T03:28:15.230264+0000 mon.a (mon.0) 1184 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-13T03:28:16.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[27879]: audit 2024-02-13T03:28:15.234026+0000 mon.b (mon.1) 140 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:16.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[27879]: audit 2024-02-13T03:28:15.234135+0000 mon.a (mon.0) 1185 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:16.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[27879]: cluster 2024-02-13T03:28:15.234439+0000 mgr.y (mgr.24473) 302 : cluster [DBG] pgmap v285: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:16.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[27879]: audit 2024-02-13T03:28:15.574179+0000 mon.c (mon.2) 278 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[19769]: audit 2024-02-13T03:28:15.229574+0000 mon.a (mon.0) 1182 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]': finished 2024-02-13T03:28:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[19769]: audit 2024-02-13T03:28:15.229799+0000 mon.a (mon.0) 1183 : audit [INF] from='client.48643 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi161-61719-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[19769]: cluster 2024-02-13T03:28:15.230264+0000 mon.a (mon.0) 1184 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-13T03:28:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[19769]: audit 2024-02-13T03:28:15.234026+0000 mon.b (mon.1) 140 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[19769]: audit 2024-02-13T03:28:15.234135+0000 mon.a (mon.0) 1185 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[19769]: cluster 2024-02-13T03:28:15.234439+0000 mgr.y (mgr.24473) 302 : cluster [DBG] pgmap v285: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:16.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:16 smithi161 bash[19769]: audit 2024-02-13T03:28:15.574179+0000 mon.c (mon.2) 278 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:16 smithi183 bash[20824]: audit 2024-02-13T03:28:15.229574+0000 mon.a (mon.0) 1182 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61893-24"}]': finished 2024-02-13T03:28:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:16 smithi183 bash[20824]: audit 2024-02-13T03:28:15.229799+0000 mon.a (mon.0) 1183 : audit [INF] from='client.48643 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi161-61719-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:16 smithi183 bash[20824]: cluster 2024-02-13T03:28:15.230264+0000 mon.a (mon.0) 1184 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-13T03:28:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:16 smithi183 bash[20824]: audit 2024-02-13T03:28:15.234026+0000 mon.b (mon.1) 140 : audit [INF] from='client.? v1:172.21.15.161:0/1895257033' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:16 smithi183 bash[20824]: audit 2024-02-13T03:28:15.234135+0000 mon.a (mon.0) 1185 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]: dispatch 2024-02-13T03:28:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:16 smithi183 bash[20824]: cluster 2024-02-13T03:28:15.234439+0000 mgr.y (mgr.24473) 302 : cluster [DBG] pgmap v285: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:16 smithi183 bash[20824]: audit 2024-02-13T03:28:15.574179+0000 mon.c (mon.2) 278 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:17 smithi161 bash[27879]: audit 2024-02-13T03:28:15.857834+0000 mgr.y (mgr.24473) 303 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:17 smithi161 bash[27879]: audit 2024-02-13T03:28:16.247271+0000 mon.a (mon.0) 1186 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]': finished 2024-02-13T03:28:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:17 smithi161 bash[27879]: cluster 2024-02-13T03:28:16.250303+0000 mon.a (mon.0) 1187 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-13T03:28:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:17 smithi161 bash[27879]: audit 2024-02-13T03:28:16.255277+0000 mon.b (mon.1) 141 : audit [INF] from='client.? v1:172.21.15.161:0/1372004417' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi161-61726-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:17 smithi161 bash[27879]: audit 2024-02-13T03:28:16.262082+0000 mon.a (mon.0) 1188 : audit [INF] from='client.48646 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi161-61726-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:17 smithi161 bash[27879]: audit 2024-02-13T03:28:16.575628+0000 mon.c (mon.2) 279 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:17.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:17 smithi161 bash[19769]: audit 2024-02-13T03:28:15.857834+0000 mgr.y (mgr.24473) 303 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:17.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:17 smithi161 bash[19769]: audit 2024-02-13T03:28:16.247271+0000 mon.a (mon.0) 1186 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]': finished 2024-02-13T03:28:17.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:17 smithi161 bash[19769]: cluster 2024-02-13T03:28:16.250303+0000 mon.a (mon.0) 1187 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-13T03:28:17.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:17 smithi161 bash[19769]: audit 2024-02-13T03:28:16.255277+0000 mon.b (mon.1) 141 : audit [INF] from='client.? v1:172.21.15.161:0/1372004417' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi161-61726-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:17.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:17 smithi161 bash[19769]: audit 2024-02-13T03:28:16.262082+0000 mon.a (mon.0) 1188 : audit [INF] from='client.48646 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi161-61726-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:17.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:17 smithi161 bash[19769]: audit 2024-02-13T03:28:16.575628+0000 mon.c (mon.2) 279 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:17.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:17 smithi183 bash[20824]: audit 2024-02-13T03:28:15.857834+0000 mgr.y (mgr.24473) 303 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:17.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:17 smithi183 bash[20824]: audit 2024-02-13T03:28:16.247271+0000 mon.a (mon.0) 1186 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61893-24"}]': finished 2024-02-13T03:28:17.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:17 smithi183 bash[20824]: cluster 2024-02-13T03:28:16.250303+0000 mon.a (mon.0) 1187 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-13T03:28:17.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:17 smithi183 bash[20824]: audit 2024-02-13T03:28:16.255277+0000 mon.b (mon.1) 141 : audit [INF] from='client.? v1:172.21.15.161:0/1372004417' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi161-61726-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:17.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:17 smithi183 bash[20824]: audit 2024-02-13T03:28:16.262082+0000 mon.a (mon.0) 1188 : audit [INF] from='client.48646 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi161-61726-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:17.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:17 smithi183 bash[20824]: audit 2024-02-13T03:28:16.575628+0000 mon.c (mon.2) 279 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:18.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[19769]: cluster 2024-02-13T03:28:17.235402+0000 mgr.y (mgr.24473) 304 : cluster [DBG] pgmap v287: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 763 B/s wr, 2 op/s 2024-02-13T03:28:18.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[19769]: audit 2024-02-13T03:28:17.246781+0000 mon.a (mon.0) 1189 : audit [INF] from='client.48646 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi161-61726-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:18.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[19769]: cluster 2024-02-13T03:28:17.247019+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-13T03:28:18.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[19769]: audit 2024-02-13T03:28:17.255762+0000 mon.c (mon.2) 280 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:18.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[19769]: audit 2024-02-13T03:28:17.256169+0000 mon.b (mon.1) 142 : audit [INF] from='client.? v1:172.21.15.161:0/1167979601' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi161-61893-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:18.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[19769]: audit 2024-02-13T03:28:17.256932+0000 mon.a (mon.0) 1191 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:18.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[19769]: audit 2024-02-13T03:28:17.257486+0000 mon.a (mon.0) 1192 : audit [INF] from='client.48655 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi161-61893-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:18.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[19769]: audit 2024-02-13T03:28:17.271232+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? v1:172.21.15.161:0/3632609694' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi161-61719-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:18.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[19769]: audit 2024-02-13T03:28:17.577061+0000 mon.c (mon.2) 281 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:18.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[27879]: cluster 2024-02-13T03:28:17.235402+0000 mgr.y (mgr.24473) 304 : cluster [DBG] pgmap v287: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 763 B/s wr, 2 op/s 2024-02-13T03:28:18.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[27879]: audit 2024-02-13T03:28:17.246781+0000 mon.a (mon.0) 1189 : audit [INF] from='client.48646 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi161-61726-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:18.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[27879]: cluster 2024-02-13T03:28:17.247019+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-13T03:28:18.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[27879]: audit 2024-02-13T03:28:17.255762+0000 mon.c (mon.2) 280 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:18.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[27879]: audit 2024-02-13T03:28:17.256169+0000 mon.b (mon.1) 142 : audit [INF] from='client.? v1:172.21.15.161:0/1167979601' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi161-61893-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:18.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[27879]: audit 2024-02-13T03:28:17.256932+0000 mon.a (mon.0) 1191 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:18.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[27879]: audit 2024-02-13T03:28:17.257486+0000 mon.a (mon.0) 1192 : audit [INF] from='client.48655 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi161-61893-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:18.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[27879]: audit 2024-02-13T03:28:17.271232+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? v1:172.21.15.161:0/3632609694' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi161-61719-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:18.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:18 smithi161 bash[27879]: audit 2024-02-13T03:28:17.577061+0000 mon.c (mon.2) 281 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:18 smithi183 bash[20824]: cluster 2024-02-13T03:28:17.235402+0000 mgr.y (mgr.24473) 304 : cluster [DBG] pgmap v287: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 763 B/s wr, 2 op/s 2024-02-13T03:28:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:18 smithi183 bash[20824]: audit 2024-02-13T03:28:17.246781+0000 mon.a (mon.0) 1189 : audit [INF] from='client.48646 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi161-61726-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:18 smithi183 bash[20824]: cluster 2024-02-13T03:28:17.247019+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-13T03:28:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:18 smithi183 bash[20824]: audit 2024-02-13T03:28:17.255762+0000 mon.c (mon.2) 280 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:18 smithi183 bash[20824]: audit 2024-02-13T03:28:17.256169+0000 mon.b (mon.1) 142 : audit [INF] from='client.? v1:172.21.15.161:0/1167979601' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi161-61893-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:18 smithi183 bash[20824]: audit 2024-02-13T03:28:17.256932+0000 mon.a (mon.0) 1191 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:18 smithi183 bash[20824]: audit 2024-02-13T03:28:17.257486+0000 mon.a (mon.0) 1192 : audit [INF] from='client.48655 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi161-61893-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:18 smithi183 bash[20824]: audit 2024-02-13T03:28:17.271232+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? v1:172.21.15.161:0/3632609694' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi161-61719-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:18 smithi183 bash[20824]: audit 2024-02-13T03:28:17.577061+0000 mon.c (mon.2) 281 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:19.272 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP api_snapshots_pp: Running main() from gmock_main.cc 2024-02-13T03:28:19.272 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [==========] Running 20 tests from 5 test suites. 2024-02-13T03:28:19.272 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-02-13T03:28:19.272 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-02-13T03:28:19.272 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: seed 61993 2024-02-13T03:28:19.272 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-02-13T03:28:19.272 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (2496 ms) 2024-02-13T03:28:19.272 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-02-13T03:28:19.272 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (3713 ms) 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (3608 ms) 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2000 ms) 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (5054 ms) 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (16871 ms total) 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4305 ms) 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (4541 ms) 2024-02-13T03:28:19.273 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (5599 ms) 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4997 ms) 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2051 ms) 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi161-61993-7 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: waiting for snaps to purge 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (18413 ms) 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP (39913 ms total) 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (8 ms) 2024-02-13T03:28:19.274 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5111 ms) 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5120 ms total) 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3011 ms) 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (2014 ms) 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (1838 ms) 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2016 ms) 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8879 ms total) 2024-02-13T03:28:19.275 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: 2024-02-13T03:28:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[27879]: audit 2024-02-13T03:28:18.252203+0000 mon.a (mon.0) 1194 : audit [INF] from='client.47933 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]': finished 2024-02-13T03:28:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[27879]: audit 2024-02-13T03:28:18.252525+0000 mon.a (mon.0) 1195 : audit [INF] from='client.48655 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi161-61893-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[27879]: audit 2024-02-13T03:28:18.253407+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? v1:172.21.15.161:0/3632609694' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi161-61719-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[27879]: cluster 2024-02-13T03:28:18.254596+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-13T03:28:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[27879]: audit 2024-02-13T03:28:18.256264+0000 mon.c (mon.2) 282 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[27879]: audit 2024-02-13T03:28:18.265100+0000 mon.a (mon.0) 1198 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[27879]: audit 2024-02-13T03:28:18.578589+0000 mon.c (mon.2) 283 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:19.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[27879]: audit 2024-02-13T03:28:19.257663+0000 mon.a (mon.0) 1199 : audit [INF] from='client.47933 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]': finished 2024-02-13T03:28:19.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[27879]: cluster 2024-02-13T03:28:19.257892+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-13T03:28:19.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[19769]: audit 2024-02-13T03:28:18.252203+0000 mon.a (mon.0) 1194 : audit [INF] from='client.47933 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]': finished 2024-02-13T03:28:19.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[19769]: audit 2024-02-13T03:28:18.252525+0000 mon.a (mon.0) 1195 : audit [INF] from='client.48655 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi161-61893-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:19.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[19769]: audit 2024-02-13T03:28:18.253407+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? v1:172.21.15.161:0/3632609694' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi161-61719-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:19.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[19769]: cluster 2024-02-13T03:28:18.254596+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-13T03:28:19.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[19769]: audit 2024-02-13T03:28:18.256264+0000 mon.c (mon.2) 282 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:19.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[19769]: audit 2024-02-13T03:28:18.265100+0000 mon.a (mon.0) 1198 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:19.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[19769]: audit 2024-02-13T03:28:18.578589+0000 mon.c (mon.2) 283 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:19.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[19769]: audit 2024-02-13T03:28:19.257663+0000 mon.a (mon.0) 1199 : audit [INF] from='client.47933 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]': finished 2024-02-13T03:28:19.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:19 smithi161 bash[19769]: cluster 2024-02-13T03:28:19.257892+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-13T03:28:19.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:28:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:28:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:28:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:19 smithi183 bash[20824]: audit 2024-02-13T03:28:18.252203+0000 mon.a (mon.0) 1194 : audit [INF] from='client.47933 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi161-61993-15"}]': finished 2024-02-13T03:28:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:19 smithi183 bash[20824]: audit 2024-02-13T03:28:18.252525+0000 mon.a (mon.0) 1195 : audit [INF] from='client.48655 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi161-61893-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:19 smithi183 bash[20824]: audit 2024-02-13T03:28:18.253407+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? v1:172.21.15.161:0/3632609694' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi161-61719-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:19 smithi183 bash[20824]: cluster 2024-02-13T03:28:18.254596+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-13T03:28:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:19 smithi183 bash[20824]: audit 2024-02-13T03:28:18.256264+0000 mon.c (mon.2) 282 : audit [INF] from='client.? v1:172.21.15.161:0/4073669901' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:19 smithi183 bash[20824]: audit 2024-02-13T03:28:18.265100+0000 mon.a (mon.0) 1198 : audit [INF] from='client.47933 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]: dispatch 2024-02-13T03:28:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:19 smithi183 bash[20824]: audit 2024-02-13T03:28:18.578589+0000 mon.c (mon.2) 283 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:19 smithi183 bash[20824]: audit 2024-02-13T03:28:19.257663+0000 mon.a (mon.0) 1199 : audit [INF] from='client.47933 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi161-61993-15"}]': finished 2024-02-13T03:28:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:19 smithi183 bash[20824]: cluster 2024-02-13T03:28:19.257892+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-13T03:28:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: cluster 2024-02-13T03:28:19.236475+0000 mgr.y (mgr.24473) 305 : cluster [DBG] pgmap v290: 324 pgs: 64 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: cluster 2024-02-13T03:28:19.269225+0000 mon.a (mon.0) 1201 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.272327+0000 mon.b (mon.1) 143 : audit [INF] from='client.? v1:172.21.15.161:0/1544979977' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61726-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.272732+0000 mon.a (mon.0) 1202 : audit [INF] from='client.48664 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61726-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.306102+0000 mon.a (mon.0) 1203 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.307011+0000 mon.a (mon.0) 1204 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.308300+0000 mon.a (mon.0) 1205 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.308785+0000 mon.a (mon.0) 1206 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.308948+0000 mon.b (mon.1) 144 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.309112+0000 mon.a (mon.0) 1207 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi161-61719-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.309649+0000 mon.b (mon.1) 145 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.311163+0000 mon.b (mon.1) 146 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.311513+0000 mon.b (mon.1) 147 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.312030+0000 mon.b (mon.1) 148 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi161-61719-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:20.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: cluster 2024-02-13T03:28:19.236475+0000 mgr.y (mgr.24473) 305 : cluster [DBG] pgmap v290: 324 pgs: 64 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:20.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: cluster 2024-02-13T03:28:19.269225+0000 mon.a (mon.0) 1201 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:20.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.272327+0000 mon.b (mon.1) 143 : audit [INF] from='client.? v1:172.21.15.161:0/1544979977' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61726-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:20.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.272732+0000 mon.a (mon.0) 1202 : audit [INF] from='client.48664 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61726-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:20.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.306102+0000 mon.a (mon.0) 1203 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.307011+0000 mon.a (mon.0) 1204 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.308300+0000 mon.a (mon.0) 1205 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.308785+0000 mon.a (mon.0) 1206 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.308948+0000 mon.b (mon.1) 144 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.309112+0000 mon.a (mon.0) 1207 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi161-61719-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:20.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.309649+0000 mon.b (mon.1) 145 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.311163+0000 mon.b (mon.1) 146 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.311513+0000 mon.b (mon.1) 147 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.312030+0000 mon.b (mon.1) 148 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi161-61719-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:20.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.312694+0000 mon.b (mon.1) 149 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:20.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.579877+0000 mon.c (mon.2) 284 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:20.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.740133+0000 mon.a (mon.0) 1208 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:20.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:19.779910+0000 mon.a (mon.0) 1209 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:28:20.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:20.261238+0000 mon.a (mon.0) 1210 : audit [INF] from='client.48664 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61726-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:20.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:20.261634+0000 mon.a (mon.0) 1211 : audit [INF] from='client.48679 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi161-61719-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:20.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:20.261877+0000 mon.a (mon.0) 1212 : audit [INF] from='client.48676 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:20.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: cluster 2024-02-13T03:28:20.262003+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-13T03:28:20.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:20.267888+0000 mon.b (mon.1) 150 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi161-61719-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:20.268243+0000 mon.a (mon.0) 1214 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi161-61719-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:20.268288+0000 mon.b (mon.1) 151 : audit [INF] from='client.? v1:172.21.15.161:0/914591955' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi161-61893-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:20.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:20.268830+0000 mon.b (mon.1) 152 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:20.268962+0000 mon.a (mon.0) 1215 : audit [INF] from='client.48682 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi161-61893-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:20.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[27879]: audit 2024-02-13T03:28:20.269388+0000 mon.a (mon.0) 1216 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.312694+0000 mon.b (mon.1) 149 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:20.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.579877+0000 mon.c (mon.2) 284 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:20.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.740133+0000 mon.a (mon.0) 1208 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:20.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:19.779910+0000 mon.a (mon.0) 1209 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:28:20.681 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:20.261238+0000 mon.a (mon.0) 1210 : audit [INF] from='client.48664 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61726-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:20.682 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:20.261634+0000 mon.a (mon.0) 1211 : audit [INF] from='client.48679 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi161-61719-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:20.682 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:20.261877+0000 mon.a (mon.0) 1212 : audit [INF] from='client.48676 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:20.682 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: cluster 2024-02-13T03:28:20.262003+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-13T03:28:20.682 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:20.267888+0000 mon.b (mon.1) 150 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi161-61719-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.682 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:20.268243+0000 mon.a (mon.0) 1214 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi161-61719-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.682 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:20.268288+0000 mon.b (mon.1) 151 : audit [INF] from='client.? v1:172.21.15.161:0/914591955' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi161-61893-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:20.682 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:20.268830+0000 mon.b (mon.1) 152 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.682 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:20.268962+0000 mon.a (mon.0) 1215 : audit [INF] from='client.48682 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi161-61893-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:20.682 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:20 smithi161 bash[19769]: audit 2024-02-13T03:28:20.269388+0000 mon.a (mon.0) 1216 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: cluster 2024-02-13T03:28:19.236475+0000 mgr.y (mgr.24473) 305 : cluster [DBG] pgmap v290: 324 pgs: 64 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: cluster 2024-02-13T03:28:19.269225+0000 mon.a (mon.0) 1201 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.272327+0000 mon.b (mon.1) 143 : audit [INF] from='client.? v1:172.21.15.161:0/1544979977' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61726-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.272732+0000 mon.a (mon.0) 1202 : audit [INF] from='client.48664 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61726-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.306102+0000 mon.a (mon.0) 1203 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.307011+0000 mon.a (mon.0) 1204 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.308300+0000 mon.a (mon.0) 1205 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.308785+0000 mon.a (mon.0) 1206 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.308948+0000 mon.b (mon.1) 144 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.309112+0000 mon.a (mon.0) 1207 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi161-61719-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.309649+0000 mon.b (mon.1) 145 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.311163+0000 mon.b (mon.1) 146 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.311513+0000 mon.b (mon.1) 147 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.312030+0000 mon.b (mon.1) 148 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi161-61719-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.312694+0000 mon.b (mon.1) 149 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.579877+0000 mon.c (mon.2) 284 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.740133+0000 mon.a (mon.0) 1208 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:19.779910+0000 mon.a (mon.0) 1209 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:20.261238+0000 mon.a (mon.0) 1210 : audit [INF] from='client.48664 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi161-61726-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:20.261634+0000 mon.a (mon.0) 1211 : audit [INF] from='client.48679 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi161-61719-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:20.261877+0000 mon.a (mon.0) 1212 : audit [INF] from='client.48676 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:20.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: cluster 2024-02-13T03:28:20.262003+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-13T03:28:20.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:20.267888+0000 mon.b (mon.1) 150 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi161-61719-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:20.268243+0000 mon.a (mon.0) 1214 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi161-61719-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:20.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:20.268288+0000 mon.b (mon.1) 151 : audit [INF] from='client.? v1:172.21.15.161:0/914591955' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi161-61893-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:20.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:20.268830+0000 mon.b (mon.1) 152 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:20.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:20.268962+0000 mon.a (mon.0) 1215 : audit [INF] from='client.48682 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi161-61893-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:20.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:20 smithi183 bash[20824]: audit 2024-02-13T03:28:20.269388+0000 mon.a (mon.0) 1216 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:21.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:28:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:28:21] "GET /metrics HTTP/1.1" 200 188963 "" "Prometheus/2.33.4" 2024-02-13T03:28:22.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:22 smithi161 bash[27879]: audit 2024-02-13T03:28:20.581388+0000 mon.c (mon.2) 285 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:22.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:22 smithi161 bash[19769]: audit 2024-02-13T03:28:20.581388+0000 mon.c (mon.2) 285 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:22.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:22 smithi183 bash[20824]: audit 2024-02-13T03:28:20.581388+0000 mon.c (mon.2) 285 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[19769]: cluster 2024-02-13T03:28:21.239277+0000 mgr.y (mgr.24473) 306 : cluster [DBG] pgmap v293: 324 pgs: 32 creating+peering, 292 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-13T03:28:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[19769]: audit 2024-02-13T03:28:21.583076+0000 mon.c (mon.2) 286 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[19769]: audit 2024-02-13T03:28:21.695995+0000 mon.a (mon.0) 1217 : audit [INF] from='client.48682 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi161-61893-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[19769]: cluster 2024-02-13T03:28:22.101171+0000 mon.a (mon.0) 1218 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-13T03:28:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[19769]: audit 2024-02-13T03:28:22.572662+0000 mon.a (mon.0) 1219 : audit [INF] from='client.48679 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi161-61719-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi161-61719-26"}]': finished 2024-02-13T03:28:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[19769]: audit 2024-02-13T03:28:22.572935+0000 mon.a (mon.0) 1220 : audit [INF] from='client.48676 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]': finished 2024-02-13T03:28:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[19769]: cluster 2024-02-13T03:28:22.573062+0000 mon.a (mon.0) 1221 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-02-13T03:28:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[19769]: audit 2024-02-13T03:28:22.577025+0000 mon.c (mon.2) 287 : audit [INF] from='client.? v1:172.21.15.161:0/1003991303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi161-61726-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[19769]: audit 2024-02-13T03:28:22.584266+0000 mon.c (mon.2) 288 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[19769]: audit 2024-02-13T03:28:22.587394+0000 mon.a (mon.0) 1222 : audit [INF] from='client.48023 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi161-61726-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[27879]: cluster 2024-02-13T03:28:21.239277+0000 mgr.y (mgr.24473) 306 : cluster [DBG] pgmap v293: 324 pgs: 32 creating+peering, 292 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-13T03:28:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[27879]: audit 2024-02-13T03:28:21.583076+0000 mon.c (mon.2) 286 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[27879]: audit 2024-02-13T03:28:21.695995+0000 mon.a (mon.0) 1217 : audit [INF] from='client.48682 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi161-61893-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[27879]: cluster 2024-02-13T03:28:22.101171+0000 mon.a (mon.0) 1218 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-13T03:28:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[27879]: audit 2024-02-13T03:28:22.572662+0000 mon.a (mon.0) 1219 : audit [INF] from='client.48679 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi161-61719-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi161-61719-26"}]': finished 2024-02-13T03:28:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[27879]: audit 2024-02-13T03:28:22.572935+0000 mon.a (mon.0) 1220 : audit [INF] from='client.48676 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]': finished 2024-02-13T03:28:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[27879]: cluster 2024-02-13T03:28:22.573062+0000 mon.a (mon.0) 1221 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-02-13T03:28:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[27879]: audit 2024-02-13T03:28:22.577025+0000 mon.c (mon.2) 287 : audit [INF] from='client.? v1:172.21.15.161:0/1003991303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi161-61726-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[27879]: audit 2024-02-13T03:28:22.584266+0000 mon.c (mon.2) 288 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:23.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:23 smithi161 bash[27879]: audit 2024-02-13T03:28:22.587394+0000 mon.a (mon.0) 1222 : audit [INF] from='client.48023 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi161-61726-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:23 smithi183 bash[20824]: cluster 2024-02-13T03:28:21.239277+0000 mgr.y (mgr.24473) 306 : cluster [DBG] pgmap v293: 324 pgs: 32 creating+peering, 292 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-13T03:28:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:23 smithi183 bash[20824]: audit 2024-02-13T03:28:21.583076+0000 mon.c (mon.2) 286 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:23 smithi183 bash[20824]: audit 2024-02-13T03:28:21.695995+0000 mon.a (mon.0) 1217 : audit [INF] from='client.48682 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi161-61893-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:23 smithi183 bash[20824]: cluster 2024-02-13T03:28:22.101171+0000 mon.a (mon.0) 1218 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-13T03:28:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:23 smithi183 bash[20824]: audit 2024-02-13T03:28:22.572662+0000 mon.a (mon.0) 1219 : audit [INF] from='client.48679 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi161-61719-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi161-61719-26"}]': finished 2024-02-13T03:28:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:23 smithi183 bash[20824]: audit 2024-02-13T03:28:22.572935+0000 mon.a (mon.0) 1220 : audit [INF] from='client.48676 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]': finished 2024-02-13T03:28:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:23 smithi183 bash[20824]: cluster 2024-02-13T03:28:22.573062+0000 mon.a (mon.0) 1221 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-02-13T03:28:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:23 smithi183 bash[20824]: audit 2024-02-13T03:28:22.577025+0000 mon.c (mon.2) 287 : audit [INF] from='client.? v1:172.21.15.161:0/1003991303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi161-61726-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:23 smithi183 bash[20824]: audit 2024-02-13T03:28:22.584266+0000 mon.c (mon.2) 288 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:23 smithi183 bash[20824]: audit 2024-02-13T03:28:22.587394+0000 mon.a (mon.0) 1222 : audit [INF] from='client.48023 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi161-61726-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:24 smithi183 bash[20824]: cluster 2024-02-13T03:28:23.240258+0000 mgr.y (mgr.24473) 307 : cluster [DBG] pgmap v296: 308 pgs: 48 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:24 smithi183 bash[20824]: audit 2024-02-13T03:28:23.577106+0000 mon.a (mon.0) 1223 : audit [INF] from='client.48023 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi161-61726-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:24 smithi183 bash[20824]: cluster 2024-02-13T03:28:23.578397+0000 mon.a (mon.0) 1224 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-02-13T03:28:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:24 smithi183 bash[20824]: audit 2024-02-13T03:28:23.586720+0000 mon.c (mon.2) 289 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:24 smithi183 bash[20824]: audit 2024-02-13T03:28:23.588222+0000 mon.a (mon.0) 1225 : audit [INF] from='client.? v1:172.21.15.161:0/3097848257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi161-61893-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:24 smithi161 bash[27879]: cluster 2024-02-13T03:28:23.240258+0000 mgr.y (mgr.24473) 307 : cluster [DBG] pgmap v296: 308 pgs: 48 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:24 smithi161 bash[27879]: audit 2024-02-13T03:28:23.577106+0000 mon.a (mon.0) 1223 : audit [INF] from='client.48023 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi161-61726-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:24 smithi161 bash[27879]: cluster 2024-02-13T03:28:23.578397+0000 mon.a (mon.0) 1224 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-02-13T03:28:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:24 smithi161 bash[27879]: audit 2024-02-13T03:28:23.586720+0000 mon.c (mon.2) 289 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:24 smithi161 bash[27879]: audit 2024-02-13T03:28:23.588222+0000 mon.a (mon.0) 1225 : audit [INF] from='client.? v1:172.21.15.161:0/3097848257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi161-61893-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:24 smithi161 bash[19769]: cluster 2024-02-13T03:28:23.240258+0000 mgr.y (mgr.24473) 307 : cluster [DBG] pgmap v296: 308 pgs: 48 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:24 smithi161 bash[19769]: audit 2024-02-13T03:28:23.577106+0000 mon.a (mon.0) 1223 : audit [INF] from='client.48023 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi161-61726-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:24 smithi161 bash[19769]: cluster 2024-02-13T03:28:23.578397+0000 mon.a (mon.0) 1224 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-02-13T03:28:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:24 smithi161 bash[19769]: audit 2024-02-13T03:28:23.586720+0000 mon.c (mon.2) 289 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:24 smithi161 bash[19769]: audit 2024-02-13T03:28:23.588222+0000 mon.a (mon.0) 1225 : audit [INF] from='client.? v1:172.21.15.161:0/3097848257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi161-61893-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:25.872 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:25 smithi183 bash[20824]: audit 2024-02-13T03:28:24.581348+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? v1:172.21.15.161:0/3097848257' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi161-61893-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:25.872 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:25 smithi183 bash[20824]: cluster 2024-02-13T03:28:24.583253+0000 mon.a (mon.0) 1227 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-02-13T03:28:25.872 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:25 smithi183 bash[20824]: audit 2024-02-13T03:28:24.593619+0000 mon.c (mon.2) 290 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:25.872 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:25 smithi183 bash[20824]: audit 2024-02-13T03:28:24.596840+0000 mon.a (mon.0) 1228 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:25.872 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:25 smithi183 bash[20824]: audit 2024-02-13T03:28:24.599529+0000 mon.b (mon.1) 153 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:25.872 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:25 smithi183 bash[20824]: cluster 2024-02-13T03:28:25.241158+0000 mgr.y (mgr.24473) 308 : cluster [DBG] pgmap v299: 300 pgs: 40 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:25 smithi161 bash[27879]: audit 2024-02-13T03:28:24.581348+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? v1:172.21.15.161:0/3097848257' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi161-61893-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:25 smithi161 bash[27879]: cluster 2024-02-13T03:28:24.583253+0000 mon.a (mon.0) 1227 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-02-13T03:28:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:25 smithi161 bash[27879]: audit 2024-02-13T03:28:24.593619+0000 mon.c (mon.2) 290 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:25 smithi161 bash[27879]: audit 2024-02-13T03:28:24.596840+0000 mon.a (mon.0) 1228 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:25 smithi161 bash[27879]: audit 2024-02-13T03:28:24.599529+0000 mon.b (mon.1) 153 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:25 smithi161 bash[27879]: cluster 2024-02-13T03:28:25.241158+0000 mgr.y (mgr.24473) 308 : cluster [DBG] pgmap v299: 300 pgs: 40 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:25 smithi161 bash[19769]: audit 2024-02-13T03:28:24.581348+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? v1:172.21.15.161:0/3097848257' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi161-61893-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:25 smithi161 bash[19769]: cluster 2024-02-13T03:28:24.583253+0000 mon.a (mon.0) 1227 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-02-13T03:28:25.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:25 smithi161 bash[19769]: audit 2024-02-13T03:28:24.593619+0000 mon.c (mon.2) 290 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:25.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:25 smithi161 bash[19769]: audit 2024-02-13T03:28:24.596840+0000 mon.a (mon.0) 1228 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:25.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:25 smithi161 bash[19769]: audit 2024-02-13T03:28:24.599529+0000 mon.b (mon.1) 153 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:25.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:25 smithi161 bash[19769]: cluster 2024-02-13T03:28:25.241158+0000 mgr.y (mgr.24473) 308 : cluster [DBG] pgmap v299: 300 pgs: 40 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:26.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:28:25 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:28:26.594 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp api_aio: Running main() from gmock_main.cc 2024-02-13T03:28:26.594 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [==========] Running 41 tests from 2 test suites. 2024-02-13T03:28:26.594 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [----------] Global test environment set-up. 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [----------] 25 tests from LibRadosAio 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.TooBig (12953 ms) 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (4720 ms) 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (4603 ms) 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3511 ms) 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3542 ms) 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (4249 ms) 2024-02-13T03:28:26.595 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (4241 ms) 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3574 ms) 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (2699 ms) 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (2911 ms) 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (4098 ms) 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (2902 ms) 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3453 ms) 2024-02-13T03:28:26.596 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2920 ms) 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.Flush (2645 ms) 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3014 ms) 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3012 ms) 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3022 ms) 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3019 ms) 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3058 ms) 2024-02-13T03:28:26.597 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3020 ms) 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3011 ms) 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (2837 ms) 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3038 ms) 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3009 ms) 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [----------] 25 tests from LibRadosAio (93061 ms total) 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-02-13T03:28:26.598 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7323 ms) 2024-02-13T03:28:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:25.584607+0000 mon.a (mon.0) 1229 : audit [INF] from='client.48679 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]': finished 2024-02-13T03:28:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: cluster 2024-02-13T03:28:25.584807+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-02-13T03:28:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:25.590309+0000 mon.b (mon.1) 154 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:25.591815+0000 mon.a (mon.0) 1231 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:25.592824+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? v1:172.21.15.161:0/809189383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi161-61726-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:25.594724+0000 mon.c (mon.2) 291 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:25.629243+0000 mon.c (mon.2) 292 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:25.630019+0000 mon.a (mon.0) 1233 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:25.632797+0000 mon.c (mon.2) 293 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:25.633325+0000 mon.a (mon.0) 1234 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:25.635383+0000 mon.c (mon.2) 294 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:25.636140+0000 mon.a (mon.0) 1235 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:25.869147+0000 mgr.y (mgr.24473) 309 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:26.586715+0000 mon.a (mon.0) 1236 : audit [INF] from='client.48679 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]': finished 2024-02-13T03:28:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:26.586972+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? v1:172.21.15.161:0/809189383' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi161-61726-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:26.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:25.584607+0000 mon.a (mon.0) 1229 : audit [INF] from='client.48679 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]': finished 2024-02-13T03:28:26.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: cluster 2024-02-13T03:28:25.584807+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-02-13T03:28:26.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:25.590309+0000 mon.b (mon.1) 154 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:26.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:25.591815+0000 mon.a (mon.0) 1231 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:26.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:25.592824+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? v1:172.21.15.161:0/809189383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi161-61726-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:26.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:25.594724+0000 mon.c (mon.2) 291 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:26.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:25.629243+0000 mon.c (mon.2) 292 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:25.630019+0000 mon.a (mon.0) 1233 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:25.632797+0000 mon.c (mon.2) 293 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:25.633325+0000 mon.a (mon.0) 1234 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:25.635383+0000 mon.c (mon.2) 294 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:26.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:25.636140+0000 mon.a (mon.0) 1235 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:26.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:25.869147+0000 mgr.y (mgr.24473) 309 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:26.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:26.586715+0000 mon.a (mon.0) 1236 : audit [INF] from='client.48679 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]': finished 2024-02-13T03:28:26.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:26.586972+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? v1:172.21.15.161:0/809189383' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi161-61726-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:26.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:26.587427+0000 mon.a (mon.0) 1238 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:26.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: cluster 2024-02-13T03:28:26.587712+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-02-13T03:28:26.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:26.590603+0000 mon.c (mon.2) 295 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi161-61893-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[27879]: audit 2024-02-13T03:28:26.594163+0000 mon.a (mon.0) 1240 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi161-61893-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:26.587427+0000 mon.a (mon.0) 1238 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:26.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: cluster 2024-02-13T03:28:26.587712+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-02-13T03:28:26.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:26.590603+0000 mon.c (mon.2) 295 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi161-61893-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.931 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:26 smithi161 bash[19769]: audit 2024-02-13T03:28:26.594163+0000 mon.a (mon.0) 1240 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi161-61893-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:25.584607+0000 mon.a (mon.0) 1229 : audit [INF] from='client.48679 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi161-61719-26"}]': finished 2024-02-13T03:28:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: cluster 2024-02-13T03:28:25.584807+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-02-13T03:28:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:25.590309+0000 mon.b (mon.1) 154 : audit [INF] from='client.? v1:172.21.15.161:0/1205566129' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:25.591815+0000 mon.a (mon.0) 1231 : audit [INF] from='client.48679 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]: dispatch 2024-02-13T03:28:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:25.592824+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? v1:172.21.15.161:0/809189383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi161-61726-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:25.594724+0000 mon.c (mon.2) 291 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:25.629243+0000 mon.c (mon.2) 292 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:25.630019+0000 mon.a (mon.0) 1233 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:25.632797+0000 mon.c (mon.2) 293 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:25.633325+0000 mon.a (mon.0) 1234 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:25.635383+0000 mon.c (mon.2) 294 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:25.636140+0000 mon.a (mon.0) 1235 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:25.869147+0000 mgr.y (mgr.24473) 309 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:26.586715+0000 mon.a (mon.0) 1236 : audit [INF] from='client.48679 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi161-61719-26"}]': finished 2024-02-13T03:28:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:26.586972+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? v1:172.21.15.161:0/809189383' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi161-61726-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:26.587427+0000 mon.a (mon.0) 1238 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: cluster 2024-02-13T03:28:26.587712+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-02-13T03:28:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:26.590603+0000 mon.c (mon.2) 295 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi161-61893-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:26.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:26 smithi183 bash[20824]: audit 2024-02-13T03:28:26.594163+0000 mon.a (mon.0) 1240 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi161-61893-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:28.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[19769]: audit 2024-02-13T03:28:26.600529+0000 mon.c (mon.2) 296 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:28.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[19769]: audit 2024-02-13T03:28:26.639308+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:28.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[19769]: audit 2024-02-13T03:28:26.642132+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:28.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[19769]: audit 2024-02-13T03:28:26.642936+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi161-61719-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:28.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[19769]: audit 2024-02-13T03:28:26.772861+0000 mon.a (mon.0) 1244 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:28.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[19769]: audit 2024-02-13T03:28:26.775721+0000 mon.b (mon.1) 155 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:28.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[19769]: cluster 2024-02-13T03:28:27.242152+0000 mgr.y (mgr.24473) 310 : cluster [DBG] pgmap v302: 300 pgs: 29 creating+peering, 14 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 3 unknown, 214 active+clean; 28 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-13T03:28:28.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[27879]: audit 2024-02-13T03:28:26.600529+0000 mon.c (mon.2) 296 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:28.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[27879]: audit 2024-02-13T03:28:26.639308+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:28.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[27879]: audit 2024-02-13T03:28:26.642132+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:28.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[27879]: audit 2024-02-13T03:28:26.642936+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi161-61719-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:28.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[27879]: audit 2024-02-13T03:28:26.772861+0000 mon.a (mon.0) 1244 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:28.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[27879]: audit 2024-02-13T03:28:26.775721+0000 mon.b (mon.1) 155 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:28.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:28 smithi161 bash[27879]: cluster 2024-02-13T03:28:27.242152+0000 mgr.y (mgr.24473) 310 : cluster [DBG] pgmap v302: 300 pgs: 29 creating+peering, 14 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 3 unknown, 214 active+clean; 28 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-13T03:28:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:28 smithi183 bash[20824]: audit 2024-02-13T03:28:26.600529+0000 mon.c (mon.2) 296 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:28 smithi183 bash[20824]: audit 2024-02-13T03:28:26.639308+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:28 smithi183 bash[20824]: audit 2024-02-13T03:28:26.642132+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:28 smithi183 bash[20824]: audit 2024-02-13T03:28:26.642936+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi161-61719-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:28 smithi183 bash[20824]: audit 2024-02-13T03:28:26.772861+0000 mon.a (mon.0) 1244 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:28 smithi183 bash[20824]: audit 2024-02-13T03:28:26.775721+0000 mon.b (mon.1) 155 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:28 smithi183 bash[20824]: cluster 2024-02-13T03:28:27.242152+0000 mgr.y (mgr.24473) 310 : cluster [DBG] pgmap v302: 300 pgs: 29 creating+peering, 14 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 3 unknown, 214 active+clean; 28 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-13T03:28:29.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: audit 2024-02-13T03:28:27.601818+0000 mon.c (mon.2) 297 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:29.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: audit 2024-02-13T03:28:28.015586+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi161-61719-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:29.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: audit 2024-02-13T03:28:28.015899+0000 mon.a (mon.0) 1246 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:28:29.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: cluster 2024-02-13T03:28:28.016114+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-02-13T03:28:29.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: cluster 2024-02-13T03:28:28.017859+0000 mon.a (mon.0) 1248 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:29.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: audit 2024-02-13T03:28:28.018509+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi161-61719-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:29.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: audit 2024-02-13T03:28:28.018955+0000 mon.a (mon.0) 1250 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13"}]: dispatch 2024-02-13T03:28:29.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: audit 2024-02-13T03:28:28.020625+0000 mon.b (mon.1) 156 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13"}]: dispatch 2024-02-13T03:28:29.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: cluster 2024-02-13T03:28:28.476331+0000 mon.a (mon.0) 1251 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:28:29.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: audit 2024-02-13T03:28:28.484092+0000 mon.a (mon.0) 1252 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi161-61893-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi161-61893-36"}]': finished 2024-02-13T03:28:29.332 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: audit 2024-02-13T03:28:28.603166+0000 mon.c (mon.2) 298 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:29.332 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: audit 2024-02-13T03:28:28.903626+0000 mon.a (mon.0) 1253 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13"}]': finished 2024-02-13T03:28:29.332 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: cluster 2024-02-13T03:28:28.904047+0000 mon.a (mon.0) 1254 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-02-13T03:28:29.332 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: audit 2024-02-13T03:28:28.908446+0000 mon.c (mon.2) 299 : audit [INF] from='client.? v1:172.21.15.161:0/3174965280' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi161-61726-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:29.332 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[20824]: audit 2024-02-13T03:28:28.916491+0000 mon.a (mon.0) 1255 : audit [INF] from='client.48053 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi161-61726-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:29.332 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:28:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:28:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:28:29.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: audit 2024-02-13T03:28:27.601818+0000 mon.c (mon.2) 297 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:29.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: audit 2024-02-13T03:28:28.015586+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi161-61719-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:29.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: audit 2024-02-13T03:28:28.015899+0000 mon.a (mon.0) 1246 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:28:29.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: cluster 2024-02-13T03:28:28.016114+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-02-13T03:28:29.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: cluster 2024-02-13T03:28:28.017859+0000 mon.a (mon.0) 1248 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:29.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: audit 2024-02-13T03:28:28.018509+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi161-61719-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:29.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: audit 2024-02-13T03:28:28.018955+0000 mon.a (mon.0) 1250 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13"}]: dispatch 2024-02-13T03:28:29.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: audit 2024-02-13T03:28:28.020625+0000 mon.b (mon.1) 156 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13"}]: dispatch 2024-02-13T03:28:29.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: cluster 2024-02-13T03:28:28.476331+0000 mon.a (mon.0) 1251 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:28:29.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: audit 2024-02-13T03:28:28.484092+0000 mon.a (mon.0) 1252 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi161-61893-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi161-61893-36"}]': finished 2024-02-13T03:28:29.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: audit 2024-02-13T03:28:28.603166+0000 mon.c (mon.2) 298 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:29.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: audit 2024-02-13T03:28:28.903626+0000 mon.a (mon.0) 1253 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13"}]': finished 2024-02-13T03:28:29.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: cluster 2024-02-13T03:28:28.904047+0000 mon.a (mon.0) 1254 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-02-13T03:28:29.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: audit 2024-02-13T03:28:28.908446+0000 mon.c (mon.2) 299 : audit [INF] from='client.? v1:172.21.15.161:0/3174965280' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi161-61726-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:29.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[19769]: audit 2024-02-13T03:28:28.916491+0000 mon.a (mon.0) 1255 : audit [INF] from='client.48053 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi161-61726-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:29.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: audit 2024-02-13T03:28:27.601818+0000 mon.c (mon.2) 297 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:29.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: audit 2024-02-13T03:28:28.015586+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi161-61719-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:29.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: audit 2024-02-13T03:28:28.015899+0000 mon.a (mon.0) 1246 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:28:29.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: cluster 2024-02-13T03:28:28.016114+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-02-13T03:28:29.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: cluster 2024-02-13T03:28:28.017859+0000 mon.a (mon.0) 1248 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:29.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: audit 2024-02-13T03:28:28.018509+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi161-61719-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:29.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: audit 2024-02-13T03:28:28.018955+0000 mon.a (mon.0) 1250 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13"}]: dispatch 2024-02-13T03:28:29.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: audit 2024-02-13T03:28:28.020625+0000 mon.b (mon.1) 156 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13"}]: dispatch 2024-02-13T03:28:29.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: cluster 2024-02-13T03:28:28.476331+0000 mon.a (mon.0) 1251 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:28:29.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: audit 2024-02-13T03:28:28.484092+0000 mon.a (mon.0) 1252 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi161-61893-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi161-61893-36"}]': finished 2024-02-13T03:28:29.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: audit 2024-02-13T03:28:28.603166+0000 mon.c (mon.2) 298 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:29.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: audit 2024-02-13T03:28:28.903626+0000 mon.a (mon.0) 1253 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-13"}]': finished 2024-02-13T03:28:29.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: cluster 2024-02-13T03:28:28.904047+0000 mon.a (mon.0) 1254 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-02-13T03:28:29.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: audit 2024-02-13T03:28:28.908446+0000 mon.c (mon.2) 299 : audit [INF] from='client.? v1:172.21.15.161:0/3174965280' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi161-61726-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:29.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:29 smithi161 bash[27879]: audit 2024-02-13T03:28:28.916491+0000 mon.a (mon.0) 1255 : audit [INF] from='client.48053 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi161-61726-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:30 smithi161 bash[27879]: cluster 2024-02-13T03:28:29.243087+0000 mgr.y (mgr.24473) 311 : cluster [DBG] pgmap v305: 308 pgs: 14 active+clean+snaptrim, 40 unknown, 40 active+clean+snaptrim_wait, 214 active+clean; 28 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-13T03:28:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:30 smithi161 bash[27879]: audit 2024-02-13T03:28:29.491023+0000 mon.a (mon.0) 1256 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi161-61719-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi161-61719-27"}]': finished 2024-02-13T03:28:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:30 smithi161 bash[27879]: audit 2024-02-13T03:28:29.491377+0000 mon.a (mon.0) 1257 : audit [INF] from='client.48053 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi161-61726-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:30 smithi161 bash[27879]: cluster 2024-02-13T03:28:29.491484+0000 mon.a (mon.0) 1258 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-02-13T03:28:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:30 smithi161 bash[27879]: audit 2024-02-13T03:28:29.604536+0000 mon.c (mon.2) 300 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:30.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:30 smithi161 bash[19769]: cluster 2024-02-13T03:28:29.243087+0000 mgr.y (mgr.24473) 311 : cluster [DBG] pgmap v305: 308 pgs: 14 active+clean+snaptrim, 40 unknown, 40 active+clean+snaptrim_wait, 214 active+clean; 28 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-13T03:28:30.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:30 smithi161 bash[19769]: audit 2024-02-13T03:28:29.491023+0000 mon.a (mon.0) 1256 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi161-61719-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi161-61719-27"}]': finished 2024-02-13T03:28:30.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:30 smithi161 bash[19769]: audit 2024-02-13T03:28:29.491377+0000 mon.a (mon.0) 1257 : audit [INF] from='client.48053 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi161-61726-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:30.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:30 smithi161 bash[19769]: cluster 2024-02-13T03:28:29.491484+0000 mon.a (mon.0) 1258 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-02-13T03:28:30.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:30 smithi161 bash[19769]: audit 2024-02-13T03:28:29.604536+0000 mon.c (mon.2) 300 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:30 smithi183 bash[20824]: cluster 2024-02-13T03:28:29.243087+0000 mgr.y (mgr.24473) 311 : cluster [DBG] pgmap v305: 308 pgs: 14 active+clean+snaptrim, 40 unknown, 40 active+clean+snaptrim_wait, 214 active+clean; 28 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-13T03:28:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:30 smithi183 bash[20824]: audit 2024-02-13T03:28:29.491023+0000 mon.a (mon.0) 1256 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi161-61719-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi161-61719-27"}]': finished 2024-02-13T03:28:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:30 smithi183 bash[20824]: audit 2024-02-13T03:28:29.491377+0000 mon.a (mon.0) 1257 : audit [INF] from='client.48053 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi161-61726-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:30 smithi183 bash[20824]: cluster 2024-02-13T03:28:29.491484+0000 mon.a (mon.0) 1258 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-02-13T03:28:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:30 smithi183 bash[20824]: audit 2024-02-13T03:28:29.604536+0000 mon.c (mon.2) 300 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:31.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:28:31] "GET /metrics HTTP/1.1" 200 188963 "" "Prometheus/2.33.4" 2024-02-13T03:28:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[27879]: cluster 2024-02-13T03:28:30.496694+0000 mon.a (mon.0) 1259 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-02-13T03:28:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[27879]: audit 2024-02-13T03:28:30.500798+0000 mon.a (mon.0) 1260 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[27879]: audit 2024-02-13T03:28:30.502221+0000 mon.b (mon.1) 157 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[27879]: audit 2024-02-13T03:28:30.506903+0000 mon.c (mon.2) 301 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[27879]: audit 2024-02-13T03:28:30.510456+0000 mon.a (mon.0) 1261 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[27879]: audit 2024-02-13T03:28:30.605900+0000 mon.c (mon.2) 302 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[27879]: cluster 2024-02-13T03:28:31.245744+0000 mgr.y (mgr.24473) 312 : cluster [DBG] pgmap v308: 276 pgs: 32 creating+peering, 244 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s wr, 1 op/s 2024-02-13T03:28:31.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[19769]: cluster 2024-02-13T03:28:30.496694+0000 mon.a (mon.0) 1259 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-02-13T03:28:31.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[19769]: audit 2024-02-13T03:28:30.500798+0000 mon.a (mon.0) 1260 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:31.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[19769]: audit 2024-02-13T03:28:30.502221+0000 mon.b (mon.1) 157 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:31.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[19769]: audit 2024-02-13T03:28:30.506903+0000 mon.c (mon.2) 301 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:31.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[19769]: audit 2024-02-13T03:28:30.510456+0000 mon.a (mon.0) 1261 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:31.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[19769]: audit 2024-02-13T03:28:30.605900+0000 mon.c (mon.2) 302 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:31.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:31 smithi161 bash[19769]: cluster 2024-02-13T03:28:31.245744+0000 mgr.y (mgr.24473) 312 : cluster [DBG] pgmap v308: 276 pgs: 32 creating+peering, 244 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s wr, 1 op/s 2024-02-13T03:28:31.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:31 smithi183 bash[20824]: cluster 2024-02-13T03:28:30.496694+0000 mon.a (mon.0) 1259 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-02-13T03:28:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:31 smithi183 bash[20824]: audit 2024-02-13T03:28:30.500798+0000 mon.a (mon.0) 1260 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:31 smithi183 bash[20824]: audit 2024-02-13T03:28:30.502221+0000 mon.b (mon.1) 157 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:31 smithi183 bash[20824]: audit 2024-02-13T03:28:30.506903+0000 mon.c (mon.2) 301 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:31 smithi183 bash[20824]: audit 2024-02-13T03:28:30.510456+0000 mon.a (mon.0) 1261 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:31 smithi183 bash[20824]: audit 2024-02-13T03:28:30.605900+0000 mon.c (mon.2) 302 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:31 smithi183 bash[20824]: cluster 2024-02-13T03:28:31.245744+0000 mgr.y (mgr.24473) 312 : cluster [DBG] pgmap v308: 276 pgs: 32 creating+peering, 244 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.2 KiB/s wr, 1 op/s 2024-02-13T03:28:32.516 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForCompl(67687 ms total) 2024-02-13T03:28:32.516 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: 2024-02-13T03:28:32.516 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-02-13T03:28:32.516 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-02-13T03:28:32.517 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (160 ms) 2024-02-13T03:28:32.517 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (160 ms total) 2024-02-13T03:28:32.517 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: 2024-02-13T03:28:32.517 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-02-13T03:28:32.517 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-02-13T03:28:32.518 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (50 ms) 2024-02-13T03:28:32.518 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-02-13T03:28:32.518 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (3 ms) 2024-02-13T03:28:32.518 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (53 ms total) 2024-02-13T03:28:32.518 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: 2024-02-13T03:28:32.518 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-02-13T03:28:32.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-02-13T03:28:32.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (425 ms) 2024-02-13T03:28:32.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-02-13T03:28:32.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (4 ms) 2024-02-13T03:28:32.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (429 ms total) 2024-02-13T03:28:32.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: 2024-02-13T03:28:32.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-02-13T03:28:32.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-02-13T03:28:32.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (40 ms) 2024-02-13T03:28:32.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-02-13T03:28:32.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (7 ms) 2024-02-13T03:28:32.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (47 ms total) 2024-02-13T03:28:32.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: 2024-02-13T03:28:32.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-02-13T03:28:32.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-02-13T03:28:32.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1045 ms) 2024-02-13T03:28:32.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1045 ms total) 2024-02-13T03:28:32.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: 2024-02-13T03:28:32.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-02-13T03:28:32.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (106227 ms total) 2024-02-13T03:28:32.521 INFO:tasks.workunit.client.0.smithi161.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-02-13T03:28:32.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: audit 2024-02-13T03:28:31.501858+0000 mon.a (mon.0) 1262 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:32.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: audit 2024-02-13T03:28:31.502068+0000 mon.a (mon.0) 1263 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]': finished 2024-02-13T03:28:32.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: cluster 2024-02-13T03:28:31.502333+0000 mon.a (mon.0) 1264 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-02-13T03:28:32.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: audit 2024-02-13T03:28:31.503135+0000 mon.c (mon.2) 303 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: audit 2024-02-13T03:28:31.503650+0000 mon.c (mon.2) 304 : audit [INF] from='client.? v1:172.21.15.161:0/872804178' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi161-61726-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: audit 2024-02-13T03:28:31.506756+0000 mon.a (mon.0) 1265 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: audit 2024-02-13T03:28:31.507198+0000 mon.a (mon.0) 1266 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi161-61726-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: audit 2024-02-13T03:28:31.507457+0000 mon.a (mon.0) 1267 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: audit 2024-02-13T03:28:31.607366+0000 mon.c (mon.2) 305 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: audit 2024-02-13T03:28:32.503877+0000 mon.a (mon.0) 1268 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]': finished 2024-02-13T03:28:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: audit 2024-02-13T03:28:32.504057+0000 mon.a (mon.0) 1269 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi161-61726-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: audit 2024-02-13T03:28:32.504217+0000 mon.a (mon.0) 1270 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi161-61719-27"}]': finished 2024-02-13T03:28:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: cluster 2024-02-13T03:28:32.504674+0000 mon.a (mon.0) 1271 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-02-13T03:28:32.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[27879]: audit 2024-02-13T03:28:32.507944+0000 mon.a (mon.0) 1272 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:32.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: audit 2024-02-13T03:28:31.501858+0000 mon.a (mon.0) 1262 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:32.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: audit 2024-02-13T03:28:31.502068+0000 mon.a (mon.0) 1263 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]': finished 2024-02-13T03:28:32.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: cluster 2024-02-13T03:28:31.502333+0000 mon.a (mon.0) 1264 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-02-13T03:28:32.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: audit 2024-02-13T03:28:31.503135+0000 mon.c (mon.2) 303 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:32.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: audit 2024-02-13T03:28:31.503650+0000 mon.c (mon.2) 304 : audit [INF] from='client.? v1:172.21.15.161:0/872804178' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi161-61726-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:32.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: audit 2024-02-13T03:28:31.506756+0000 mon.a (mon.0) 1265 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:32.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: audit 2024-02-13T03:28:31.507198+0000 mon.a (mon.0) 1266 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi161-61726-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:32.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: audit 2024-02-13T03:28:31.507457+0000 mon.a (mon.0) 1267 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:32.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: audit 2024-02-13T03:28:31.607366+0000 mon.c (mon.2) 305 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:32.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: audit 2024-02-13T03:28:32.503877+0000 mon.a (mon.0) 1268 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]': finished 2024-02-13T03:28:32.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: audit 2024-02-13T03:28:32.504057+0000 mon.a (mon.0) 1269 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi161-61726-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:32.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: audit 2024-02-13T03:28:32.504217+0000 mon.a (mon.0) 1270 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi161-61719-27"}]': finished 2024-02-13T03:28:32.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: cluster 2024-02-13T03:28:32.504674+0000 mon.a (mon.0) 1271 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-02-13T03:28:32.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:32 smithi161 bash[19769]: audit 2024-02-13T03:28:32.507944+0000 mon.a (mon.0) 1272 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: audit 2024-02-13T03:28:31.501858+0000 mon.a (mon.0) 1262 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: audit 2024-02-13T03:28:31.502068+0000 mon.a (mon.0) 1263 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi161-61893-36"}]': finished 2024-02-13T03:28:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: cluster 2024-02-13T03:28:31.502333+0000 mon.a (mon.0) 1264 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-02-13T03:28:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: audit 2024-02-13T03:28:31.503135+0000 mon.c (mon.2) 303 : audit [INF] from='client.? v1:172.21.15.161:0/1074132221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: audit 2024-02-13T03:28:31.503650+0000 mon.c (mon.2) 304 : audit [INF] from='client.? v1:172.21.15.161:0/872804178' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi161-61726-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: audit 2024-02-13T03:28:31.506756+0000 mon.a (mon.0) 1265 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]: dispatch 2024-02-13T03:28:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: audit 2024-02-13T03:28:31.507198+0000 mon.a (mon.0) 1266 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi161-61726-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: audit 2024-02-13T03:28:31.507457+0000 mon.a (mon.0) 1267 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: audit 2024-02-13T03:28:31.607366+0000 mon.c (mon.2) 305 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: audit 2024-02-13T03:28:32.503877+0000 mon.a (mon.0) 1268 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi161-61893-36"}]': finished 2024-02-13T03:28:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: audit 2024-02-13T03:28:32.504057+0000 mon.a (mon.0) 1269 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi161-61726-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:32.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: audit 2024-02-13T03:28:32.504217+0000 mon.a (mon.0) 1270 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi161-61719-27"}]': finished 2024-02-13T03:28:32.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: cluster 2024-02-13T03:28:32.504674+0000 mon.a (mon.0) 1271 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-02-13T03:28:32.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:32 smithi183 bash[20824]: audit 2024-02-13T03:28:32.507944+0000 mon.a (mon.0) 1272 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi161-61719-27"}]: dispatch 2024-02-13T03:28:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[27879]: audit 2024-02-13T03:28:32.525382+0000 mon.a (mon.0) 1273 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[27879]: audit 2024-02-13T03:28:32.527822+0000 mon.b (mon.1) 158 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[27879]: audit 2024-02-13T03:28:32.608948+0000 mon.c (mon.2) 306 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[27879]: cluster 2024-02-13T03:28:33.246690+0000 mgr.y (mgr.24473) 313 : cluster [DBG] pgmap v311: 300 pgs: 32 unknown, 32 creating+peering, 236 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-02-13T03:28:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[27879]: cluster 2024-02-13T03:28:33.478439+0000 mon.a (mon.0) 1274 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[27879]: audit 2024-02-13T03:28:33.491010+0000 mon.a (mon.0) 1275 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi161-61719-27"}]': finished 2024-02-13T03:28:33.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[27879]: audit 2024-02-13T03:28:33.491460+0000 mon.a (mon.0) 1276 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:28:33.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[27879]: cluster 2024-02-13T03:28:33.491687+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-02-13T03:28:33.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[27879]: audit 2024-02-13T03:28:33.496954+0000 mon.a (mon.0) 1278 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-15"}]: dispatch 2024-02-13T03:28:33.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[27879]: audit 2024-02-13T03:28:33.496992+0000 mon.b (mon.1) 159 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-15"}]: dispatch 2024-02-13T03:28:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[19769]: audit 2024-02-13T03:28:32.525382+0000 mon.a (mon.0) 1273 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[19769]: audit 2024-02-13T03:28:32.527822+0000 mon.b (mon.1) 158 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[19769]: audit 2024-02-13T03:28:32.608948+0000 mon.c (mon.2) 306 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[19769]: cluster 2024-02-13T03:28:33.246690+0000 mgr.y (mgr.24473) 313 : cluster [DBG] pgmap v311: 300 pgs: 32 unknown, 32 creating+peering, 236 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-02-13T03:28:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[19769]: cluster 2024-02-13T03:28:33.478439+0000 mon.a (mon.0) 1274 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[19769]: audit 2024-02-13T03:28:33.491010+0000 mon.a (mon.0) 1275 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi161-61719-27"}]': finished 2024-02-13T03:28:33.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[19769]: audit 2024-02-13T03:28:33.491460+0000 mon.a (mon.0) 1276 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:28:33.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[19769]: cluster 2024-02-13T03:28:33.491687+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-02-13T03:28:33.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[19769]: audit 2024-02-13T03:28:33.496954+0000 mon.a (mon.0) 1278 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-15"}]: dispatch 2024-02-13T03:28:33.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:33 smithi161 bash[19769]: audit 2024-02-13T03:28:33.496992+0000 mon.b (mon.1) 159 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-15"}]: dispatch 2024-02-13T03:28:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:33 smithi183 bash[20824]: audit 2024-02-13T03:28:32.525382+0000 mon.a (mon.0) 1273 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:33 smithi183 bash[20824]: audit 2024-02-13T03:28:32.527822+0000 mon.b (mon.1) 158 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:33 smithi183 bash[20824]: audit 2024-02-13T03:28:32.608948+0000 mon.c (mon.2) 306 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:33 smithi183 bash[20824]: cluster 2024-02-13T03:28:33.246690+0000 mgr.y (mgr.24473) 313 : cluster [DBG] pgmap v311: 300 pgs: 32 unknown, 32 creating+peering, 236 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-02-13T03:28:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:33 smithi183 bash[20824]: cluster 2024-02-13T03:28:33.478439+0000 mon.a (mon.0) 1274 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:33 smithi183 bash[20824]: audit 2024-02-13T03:28:33.491010+0000 mon.a (mon.0) 1275 : audit [INF] from='client.? v1:172.21.15.161:0/3446962506' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi161-61719-27"}]': finished 2024-02-13T03:28:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:33 smithi183 bash[20824]: audit 2024-02-13T03:28:33.491460+0000 mon.a (mon.0) 1276 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:28:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:33 smithi183 bash[20824]: cluster 2024-02-13T03:28:33.491687+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-02-13T03:28:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:33 smithi183 bash[20824]: audit 2024-02-13T03:28:33.496954+0000 mon.a (mon.0) 1278 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-15"}]: dispatch 2024-02-13T03:28:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:33 smithi183 bash[20824]: audit 2024-02-13T03:28:33.496992+0000 mon.b (mon.1) 159 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-15"}]: dispatch 2024-02-13T03:28:34.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:33.527219+0000 mon.c (mon.2) 307 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:33.610549+0000 mon.c (mon.2) 308 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:34.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:33.965184+0000 mon.a (mon.0) 1279 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:33.966268+0000 mon.c (mon.2) 309 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:33.966730+0000 mon.a (mon.0) 1280 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:33.967403+0000 mon.c (mon.2) 310 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi161-61719-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:34.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:33.967844+0000 mon.a (mon.0) 1281 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi161-61719-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:34.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:34.494223+0000 mon.a (mon.0) 1282 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-15"}]': finished 2024-02-13T03:28:34.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:34.494357+0000 mon.a (mon.0) 1283 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi161-61719-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:34.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: cluster 2024-02-13T03:28:34.494440+0000 mon.a (mon.0) 1284 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-02-13T03:28:34.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:34.500125+0000 mon.c (mon.2) 311 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi161-61719-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:34.500797+0000 mon.b (mon.1) 160 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-15", "mode": "writeback"}]: dispatch 2024-02-13T03:28:34.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:34.501434+0000 mon.a (mon.0) 1285 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-15", "mode": "writeback"}]: dispatch 2024-02-13T03:28:34.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:34.501791+0000 mon.b (mon.1) 161 : audit [INF] from='client.? v1:172.21.15.161:0/431041554' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi161-61726-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:34.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:34.502100+0000 mon.a (mon.0) 1286 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi161-61719-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[27879]: audit 2024-02-13T03:28:34.503223+0000 mon.a (mon.0) 1287 : audit [INF] from='client.48727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi161-61726-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:34.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:33.527219+0000 mon.c (mon.2) 307 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:33.610549+0000 mon.c (mon.2) 308 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:34.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:33.965184+0000 mon.a (mon.0) 1279 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:33.966268+0000 mon.c (mon.2) 309 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:33.966730+0000 mon.a (mon.0) 1280 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:33.967403+0000 mon.c (mon.2) 310 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi161-61719-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:34.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:33.967844+0000 mon.a (mon.0) 1281 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi161-61719-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:34.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:34.494223+0000 mon.a (mon.0) 1282 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-15"}]': finished 2024-02-13T03:28:34.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:34.494357+0000 mon.a (mon.0) 1283 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi161-61719-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:34.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: cluster 2024-02-13T03:28:34.494440+0000 mon.a (mon.0) 1284 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-02-13T03:28:34.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:34.500125+0000 mon.c (mon.2) 311 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi161-61719-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:34.500797+0000 mon.b (mon.1) 160 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-15", "mode": "writeback"}]: dispatch 2024-02-13T03:28:34.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:34.501434+0000 mon.a (mon.0) 1285 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-15", "mode": "writeback"}]: dispatch 2024-02-13T03:28:34.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:34.501791+0000 mon.b (mon.1) 161 : audit [INF] from='client.? v1:172.21.15.161:0/431041554' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi161-61726-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:34.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:34.502100+0000 mon.a (mon.0) 1286 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi161-61719-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:34 smithi161 bash[19769]: audit 2024-02-13T03:28:34.503223+0000 mon.a (mon.0) 1287 : audit [INF] from='client.48727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi161-61726-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:33.527219+0000 mon.c (mon.2) 307 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:33.610549+0000 mon.c (mon.2) 308 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:33.965184+0000 mon.a (mon.0) 1279 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:33.966268+0000 mon.c (mon.2) 309 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:33.966730+0000 mon.a (mon.0) 1280 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:33.967403+0000 mon.c (mon.2) 310 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi161-61719-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:33.967844+0000 mon.a (mon.0) 1281 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi161-61719-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:34.494223+0000 mon.a (mon.0) 1282 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-15"}]': finished 2024-02-13T03:28:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:34.494357+0000 mon.a (mon.0) 1283 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi161-61719-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: cluster 2024-02-13T03:28:34.494440+0000 mon.a (mon.0) 1284 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-02-13T03:28:34.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:34.500125+0000 mon.c (mon.2) 311 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi161-61719-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:34.500797+0000 mon.b (mon.1) 160 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-15", "mode": "writeback"}]: dispatch 2024-02-13T03:28:34.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:34.501434+0000 mon.a (mon.0) 1285 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-15", "mode": "writeback"}]: dispatch 2024-02-13T03:28:34.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:34.501791+0000 mon.b (mon.1) 161 : audit [INF] from='client.? v1:172.21.15.161:0/431041554' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi161-61726-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:34.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:34.502100+0000 mon.a (mon.0) 1286 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi161-61719-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:34.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:34 smithi183 bash[20824]: audit 2024-02-13T03:28:34.503223+0000 mon.a (mon.0) 1287 : audit [INF] from='client.48727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi161-61726-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:36.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:28:35 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:28:36.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:35 smithi183 bash[20824]: audit 2024-02-13T03:28:34.612158+0000 mon.c (mon.2) 312 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:36.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:35 smithi183 bash[20824]: audit 2024-02-13T03:28:34.779310+0000 mon.a (mon.0) 1288 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:28:36.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:35 smithi183 bash[20824]: cluster 2024-02-13T03:28:35.247624+0000 mgr.y (mgr.24473) 314 : cluster [DBG] pgmap v314: 300 pgs: 32 unknown, 32 creating+peering, 236 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:36.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:35 smithi183 bash[20824]: cluster 2024-02-13T03:28:35.488182+0000 mon.a (mon.0) 1289 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:28:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:35 smithi161 bash[27879]: audit 2024-02-13T03:28:34.612158+0000 mon.c (mon.2) 312 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:35 smithi161 bash[27879]: audit 2024-02-13T03:28:34.779310+0000 mon.a (mon.0) 1288 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:28:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:35 smithi161 bash[27879]: cluster 2024-02-13T03:28:35.247624+0000 mgr.y (mgr.24473) 314 : cluster [DBG] pgmap v314: 300 pgs: 32 unknown, 32 creating+peering, 236 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:35 smithi161 bash[27879]: cluster 2024-02-13T03:28:35.488182+0000 mon.a (mon.0) 1289 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:28:36.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:35 smithi161 bash[19769]: audit 2024-02-13T03:28:34.612158+0000 mon.c (mon.2) 312 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:36.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:35 smithi161 bash[19769]: audit 2024-02-13T03:28:34.779310+0000 mon.a (mon.0) 1288 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:28:36.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:35 smithi161 bash[19769]: cluster 2024-02-13T03:28:35.247624+0000 mgr.y (mgr.24473) 314 : cluster [DBG] pgmap v314: 300 pgs: 32 unknown, 32 creating+peering, 236 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:36.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:35 smithi161 bash[19769]: cluster 2024-02-13T03:28:35.488182+0000 mon.a (mon.0) 1289 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:28:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:36 smithi183 bash[20824]: audit 2024-02-13T03:28:35.499700+0000 mon.a (mon.0) 1290 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-15", "mode": "writeback"}]': finished 2024-02-13T03:28:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:36 smithi183 bash[20824]: audit 2024-02-13T03:28:35.613772+0000 mon.c (mon.2) 313 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:36 smithi183 bash[20824]: audit 2024-02-13T03:28:35.872677+0000 mgr.y (mgr.24473) 315 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:36 smithi183 bash[20824]: audit 2024-02-13T03:28:35.933076+0000 mon.a (mon.0) 1291 : audit [INF] from='client.48727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi161-61726-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:36 smithi183 bash[20824]: cluster 2024-02-13T03:28:35.933378+0000 mon.a (mon.0) 1292 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-02-13T03:28:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:36 smithi183 bash[20824]: audit 2024-02-13T03:28:35.936807+0000 mon.a (mon.0) 1293 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:36 smithi183 bash[20824]: audit 2024-02-13T03:28:35.938241+0000 mon.b (mon.1) 162 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:36 smithi183 bash[20824]: audit 2024-02-13T03:28:36.500638+0000 mon.a (mon.0) 1294 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi161-61719-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi161-61719-28"}]': finished 2024-02-13T03:28:37.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:36 smithi183 bash[20824]: audit 2024-02-13T03:28:36.500846+0000 mon.a (mon.0) 1295 : audit [INF] from='client.48676 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]': finished 2024-02-13T03:28:37.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:36 smithi183 bash[20824]: cluster 2024-02-13T03:28:36.500995+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-02-13T03:28:37.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:36 smithi183 bash[20824]: audit 2024-02-13T03:28:36.505349+0000 mon.b (mon.1) 163 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:37.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:36 smithi183 bash[20824]: audit 2024-02-13T03:28:36.506595+0000 mon.a (mon.0) 1297 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:37.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:36 smithi183 bash[20824]: audit 2024-02-13T03:28:36.543882+0000 mon.b (mon.1) 164 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:37.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[19769]: audit 2024-02-13T03:28:35.499700+0000 mon.a (mon.0) 1290 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-15", "mode": "writeback"}]': finished 2024-02-13T03:28:37.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[19769]: audit 2024-02-13T03:28:35.613772+0000 mon.c (mon.2) 313 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:37.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[19769]: audit 2024-02-13T03:28:35.872677+0000 mgr.y (mgr.24473) 315 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:37.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[19769]: audit 2024-02-13T03:28:35.933076+0000 mon.a (mon.0) 1291 : audit [INF] from='client.48727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi161-61726-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:37.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[19769]: cluster 2024-02-13T03:28:35.933378+0000 mon.a (mon.0) 1292 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-02-13T03:28:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[19769]: audit 2024-02-13T03:28:35.936807+0000 mon.a (mon.0) 1293 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[19769]: audit 2024-02-13T03:28:35.938241+0000 mon.b (mon.1) 162 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[19769]: audit 2024-02-13T03:28:36.500638+0000 mon.a (mon.0) 1294 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi161-61719-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi161-61719-28"}]': finished 2024-02-13T03:28:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[19769]: audit 2024-02-13T03:28:36.500846+0000 mon.a (mon.0) 1295 : audit [INF] from='client.48676 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]': finished 2024-02-13T03:28:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[19769]: cluster 2024-02-13T03:28:36.500995+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-02-13T03:28:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[19769]: audit 2024-02-13T03:28:36.505349+0000 mon.b (mon.1) 163 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[19769]: audit 2024-02-13T03:28:36.506595+0000 mon.a (mon.0) 1297 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[19769]: audit 2024-02-13T03:28:36.543882+0000 mon.b (mon.1) 164 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:37.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[27879]: audit 2024-02-13T03:28:35.499700+0000 mon.a (mon.0) 1290 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-15", "mode": "writeback"}]': finished 2024-02-13T03:28:37.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[27879]: audit 2024-02-13T03:28:35.613772+0000 mon.c (mon.2) 313 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:37.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[27879]: audit 2024-02-13T03:28:35.872677+0000 mgr.y (mgr.24473) 315 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:37.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[27879]: audit 2024-02-13T03:28:35.933076+0000 mon.a (mon.0) 1291 : audit [INF] from='client.48727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi161-61726-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:37.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[27879]: cluster 2024-02-13T03:28:35.933378+0000 mon.a (mon.0) 1292 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-02-13T03:28:37.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[27879]: audit 2024-02-13T03:28:35.936807+0000 mon.a (mon.0) 1293 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:37.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[27879]: audit 2024-02-13T03:28:35.938241+0000 mon.b (mon.1) 162 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:37.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[27879]: audit 2024-02-13T03:28:36.500638+0000 mon.a (mon.0) 1294 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi161-61719-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi161-61719-28"}]': finished 2024-02-13T03:28:37.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[27879]: audit 2024-02-13T03:28:36.500846+0000 mon.a (mon.0) 1295 : audit [INF] from='client.48676 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]': finished 2024-02-13T03:28:37.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[27879]: cluster 2024-02-13T03:28:36.500995+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-02-13T03:28:37.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[27879]: audit 2024-02-13T03:28:36.505349+0000 mon.b (mon.1) 163 : audit [INF] from='client.? v1:172.21.15.161:0/3452734021' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:37.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[27879]: audit 2024-02-13T03:28:36.506595+0000 mon.a (mon.0) 1297 : audit [INF] from='client.48676 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]: dispatch 2024-02-13T03:28:37.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:36 smithi161 bash[27879]: audit 2024-02-13T03:28:36.543882+0000 mon.b (mon.1) 164 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:37.519 INFO:tasks.workunit.client.0.smithi161.stdout:: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-02-13T03:28:37.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-02-13T03:28:37.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4035 ms) 2024-02-13T03:28:37.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-02-13T03:28:37.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3888 ms) 2024-02-13T03:28:37.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-02-13T03:28:37.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3997 ms) 2024-02-13T03:28:37.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11920 ms total) 2024-02-13T03:28:37.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: 2024-02-13T03:28:37.519 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-02-13T03:28:37.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [==========] 20 tests from 5 test suites ran. (111188 ms total) 2024-02-13T03:28:37.520 INFO:tasks.workunit.client.0.smithi161.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-02-13T03:28:37.520 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:28:37.520 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61839 2024-02-13T03:28:37.520 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61839 2024-02-13T03:28:37.520 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:28:37.520 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61778 2024-02-13T03:28:37.520 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61778 2024-02-13T03:28:37.520 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:28:37.521 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61754 2024-02-13T03:28:37.521 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61754 2024-02-13T03:28:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:37 smithi183 bash[20824]: audit 2024-02-13T03:28:36.615232+0000 mon.c (mon.2) 314 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:37 smithi183 bash[20824]: audit 2024-02-13T03:28:36.953875+0000 mon.a (mon.0) 1298 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:37 smithi183 bash[20824]: cluster 2024-02-13T03:28:37.248521+0000 mgr.y (mgr.24473) 316 : cluster [DBG] pgmap v317: 268 pgs: 8 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:37 smithi183 bash[20824]: audit 2024-02-13T03:28:37.510777+0000 mon.a (mon.0) 1299 : audit [INF] from='client.48676 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]': finished 2024-02-13T03:28:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:37 smithi183 bash[20824]: audit 2024-02-13T03:28:37.511516+0000 mon.a (mon.0) 1300 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:28:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:37 smithi183 bash[20824]: cluster 2024-02-13T03:28:37.511665+0000 mon.a (mon.0) 1301 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-02-13T03:28:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:37 smithi183 bash[20824]: audit 2024-02-13T03:28:37.514869+0000 mon.a (mon.0) 1302 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15"}]: dispatch 2024-02-13T03:28:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:37 smithi183 bash[20824]: audit 2024-02-13T03:28:37.516356+0000 mon.b (mon.1) 165 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15"}]: dispatch 2024-02-13T03:28:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:37 smithi183 bash[20824]: audit 2024-02-13T03:28:37.524867+0000 mon.a (mon.0) 1303 : audit [INF] from='client.48739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi161-61726-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:37 smithi183 bash[20824]: audit 2024-02-13T03:28:37.524923+0000 mon.b (mon.1) 166 : audit [INF] from='client.? v1:172.21.15.161:0/1339343679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi161-61726-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:37 smithi183 bash[20824]: audit 2024-02-13T03:28:37.616758+0000 mon.c (mon.2) 315 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:38.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[27879]: audit 2024-02-13T03:28:36.615232+0000 mon.c (mon.2) 314 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:38.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[27879]: audit 2024-02-13T03:28:36.953875+0000 mon.a (mon.0) 1298 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:38.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[27879]: cluster 2024-02-13T03:28:37.248521+0000 mgr.y (mgr.24473) 316 : cluster [DBG] pgmap v317: 268 pgs: 8 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:38.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[27879]: audit 2024-02-13T03:28:37.510777+0000 mon.a (mon.0) 1299 : audit [INF] from='client.48676 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]': finished 2024-02-13T03:28:38.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[27879]: audit 2024-02-13T03:28:37.511516+0000 mon.a (mon.0) 1300 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:28:38.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[27879]: cluster 2024-02-13T03:28:37.511665+0000 mon.a (mon.0) 1301 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-02-13T03:28:38.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[27879]: audit 2024-02-13T03:28:37.514869+0000 mon.a (mon.0) 1302 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15"}]: dispatch 2024-02-13T03:28:38.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[27879]: audit 2024-02-13T03:28:37.516356+0000 mon.b (mon.1) 165 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15"}]: dispatch 2024-02-13T03:28:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[27879]: audit 2024-02-13T03:28:37.524867+0000 mon.a (mon.0) 1303 : audit [INF] from='client.48739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi161-61726-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[27879]: audit 2024-02-13T03:28:37.524923+0000 mon.b (mon.1) 166 : audit [INF] from='client.? v1:172.21.15.161:0/1339343679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi161-61726-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[27879]: audit 2024-02-13T03:28:37.616758+0000 mon.c (mon.2) 315 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:38.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[19769]: audit 2024-02-13T03:28:36.615232+0000 mon.c (mon.2) 314 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:38.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[19769]: audit 2024-02-13T03:28:36.953875+0000 mon.a (mon.0) 1298 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:38.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[19769]: cluster 2024-02-13T03:28:37.248521+0000 mgr.y (mgr.24473) 316 : cluster [DBG] pgmap v317: 268 pgs: 8 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:38.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[19769]: audit 2024-02-13T03:28:37.510777+0000 mon.a (mon.0) 1299 : audit [INF] from='client.48676 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi161-61993-20"}]': finished 2024-02-13T03:28:38.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[19769]: audit 2024-02-13T03:28:37.511516+0000 mon.a (mon.0) 1300 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:28:38.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[19769]: cluster 2024-02-13T03:28:37.511665+0000 mon.a (mon.0) 1301 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-02-13T03:28:38.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[19769]: audit 2024-02-13T03:28:37.514869+0000 mon.a (mon.0) 1302 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15"}]: dispatch 2024-02-13T03:28:38.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[19769]: audit 2024-02-13T03:28:37.516356+0000 mon.b (mon.1) 165 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15"}]: dispatch 2024-02-13T03:28:38.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[19769]: audit 2024-02-13T03:28:37.524867+0000 mon.a (mon.0) 1303 : audit [INF] from='client.48739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi161-61726-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:38.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[19769]: audit 2024-02-13T03:28:37.524923+0000 mon.b (mon.1) 166 : audit [INF] from='client.? v1:172.21.15.161:0/1339343679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi161-61726-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:38.432 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:37 smithi161 bash[19769]: audit 2024-02-13T03:28:37.616758+0000 mon.c (mon.2) 315 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:39.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:38 smithi183 bash[20824]: cluster 2024-02-13T03:28:38.482089+0000 mon.a (mon.0) 1304 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:39.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:38 smithi183 bash[20824]: cluster 2024-02-13T03:28:38.498152+0000 mon.a (mon.0) 1305 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:28:39.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:38 smithi183 bash[20824]: audit 2024-02-13T03:28:38.504741+0000 mon.a (mon.0) 1306 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15"}]': finished 2024-02-13T03:28:39.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:38 smithi183 bash[20824]: audit 2024-02-13T03:28:38.504961+0000 mon.a (mon.0) 1307 : audit [INF] from='client.48739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi161-61726-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:39.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:38 smithi183 bash[20824]: cluster 2024-02-13T03:28:38.505088+0000 mon.a (mon.0) 1308 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-02-13T03:28:39.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:38 smithi183 bash[20824]: audit 2024-02-13T03:28:38.508429+0000 mon.c (mon.2) 316 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:39.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:38 smithi183 bash[20824]: audit 2024-02-13T03:28:38.511427+0000 mon.a (mon.0) 1309 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:39.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:38 smithi183 bash[20824]: audit 2024-02-13T03:28:38.618423+0000 mon.c (mon.2) 317 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[19769]: cluster 2024-02-13T03:28:38.482089+0000 mon.a (mon.0) 1304 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[19769]: cluster 2024-02-13T03:28:38.498152+0000 mon.a (mon.0) 1305 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:28:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[19769]: audit 2024-02-13T03:28:38.504741+0000 mon.a (mon.0) 1306 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15"}]': finished 2024-02-13T03:28:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[19769]: audit 2024-02-13T03:28:38.504961+0000 mon.a (mon.0) 1307 : audit [INF] from='client.48739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi161-61726-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[19769]: cluster 2024-02-13T03:28:38.505088+0000 mon.a (mon.0) 1308 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-02-13T03:28:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[19769]: audit 2024-02-13T03:28:38.508429+0000 mon.c (mon.2) 316 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[19769]: audit 2024-02-13T03:28:38.511427+0000 mon.a (mon.0) 1309 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[19769]: audit 2024-02-13T03:28:38.618423+0000 mon.c (mon.2) 317 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[27879]: cluster 2024-02-13T03:28:38.482089+0000 mon.a (mon.0) 1304 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:28:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[27879]: cluster 2024-02-13T03:28:38.498152+0000 mon.a (mon.0) 1305 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:28:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[27879]: audit 2024-02-13T03:28:38.504741+0000 mon.a (mon.0) 1306 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-15"}]': finished 2024-02-13T03:28:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[27879]: audit 2024-02-13T03:28:38.504961+0000 mon.a (mon.0) 1307 : audit [INF] from='client.48739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi161-61726-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[27879]: cluster 2024-02-13T03:28:38.505088+0000 mon.a (mon.0) 1308 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-02-13T03:28:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[27879]: audit 2024-02-13T03:28:38.508429+0000 mon.c (mon.2) 316 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[27879]: audit 2024-02-13T03:28:38.511427+0000 mon.a (mon.0) 1309 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:38 smithi161 bash[27879]: audit 2024-02-13T03:28:38.618423+0000 mon.c (mon.2) 317 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:39.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:28:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:28:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:28:40.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:40 smithi161 bash[19769]: cluster 2024-02-13T03:28:39.249390+0000 mgr.y (mgr.24473) 317 : cluster [DBG] pgmap v320: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:40.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:40 smithi161 bash[19769]: audit 2024-02-13T03:28:39.510008+0000 mon.a (mon.0) 1310 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]': finished 2024-02-13T03:28:40.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:40 smithi161 bash[19769]: cluster 2024-02-13T03:28:39.510251+0000 mon.a (mon.0) 1311 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-02-13T03:28:40.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:40 smithi161 bash[19769]: audit 2024-02-13T03:28:39.517362+0000 mon.c (mon.2) 318 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:40.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:40 smithi161 bash[19769]: audit 2024-02-13T03:28:39.619823+0000 mon.c (mon.2) 319 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:40.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:40 smithi161 bash[19769]: audit 2024-02-13T03:28:39.954596+0000 mon.a (mon.0) 1312 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:40 smithi161 bash[27879]: cluster 2024-02-13T03:28:39.249390+0000 mgr.y (mgr.24473) 317 : cluster [DBG] pgmap v320: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:40 smithi161 bash[27879]: audit 2024-02-13T03:28:39.510008+0000 mon.a (mon.0) 1310 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]': finished 2024-02-13T03:28:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:40 smithi161 bash[27879]: cluster 2024-02-13T03:28:39.510251+0000 mon.a (mon.0) 1311 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-02-13T03:28:40.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:40 smithi161 bash[27879]: audit 2024-02-13T03:28:39.517362+0000 mon.c (mon.2) 318 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:40.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:40 smithi161 bash[27879]: audit 2024-02-13T03:28:39.619823+0000 mon.c (mon.2) 319 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:40.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:40 smithi161 bash[27879]: audit 2024-02-13T03:28:39.954596+0000 mon.a (mon.0) 1312 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:40.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:40 smithi183 bash[20824]: cluster 2024-02-13T03:28:39.249390+0000 mgr.y (mgr.24473) 317 : cluster [DBG] pgmap v320: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:40.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:40 smithi183 bash[20824]: audit 2024-02-13T03:28:39.510008+0000 mon.a (mon.0) 1310 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi161-61719-28"}]': finished 2024-02-13T03:28:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:40 smithi183 bash[20824]: cluster 2024-02-13T03:28:39.510251+0000 mon.a (mon.0) 1311 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-02-13T03:28:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:40 smithi183 bash[20824]: audit 2024-02-13T03:28:39.517362+0000 mon.c (mon.2) 318 : audit [INF] from='client.? v1:172.21.15.161:0/2095820128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:40 smithi183 bash[20824]: audit 2024-02-13T03:28:39.619823+0000 mon.c (mon.2) 319 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:40 smithi183 bash[20824]: audit 2024-02-13T03:28:39.954596+0000 mon.a (mon.0) 1312 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]: dispatch 2024-02-13T03:28:41.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:28:41] "GET /metrics HTTP/1.1" 200 191831 "" "Prometheus/2.33.4" 2024-02-13T03:28:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[27879]: audit 2024-02-13T03:28:40.514306+0000 mon.a (mon.0) 1313 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]': finished 2024-02-13T03:28:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[27879]: cluster 2024-02-13T03:28:40.514599+0000 mon.a (mon.0) 1314 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-02-13T03:28:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[27879]: audit 2024-02-13T03:28:40.517810+0000 mon.c (mon.2) 320 : audit [INF] from='client.? v1:172.21.15.161:0/2303638138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi161-61726-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[27879]: audit 2024-02-13T03:28:40.521830+0000 mon.a (mon.0) 1315 : audit [INF] from='client.48074 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi161-61726-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[27879]: audit 2024-02-13T03:28:40.522979+0000 mon.a (mon.0) 1316 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[27879]: audit 2024-02-13T03:28:40.525765+0000 mon.b (mon.1) 167 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[27879]: audit 2024-02-13T03:28:40.541511+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[27879]: audit 2024-02-13T03:28:40.542726+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[27879]: audit 2024-02-13T03:28:40.543142+0000 mon.a (mon.0) 1319 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi161-61719-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[27879]: audit 2024-02-13T03:28:40.621147+0000 mon.c (mon.2) 321 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:41.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[27879]: cluster 2024-02-13T03:28:41.252040+0000 mgr.y (mgr.24473) 318 : cluster [DBG] pgmap v323: 292 pgs: 64 creating+peering, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:41.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[19769]: audit 2024-02-13T03:28:40.514306+0000 mon.a (mon.0) 1313 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]': finished 2024-02-13T03:28:41.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[19769]: cluster 2024-02-13T03:28:40.514599+0000 mon.a (mon.0) 1314 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-02-13T03:28:41.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[19769]: audit 2024-02-13T03:28:40.517810+0000 mon.c (mon.2) 320 : audit [INF] from='client.? v1:172.21.15.161:0/2303638138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi161-61726-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:41.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[19769]: audit 2024-02-13T03:28:40.521830+0000 mon.a (mon.0) 1315 : audit [INF] from='client.48074 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi161-61726-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:41.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[19769]: audit 2024-02-13T03:28:40.522979+0000 mon.a (mon.0) 1316 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:41.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[19769]: audit 2024-02-13T03:28:40.525765+0000 mon.b (mon.1) 167 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:41.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[19769]: audit 2024-02-13T03:28:40.541511+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:41.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[19769]: audit 2024-02-13T03:28:40.542726+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:41.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[19769]: audit 2024-02-13T03:28:40.543142+0000 mon.a (mon.0) 1319 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi161-61719-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:41.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[19769]: audit 2024-02-13T03:28:40.621147+0000 mon.c (mon.2) 321 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:41.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:41 smithi161 bash[19769]: cluster 2024-02-13T03:28:41.252040+0000 mgr.y (mgr.24473) 318 : cluster [DBG] pgmap v323: 292 pgs: 64 creating+peering, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:41 smithi183 bash[20824]: audit 2024-02-13T03:28:40.514306+0000 mon.a (mon.0) 1313 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi161-61719-28"}]': finished 2024-02-13T03:28:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:41 smithi183 bash[20824]: cluster 2024-02-13T03:28:40.514599+0000 mon.a (mon.0) 1314 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-02-13T03:28:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:41 smithi183 bash[20824]: audit 2024-02-13T03:28:40.517810+0000 mon.c (mon.2) 320 : audit [INF] from='client.? v1:172.21.15.161:0/2303638138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi161-61726-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:41 smithi183 bash[20824]: audit 2024-02-13T03:28:40.521830+0000 mon.a (mon.0) 1315 : audit [INF] from='client.48074 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi161-61726-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:41 smithi183 bash[20824]: audit 2024-02-13T03:28:40.522979+0000 mon.a (mon.0) 1316 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:41 smithi183 bash[20824]: audit 2024-02-13T03:28:40.525765+0000 mon.b (mon.1) 167 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:41 smithi183 bash[20824]: audit 2024-02-13T03:28:40.541511+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:41 smithi183 bash[20824]: audit 2024-02-13T03:28:40.542726+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:41 smithi183 bash[20824]: audit 2024-02-13T03:28:40.543142+0000 mon.a (mon.0) 1319 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi161-61719-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:41 smithi183 bash[20824]: audit 2024-02-13T03:28:40.621147+0000 mon.c (mon.2) 321 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:41 smithi183 bash[20824]: cluster 2024-02-13T03:28:41.252040+0000 mgr.y (mgr.24473) 318 : cluster [DBG] pgmap v323: 292 pgs: 64 creating+peering, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:42.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-02-13T03:28:42.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [==========] Running 56 tests from 4 test suites. 2024-02-13T03:28:42.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-02-13T03:28:42.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [----------] 31 tests from LibRadosAio 2024-02-13T03:28:42.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-02-13T03:28:42.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (11435 ms) 2024-02-13T03:28:42.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-02-13T03:28:42.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (22138 ms) 2024-02-13T03:28:42.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-02-13T03:28:42.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (8471 ms) 2024-02-13T03:28:42.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-02-13T03:28:42.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3061 ms) 2024-02-13T03:28:42.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-02-13T03:28:42.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3990 ms) 2024-02-13T03:28:42.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-02-13T03:28:42.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2900 ms) 2024-02-13T03:28:42.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-02-13T03:28:42.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3012 ms) 2024-02-13T03:28:42.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-02-13T03:28:42.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (4344 ms) 2024-02-13T03:28:42.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-02-13T03:28:42.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2651 ms) 2024-02-13T03:28:42.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3031 ms) 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3011 ms) 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3015 ms) 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3023 ms) 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3018 ms) 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3494 ms) 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (2580 ms) 2024-02-13T03:28:42.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3017 ms) 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (2835 ms) 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3022 ms) 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3846 ms) 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (2482 ms) 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3435 ms) 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (2480 ms) 2024-02-13T03:28:42.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-02-13T03:28:42.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2993 ms) 2024-02-13T03:28:42.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-02-13T03:28:42.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3008 ms) 2024-02-13T03:28:42.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-02-13T03:28:42.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3011 ms) 2024-02-13T03:28:42.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-02-13T03:28:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:42 smithi183 bash[20824]: audit 2024-02-13T03:28:41.518822+0000 mon.a (mon.0) 1320 : audit [INF] from='client.48074 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi161-61726-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:42 smithi183 bash[20824]: audit 2024-02-13T03:28:41.519504+0000 mon.a (mon.0) 1321 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:42 smithi183 bash[20824]: audit 2024-02-13T03:28:41.520212+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi161-61719-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:42 smithi183 bash[20824]: cluster 2024-02-13T03:28:41.520624+0000 mon.a (mon.0) 1323 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-02-13T03:28:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:42 smithi183 bash[20824]: audit 2024-02-13T03:28:41.528864+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi161-61719-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:42 smithi183 bash[20824]: audit 2024-02-13T03:28:41.551858+0000 mon.a (mon.0) 1325 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:42 smithi183 bash[20824]: audit 2024-02-13T03:28:41.553245+0000 mon.b (mon.1) 168 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:43.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:42 smithi183 bash[20824]: audit 2024-02-13T03:28:41.622724+0000 mon.c (mon.2) 322 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:43.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[19769]: audit 2024-02-13T03:28:41.518822+0000 mon.a (mon.0) 1320 : audit [INF] from='client.48074 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi161-61726-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:43.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[19769]: audit 2024-02-13T03:28:41.519504+0000 mon.a (mon.0) 1321 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:43.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[19769]: audit 2024-02-13T03:28:41.520212+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi161-61719-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:43.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[19769]: cluster 2024-02-13T03:28:41.520624+0000 mon.a (mon.0) 1323 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-02-13T03:28:43.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[19769]: audit 2024-02-13T03:28:41.528864+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi161-61719-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:43.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[19769]: audit 2024-02-13T03:28:41.551858+0000 mon.a (mon.0) 1325 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:43.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[19769]: audit 2024-02-13T03:28:41.553245+0000 mon.b (mon.1) 168 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:43.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[19769]: audit 2024-02-13T03:28:41.622724+0000 mon.c (mon.2) 322 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:43.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[27879]: audit 2024-02-13T03:28:41.518822+0000 mon.a (mon.0) 1320 : audit [INF] from='client.48074 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi161-61726-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:43.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[27879]: audit 2024-02-13T03:28:41.519504+0000 mon.a (mon.0) 1321 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:43.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[27879]: audit 2024-02-13T03:28:41.520212+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi161-61719-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:43.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[27879]: cluster 2024-02-13T03:28:41.520624+0000 mon.a (mon.0) 1323 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-02-13T03:28:43.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[27879]: audit 2024-02-13T03:28:41.528864+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi161-61719-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:43.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[27879]: audit 2024-02-13T03:28:41.551858+0000 mon.a (mon.0) 1325 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:43.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[27879]: audit 2024-02-13T03:28:41.553245+0000 mon.b (mon.1) 168 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:43.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:42 smithi161 bash[27879]: audit 2024-02-13T03:28:41.622724+0000 mon.c (mon.2) 322 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:44.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: audit 2024-02-13T03:28:42.624151+0000 mon.c (mon.2) 323 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:44.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: audit 2024-02-13T03:28:42.945802+0000 mon.a (mon.0) 1326 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:28:44.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: cluster 2024-02-13T03:28:42.946193+0000 mon.a (mon.0) 1327 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-02-13T03:28:44.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: audit 2024-02-13T03:28:42.948339+0000 mon.a (mon.0) 1328 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-17"}]: dispatch 2024-02-13T03:28:44.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: audit 2024-02-13T03:28:42.950667+0000 mon.b (mon.1) 169 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-17"}]: dispatch 2024-02-13T03:28:44.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: cluster 2024-02-13T03:28:43.252881+0000 mgr.y (mgr.24473) 319 : cluster [DBG] pgmap v326: 260 pgs: 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:44.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: audit 2024-02-13T03:28:43.496237+0000 mon.a (mon.0) 1329 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi161-61719-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi161-61719-29"}]': finished 2024-02-13T03:28:44.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: audit 2024-02-13T03:28:43.496337+0000 mon.a (mon.0) 1330 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-17"}]': finished 2024-02-13T03:28:44.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: cluster 2024-02-13T03:28:43.496431+0000 mon.a (mon.0) 1331 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-02-13T03:28:44.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: audit 2024-02-13T03:28:43.501279+0000 mon.c (mon.2) 324 : audit [INF] from='client.? v1:172.21.15.161:0/2364906163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi161-61726-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:44.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: audit 2024-02-13T03:28:43.501509+0000 mon.b (mon.1) 170 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-17", "mode": "writeback"}]: dispatch 2024-02-13T03:28:44.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: audit 2024-02-13T03:28:43.504622+0000 mon.a (mon.0) 1332 : audit [INF] from='client.48086 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi161-61726-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:44.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: audit 2024-02-13T03:28:43.504916+0000 mon.a (mon.0) 1333 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-17", "mode": "writeback"}]: dispatch 2024-02-13T03:28:44.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:43 smithi183 bash[20824]: audit 2024-02-13T03:28:43.625560+0000 mon.c (mon.2) 325 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:44.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: audit 2024-02-13T03:28:42.624151+0000 mon.c (mon.2) 323 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:44.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: audit 2024-02-13T03:28:42.945802+0000 mon.a (mon.0) 1326 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:28:44.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: cluster 2024-02-13T03:28:42.946193+0000 mon.a (mon.0) 1327 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-02-13T03:28:44.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: audit 2024-02-13T03:28:42.948339+0000 mon.a (mon.0) 1328 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-17"}]: dispatch 2024-02-13T03:28:44.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: audit 2024-02-13T03:28:42.950667+0000 mon.b (mon.1) 169 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-17"}]: dispatch 2024-02-13T03:28:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: cluster 2024-02-13T03:28:43.252881+0000 mgr.y (mgr.24473) 319 : cluster [DBG] pgmap v326: 260 pgs: 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: audit 2024-02-13T03:28:43.496237+0000 mon.a (mon.0) 1329 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi161-61719-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi161-61719-29"}]': finished 2024-02-13T03:28:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: audit 2024-02-13T03:28:43.496337+0000 mon.a (mon.0) 1330 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-17"}]': finished 2024-02-13T03:28:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: cluster 2024-02-13T03:28:43.496431+0000 mon.a (mon.0) 1331 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-02-13T03:28:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: audit 2024-02-13T03:28:43.501279+0000 mon.c (mon.2) 324 : audit [INF] from='client.? v1:172.21.15.161:0/2364906163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi161-61726-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: audit 2024-02-13T03:28:43.501509+0000 mon.b (mon.1) 170 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-17", "mode": "writeback"}]: dispatch 2024-02-13T03:28:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: audit 2024-02-13T03:28:43.504622+0000 mon.a (mon.0) 1332 : audit [INF] from='client.48086 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi161-61726-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: audit 2024-02-13T03:28:43.504916+0000 mon.a (mon.0) 1333 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-17", "mode": "writeback"}]: dispatch 2024-02-13T03:28:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[27879]: audit 2024-02-13T03:28:43.625560+0000 mon.c (mon.2) 325 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:44.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: audit 2024-02-13T03:28:42.624151+0000 mon.c (mon.2) 323 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:44.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: audit 2024-02-13T03:28:42.945802+0000 mon.a (mon.0) 1326 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:28:44.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: cluster 2024-02-13T03:28:42.946193+0000 mon.a (mon.0) 1327 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-02-13T03:28:44.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: audit 2024-02-13T03:28:42.948339+0000 mon.a (mon.0) 1328 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-17"}]: dispatch 2024-02-13T03:28:44.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: audit 2024-02-13T03:28:42.950667+0000 mon.b (mon.1) 169 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-17"}]: dispatch 2024-02-13T03:28:44.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: cluster 2024-02-13T03:28:43.252881+0000 mgr.y (mgr.24473) 319 : cluster [DBG] pgmap v326: 260 pgs: 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:44.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: audit 2024-02-13T03:28:43.496237+0000 mon.a (mon.0) 1329 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi161-61719-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi161-61719-29"}]': finished 2024-02-13T03:28:44.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: audit 2024-02-13T03:28:43.496337+0000 mon.a (mon.0) 1330 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-17"}]': finished 2024-02-13T03:28:44.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: cluster 2024-02-13T03:28:43.496431+0000 mon.a (mon.0) 1331 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-02-13T03:28:44.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: audit 2024-02-13T03:28:43.501279+0000 mon.c (mon.2) 324 : audit [INF] from='client.? v1:172.21.15.161:0/2364906163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi161-61726-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:44.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: audit 2024-02-13T03:28:43.501509+0000 mon.b (mon.1) 170 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-17", "mode": "writeback"}]: dispatch 2024-02-13T03:28:44.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: audit 2024-02-13T03:28:43.504622+0000 mon.a (mon.0) 1332 : audit [INF] from='client.48086 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi161-61726-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:44.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: audit 2024-02-13T03:28:43.504916+0000 mon.a (mon.0) 1333 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-17", "mode": "writeback"}]: dispatch 2024-02-13T03:28:44.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:43 smithi161 bash[19769]: audit 2024-02-13T03:28:43.625560+0000 mon.c (mon.2) 325 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:45.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:44 smithi183 bash[20824]: cluster 2024-02-13T03:28:44.489192+0000 mon.a (mon.0) 1334 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:28:45.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:44 smithi183 bash[20824]: audit 2024-02-13T03:28:44.499751+0000 mon.a (mon.0) 1335 : audit [INF] from='client.48086 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi161-61726-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:45.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:44 smithi183 bash[20824]: audit 2024-02-13T03:28:44.500005+0000 mon.a (mon.0) 1336 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-17", "mode": "writeback"}]': finished 2024-02-13T03:28:45.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:44 smithi183 bash[20824]: cluster 2024-02-13T03:28:44.500539+0000 mon.a (mon.0) 1337 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-02-13T03:28:45.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:44 smithi183 bash[20824]: audit 2024-02-13T03:28:44.572724+0000 mon.a (mon.0) 1338 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:45.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:44 smithi183 bash[20824]: audit 2024-02-13T03:28:44.575552+0000 mon.b (mon.1) 171 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:45.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:44 smithi183 bash[20824]: audit 2024-02-13T03:28:44.627148+0000 mon.c (mon.2) 326 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[27879]: cluster 2024-02-13T03:28:44.489192+0000 mon.a (mon.0) 1334 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:28:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[27879]: audit 2024-02-13T03:28:44.499751+0000 mon.a (mon.0) 1335 : audit [INF] from='client.48086 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi161-61726-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[27879]: audit 2024-02-13T03:28:44.500005+0000 mon.a (mon.0) 1336 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-17", "mode": "writeback"}]': finished 2024-02-13T03:28:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[27879]: cluster 2024-02-13T03:28:44.500539+0000 mon.a (mon.0) 1337 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-02-13T03:28:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[27879]: audit 2024-02-13T03:28:44.572724+0000 mon.a (mon.0) 1338 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[27879]: audit 2024-02-13T03:28:44.575552+0000 mon.b (mon.1) 171 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[27879]: audit 2024-02-13T03:28:44.627148+0000 mon.c (mon.2) 326 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:45.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[19769]: cluster 2024-02-13T03:28:44.489192+0000 mon.a (mon.0) 1334 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:28:45.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[19769]: audit 2024-02-13T03:28:44.499751+0000 mon.a (mon.0) 1335 : audit [INF] from='client.48086 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi161-61726-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:45.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[19769]: audit 2024-02-13T03:28:44.500005+0000 mon.a (mon.0) 1336 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-17", "mode": "writeback"}]': finished 2024-02-13T03:28:45.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[19769]: cluster 2024-02-13T03:28:44.500539+0000 mon.a (mon.0) 1337 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-02-13T03:28:45.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[19769]: audit 2024-02-13T03:28:44.572724+0000 mon.a (mon.0) 1338 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:45.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[19769]: audit 2024-02-13T03:28:44.575552+0000 mon.b (mon.1) 171 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:45.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:44 smithi161 bash[19769]: audit 2024-02-13T03:28:44.627148+0000 mon.c (mon.2) 326 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:46.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:28:45 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:28:46.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:45 smithi183 bash[20824]: cluster 2024-02-13T03:28:45.253798+0000 mgr.y (mgr.24473) 320 : cluster [DBG] pgmap v329: 300 pgs: 40 unknown, 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:46.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:45 smithi183 bash[20824]: audit 2024-02-13T03:28:45.500807+0000 mon.a (mon.0) 1339 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:28:46.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:45 smithi183 bash[20824]: cluster 2024-02-13T03:28:45.501348+0000 mon.a (mon.0) 1340 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-02-13T03:28:46.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:45 smithi183 bash[20824]: audit 2024-02-13T03:28:45.505464+0000 mon.a (mon.0) 1341 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17"}]: dispatch 2024-02-13T03:28:46.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:45 smithi183 bash[20824]: audit 2024-02-13T03:28:45.505872+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:46.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:45 smithi183 bash[20824]: audit 2024-02-13T03:28:45.507336+0000 mon.b (mon.1) 172 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17"}]: dispatch 2024-02-13T03:28:46.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:45 smithi161 bash[19769]: cluster 2024-02-13T03:28:45.253798+0000 mgr.y (mgr.24473) 320 : cluster [DBG] pgmap v329: 300 pgs: 40 unknown, 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:46.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:45 smithi161 bash[19769]: audit 2024-02-13T03:28:45.500807+0000 mon.a (mon.0) 1339 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:28:46.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:45 smithi161 bash[19769]: cluster 2024-02-13T03:28:45.501348+0000 mon.a (mon.0) 1340 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-02-13T03:28:46.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:45 smithi161 bash[19769]: audit 2024-02-13T03:28:45.505464+0000 mon.a (mon.0) 1341 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17"}]: dispatch 2024-02-13T03:28:46.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:45 smithi161 bash[19769]: audit 2024-02-13T03:28:45.505872+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:46.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:45 smithi161 bash[19769]: audit 2024-02-13T03:28:45.507336+0000 mon.b (mon.1) 172 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17"}]: dispatch 2024-02-13T03:28:46.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:45 smithi161 bash[27879]: cluster 2024-02-13T03:28:45.253798+0000 mgr.y (mgr.24473) 320 : cluster [DBG] pgmap v329: 300 pgs: 40 unknown, 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:46.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:45 smithi161 bash[27879]: audit 2024-02-13T03:28:45.500807+0000 mon.a (mon.0) 1339 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:28:46.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:45 smithi161 bash[27879]: cluster 2024-02-13T03:28:45.501348+0000 mon.a (mon.0) 1340 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-02-13T03:28:46.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:45 smithi161 bash[27879]: audit 2024-02-13T03:28:45.505464+0000 mon.a (mon.0) 1341 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17"}]: dispatch 2024-02-13T03:28:46.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:45 smithi161 bash[27879]: audit 2024-02-13T03:28:45.505872+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:46.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:45 smithi161 bash[27879]: audit 2024-02-13T03:28:45.507336+0000 mon.b (mon.1) 172 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17"}]: dispatch 2024-02-13T03:28:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:46 smithi183 bash[20824]: audit 2024-02-13T03:28:45.628644+0000 mon.c (mon.2) 327 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:46 smithi183 bash[20824]: audit 2024-02-13T03:28:45.883511+0000 mgr.y (mgr.24473) 321 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:46 smithi183 bash[20824]: cluster 2024-02-13T03:28:46.495226+0000 mon.a (mon.0) 1343 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:28:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:46 smithi183 bash[20824]: audit 2024-02-13T03:28:46.501203+0000 mon.a (mon.0) 1344 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17"}]': finished 2024-02-13T03:28:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:46 smithi183 bash[20824]: audit 2024-02-13T03:28:46.501322+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi161-61719-29"}]': finished 2024-02-13T03:28:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:46 smithi183 bash[20824]: cluster 2024-02-13T03:28:46.501419+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-02-13T03:28:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:46 smithi183 bash[20824]: audit 2024-02-13T03:28:46.502920+0000 mon.a (mon.0) 1347 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:46 smithi183 bash[20824]: audit 2024-02-13T03:28:46.510864+0000 mon.c (mon.2) 328 : audit [INF] from='client.? v1:172.21.15.161:0/4244480777' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi161-61726-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:46 smithi183 bash[20824]: audit 2024-02-13T03:28:46.511569+0000 mon.a (mon.0) 1348 : audit [INF] from='client.48092 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi161-61726-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:46 smithi183 bash[20824]: audit 2024-02-13T03:28:46.630167+0000 mon.c (mon.2) 329 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[27879]: audit 2024-02-13T03:28:45.628644+0000 mon.c (mon.2) 327 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[27879]: audit 2024-02-13T03:28:45.883511+0000 mgr.y (mgr.24473) 321 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[27879]: cluster 2024-02-13T03:28:46.495226+0000 mon.a (mon.0) 1343 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:28:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[27879]: audit 2024-02-13T03:28:46.501203+0000 mon.a (mon.0) 1344 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17"}]': finished 2024-02-13T03:28:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[27879]: audit 2024-02-13T03:28:46.501322+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi161-61719-29"}]': finished 2024-02-13T03:28:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[27879]: cluster 2024-02-13T03:28:46.501419+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-02-13T03:28:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[27879]: audit 2024-02-13T03:28:46.502920+0000 mon.a (mon.0) 1347 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[27879]: audit 2024-02-13T03:28:46.510864+0000 mon.c (mon.2) 328 : audit [INF] from='client.? v1:172.21.15.161:0/4244480777' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi161-61726-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:47.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[27879]: audit 2024-02-13T03:28:46.511569+0000 mon.a (mon.0) 1348 : audit [INF] from='client.48092 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi161-61726-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:47.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[27879]: audit 2024-02-13T03:28:46.630167+0000 mon.c (mon.2) 329 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[19769]: audit 2024-02-13T03:28:45.628644+0000 mon.c (mon.2) 327 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[19769]: audit 2024-02-13T03:28:45.883511+0000 mgr.y (mgr.24473) 321 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[19769]: cluster 2024-02-13T03:28:46.495226+0000 mon.a (mon.0) 1343 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:28:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[19769]: audit 2024-02-13T03:28:46.501203+0000 mon.a (mon.0) 1344 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-17"}]': finished 2024-02-13T03:28:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[19769]: audit 2024-02-13T03:28:46.501322+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi161-61719-29"}]': finished 2024-02-13T03:28:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[19769]: cluster 2024-02-13T03:28:46.501419+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-02-13T03:28:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[19769]: audit 2024-02-13T03:28:46.502920+0000 mon.a (mon.0) 1347 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi161-61719-29"}]: dispatch 2024-02-13T03:28:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[19769]: audit 2024-02-13T03:28:46.510864+0000 mon.c (mon.2) 328 : audit [INF] from='client.? v1:172.21.15.161:0/4244480777' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi161-61726-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[19769]: audit 2024-02-13T03:28:46.511569+0000 mon.a (mon.0) 1348 : audit [INF] from='client.48092 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi161-61726-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:47.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:46 smithi161 bash[19769]: audit 2024-02-13T03:28:46.630167+0000 mon.c (mon.2) 329 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:47 smithi161 bash[19769]: cluster 2024-02-13T03:28:47.254693+0000 mgr.y (mgr.24473) 322 : cluster [DBG] pgmap v332: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-13T03:28:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:47 smithi161 bash[19769]: audit 2024-02-13T03:28:47.508438+0000 mon.a (mon.0) 1349 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi161-61719-29"}]': finished 2024-02-13T03:28:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:47 smithi161 bash[19769]: audit 2024-02-13T03:28:47.508854+0000 mon.a (mon.0) 1350 : audit [INF] from='client.48092 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi161-61726-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:47 smithi161 bash[19769]: cluster 2024-02-13T03:28:47.509058+0000 mon.a (mon.0) 1351 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-02-13T03:28:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:47 smithi161 bash[19769]: audit 2024-02-13T03:28:47.631715+0000 mon.c (mon.2) 330 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:47 smithi161 bash[27879]: cluster 2024-02-13T03:28:47.254693+0000 mgr.y (mgr.24473) 322 : cluster [DBG] pgmap v332: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-13T03:28:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:47 smithi161 bash[27879]: audit 2024-02-13T03:28:47.508438+0000 mon.a (mon.0) 1349 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi161-61719-29"}]': finished 2024-02-13T03:28:48.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:47 smithi161 bash[27879]: audit 2024-02-13T03:28:47.508854+0000 mon.a (mon.0) 1350 : audit [INF] from='client.48092 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi161-61726-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:48.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:47 smithi161 bash[27879]: cluster 2024-02-13T03:28:47.509058+0000 mon.a (mon.0) 1351 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-02-13T03:28:48.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:47 smithi161 bash[27879]: audit 2024-02-13T03:28:47.631715+0000 mon.c (mon.2) 330 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:47 smithi183 bash[20824]: cluster 2024-02-13T03:28:47.254693+0000 mgr.y (mgr.24473) 322 : cluster [DBG] pgmap v332: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-13T03:28:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:47 smithi183 bash[20824]: audit 2024-02-13T03:28:47.508438+0000 mon.a (mon.0) 1349 : audit [INF] from='client.? v1:172.21.15.161:0/3442757849' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi161-61719-29"}]': finished 2024-02-13T03:28:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:47 smithi183 bash[20824]: audit 2024-02-13T03:28:47.508854+0000 mon.a (mon.0) 1350 : audit [INF] from='client.48092 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi161-61726-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:47 smithi183 bash[20824]: cluster 2024-02-13T03:28:47.509058+0000 mon.a (mon.0) 1351 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-02-13T03:28:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:47 smithi183 bash[20824]: audit 2024-02-13T03:28:47.631715+0000 mon.c (mon.2) 330 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:49.330 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:48 smithi183 bash[20824]: audit 2024-02-13T03:28:47.987644+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:49.330 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:48 smithi183 bash[20824]: audit 2024-02-13T03:28:47.991453+0000 mon.a (mon.0) 1353 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:49.330 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:48 smithi183 bash[20824]: audit 2024-02-13T03:28:47.992012+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi161-61719-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:49.330 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:48 smithi183 bash[20824]: audit 2024-02-13T03:28:48.508471+0000 mon.a (mon.0) 1355 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi161-61719-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:49.330 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:48 smithi183 bash[20824]: cluster 2024-02-13T03:28:48.508605+0000 mon.a (mon.0) 1356 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-02-13T03:28:49.330 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:48 smithi183 bash[20824]: audit 2024-02-13T03:28:48.514916+0000 mon.a (mon.0) 1357 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi161-61719-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:49.330 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:48 smithi183 bash[20824]: audit 2024-02-13T03:28:48.517431+0000 mon.a (mon.0) 1358 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:49.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:48 smithi183 bash[20824]: audit 2024-02-13T03:28:48.520087+0000 mon.b (mon.1) 173 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:49.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:48 smithi183 bash[20824]: audit 2024-02-13T03:28:48.633089+0000 mon.c (mon.2) 331 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:49.331 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:28:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:28:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:28:49.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[19769]: audit 2024-02-13T03:28:47.987644+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:49.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[19769]: audit 2024-02-13T03:28:47.991453+0000 mon.a (mon.0) 1353 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:49.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[19769]: audit 2024-02-13T03:28:47.992012+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi161-61719-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:49.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[19769]: audit 2024-02-13T03:28:48.508471+0000 mon.a (mon.0) 1355 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi161-61719-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:49.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[19769]: cluster 2024-02-13T03:28:48.508605+0000 mon.a (mon.0) 1356 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-02-13T03:28:49.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[19769]: audit 2024-02-13T03:28:48.514916+0000 mon.a (mon.0) 1357 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi161-61719-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:49.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[19769]: audit 2024-02-13T03:28:48.517431+0000 mon.a (mon.0) 1358 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:49.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[19769]: audit 2024-02-13T03:28:48.520087+0000 mon.b (mon.1) 173 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:49.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[19769]: audit 2024-02-13T03:28:48.633089+0000 mon.c (mon.2) 331 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[27879]: audit 2024-02-13T03:28:47.987644+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[27879]: audit 2024-02-13T03:28:47.991453+0000 mon.a (mon.0) 1353 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[27879]: audit 2024-02-13T03:28:47.992012+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi161-61719-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:49.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[27879]: audit 2024-02-13T03:28:48.508471+0000 mon.a (mon.0) 1355 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi161-61719-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:49.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[27879]: cluster 2024-02-13T03:28:48.508605+0000 mon.a (mon.0) 1356 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-02-13T03:28:49.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[27879]: audit 2024-02-13T03:28:48.514916+0000 mon.a (mon.0) 1357 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi161-61719-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:49.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[27879]: audit 2024-02-13T03:28:48.517431+0000 mon.a (mon.0) 1358 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:49.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[27879]: audit 2024-02-13T03:28:48.520087+0000 mon.b (mon.1) 173 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:49.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:48 smithi161 bash[27879]: audit 2024-02-13T03:28:48.633089+0000 mon.c (mon.2) 331 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:50.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[19769]: cluster 2024-02-13T03:28:49.255596+0000 mgr.y (mgr.24473) 323 : cluster [DBG] pgmap v335: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:28:50.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[19769]: audit 2024-02-13T03:28:49.634649+0000 mon.c (mon.2) 332 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:50.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[19769]: audit 2024-02-13T03:28:49.950989+0000 mon.a (mon.0) 1359 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:50.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[19769]: cluster 2024-02-13T03:28:49.951572+0000 mon.a (mon.0) 1360 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-02-13T03:28:50.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[19769]: audit 2024-02-13T03:28:49.952698+0000 mon.c (mon.2) 333 : audit [INF] from='client.? v1:172.21.15.161:0/1951181420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi161-61726-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:50.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[19769]: audit 2024-02-13T03:28:49.956595+0000 mon.b (mon.1) 174 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:50.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[19769]: audit 2024-02-13T03:28:49.958566+0000 mon.a (mon.0) 1361 : audit [INF] from='client.48104 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi161-61726-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:50.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[19769]: audit 2024-02-13T03:28:49.961435+0000 mon.a (mon.0) 1362 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[27879]: cluster 2024-02-13T03:28:49.255596+0000 mgr.y (mgr.24473) 323 : cluster [DBG] pgmap v335: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:28:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[27879]: audit 2024-02-13T03:28:49.634649+0000 mon.c (mon.2) 332 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[27879]: audit 2024-02-13T03:28:49.950989+0000 mon.a (mon.0) 1359 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[27879]: cluster 2024-02-13T03:28:49.951572+0000 mon.a (mon.0) 1360 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-02-13T03:28:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[27879]: audit 2024-02-13T03:28:49.952698+0000 mon.c (mon.2) 333 : audit [INF] from='client.? v1:172.21.15.161:0/1951181420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi161-61726-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[27879]: audit 2024-02-13T03:28:49.956595+0000 mon.b (mon.1) 174 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[27879]: audit 2024-02-13T03:28:49.958566+0000 mon.a (mon.0) 1361 : audit [INF] from='client.48104 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi161-61726-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:50 smithi161 bash[27879]: audit 2024-02-13T03:28:49.961435+0000 mon.a (mon.0) 1362 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:50.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:50 smithi183 bash[20824]: cluster 2024-02-13T03:28:49.255596+0000 mgr.y (mgr.24473) 323 : cluster [DBG] pgmap v335: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:28:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:50 smithi183 bash[20824]: audit 2024-02-13T03:28:49.634649+0000 mon.c (mon.2) 332 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:50 smithi183 bash[20824]: audit 2024-02-13T03:28:49.950989+0000 mon.a (mon.0) 1359 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:50 smithi183 bash[20824]: cluster 2024-02-13T03:28:49.951572+0000 mon.a (mon.0) 1360 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-02-13T03:28:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:50 smithi183 bash[20824]: audit 2024-02-13T03:28:49.952698+0000 mon.c (mon.2) 333 : audit [INF] from='client.? v1:172.21.15.161:0/1951181420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi161-61726-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:50 smithi183 bash[20824]: audit 2024-02-13T03:28:49.956595+0000 mon.b (mon.1) 174 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:50 smithi183 bash[20824]: audit 2024-02-13T03:28:49.958566+0000 mon.a (mon.0) 1361 : audit [INF] from='client.48104 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi161-61726-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:50 smithi183 bash[20824]: audit 2024-02-13T03:28:49.961435+0000 mon.a (mon.0) 1362 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:28:51] "GET /metrics HTTP/1.1" 200 189605 "" "Prometheus/2.33.4" 2024-02-13T03:28:51.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[27879]: audit 2024-02-13T03:28:50.520775+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi161-61719-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi161-61719-30"}]': finished 2024-02-13T03:28:51.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[27879]: audit 2024-02-13T03:28:50.521188+0000 mon.a (mon.0) 1364 : audit [INF] from='client.48104 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi161-61726-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:51.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[27879]: audit 2024-02-13T03:28:50.521483+0000 mon.a (mon.0) 1365 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:28:51.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[27879]: cluster 2024-02-13T03:28:50.522231+0000 mon.a (mon.0) 1366 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-02-13T03:28:51.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[27879]: audit 2024-02-13T03:28:50.530586+0000 mon.a (mon.0) 1367 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-19"}]: dispatch 2024-02-13T03:28:51.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[27879]: audit 2024-02-13T03:28:50.531501+0000 mon.b (mon.1) 175 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-19"}]: dispatch 2024-02-13T03:28:51.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[27879]: audit 2024-02-13T03:28:50.636257+0000 mon.c (mon.2) 334 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:51.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[27879]: cluster 2024-02-13T03:28:51.258226+0000 mgr.y (mgr.24473) 324 : cluster [DBG] pgmap v338: 300 pgs: 8 creating+peering, 292 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T03:28:51.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[19769]: audit 2024-02-13T03:28:50.520775+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi161-61719-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi161-61719-30"}]': finished 2024-02-13T03:28:51.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[19769]: audit 2024-02-13T03:28:50.521188+0000 mon.a (mon.0) 1364 : audit [INF] from='client.48104 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi161-61726-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:51.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[19769]: audit 2024-02-13T03:28:50.521483+0000 mon.a (mon.0) 1365 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:28:51.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[19769]: cluster 2024-02-13T03:28:50.522231+0000 mon.a (mon.0) 1366 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-02-13T03:28:51.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[19769]: audit 2024-02-13T03:28:50.530586+0000 mon.a (mon.0) 1367 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-19"}]: dispatch 2024-02-13T03:28:51.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[19769]: audit 2024-02-13T03:28:50.531501+0000 mon.b (mon.1) 175 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-19"}]: dispatch 2024-02-13T03:28:51.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[19769]: audit 2024-02-13T03:28:50.636257+0000 mon.c (mon.2) 334 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:51.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:51 smithi161 bash[19769]: cluster 2024-02-13T03:28:51.258226+0000 mgr.y (mgr.24473) 324 : cluster [DBG] pgmap v338: 300 pgs: 8 creating+peering, 292 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T03:28:51.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:51 smithi183 bash[20824]: audit 2024-02-13T03:28:50.520775+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi161-61719-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi161-61719-30"}]': finished 2024-02-13T03:28:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:51 smithi183 bash[20824]: audit 2024-02-13T03:28:50.521188+0000 mon.a (mon.0) 1364 : audit [INF] from='client.48104 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi161-61726-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:51 smithi183 bash[20824]: audit 2024-02-13T03:28:50.521483+0000 mon.a (mon.0) 1365 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:28:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:51 smithi183 bash[20824]: cluster 2024-02-13T03:28:50.522231+0000 mon.a (mon.0) 1366 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-02-13T03:28:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:51 smithi183 bash[20824]: audit 2024-02-13T03:28:50.530586+0000 mon.a (mon.0) 1367 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-19"}]: dispatch 2024-02-13T03:28:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:51 smithi183 bash[20824]: audit 2024-02-13T03:28:50.531501+0000 mon.b (mon.1) 175 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-19"}]: dispatch 2024-02-13T03:28:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:51 smithi183 bash[20824]: audit 2024-02-13T03:28:50.636257+0000 mon.c (mon.2) 334 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:51 smithi183 bash[20824]: cluster 2024-02-13T03:28:51.258226+0000 mgr.y (mgr.24473) 324 : cluster [DBG] pgmap v338: 300 pgs: 8 creating+peering, 292 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T03:28:52.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:52 smithi161 bash[27879]: audit 2024-02-13T03:28:51.522224+0000 mon.a (mon.0) 1368 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-19"}]': finished 2024-02-13T03:28:52.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:52 smithi161 bash[27879]: cluster 2024-02-13T03:28:51.522527+0000 mon.a (mon.0) 1369 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-02-13T03:28:52.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:52 smithi161 bash[27879]: audit 2024-02-13T03:28:51.530797+0000 mon.b (mon.1) 176 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-19", "mode": "writeback"}]: dispatch 2024-02-13T03:28:52.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:52 smithi161 bash[27879]: audit 2024-02-13T03:28:51.532369+0000 mon.a (mon.0) 1370 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-19", "mode": "writeback"}]: dispatch 2024-02-13T03:28:52.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:52 smithi161 bash[27879]: audit 2024-02-13T03:28:51.637811+0000 mon.c (mon.2) 335 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:52.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:52 smithi161 bash[27879]: cluster 2024-02-13T03:28:52.517740+0000 mon.a (mon.0) 1371 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:28:52.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:52 smithi161 bash[19769]: audit 2024-02-13T03:28:51.522224+0000 mon.a (mon.0) 1368 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-19"}]': finished 2024-02-13T03:28:52.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:52 smithi161 bash[19769]: cluster 2024-02-13T03:28:51.522527+0000 mon.a (mon.0) 1369 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-02-13T03:28:52.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:52 smithi161 bash[19769]: audit 2024-02-13T03:28:51.530797+0000 mon.b (mon.1) 176 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-19", "mode": "writeback"}]: dispatch 2024-02-13T03:28:52.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:52 smithi161 bash[19769]: audit 2024-02-13T03:28:51.532369+0000 mon.a (mon.0) 1370 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-19", "mode": "writeback"}]: dispatch 2024-02-13T03:28:52.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:52 smithi161 bash[19769]: audit 2024-02-13T03:28:51.637811+0000 mon.c (mon.2) 335 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:52.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:52 smithi161 bash[19769]: cluster 2024-02-13T03:28:52.517740+0000 mon.a (mon.0) 1371 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:28:52.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:52 smithi183 bash[20824]: audit 2024-02-13T03:28:51.522224+0000 mon.a (mon.0) 1368 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-19"}]': finished 2024-02-13T03:28:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:52 smithi183 bash[20824]: cluster 2024-02-13T03:28:51.522527+0000 mon.a (mon.0) 1369 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-02-13T03:28:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:52 smithi183 bash[20824]: audit 2024-02-13T03:28:51.530797+0000 mon.b (mon.1) 176 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-19", "mode": "writeback"}]: dispatch 2024-02-13T03:28:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:52 smithi183 bash[20824]: audit 2024-02-13T03:28:51.532369+0000 mon.a (mon.0) 1370 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-19", "mode": "writeback"}]: dispatch 2024-02-13T03:28:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:52 smithi183 bash[20824]: audit 2024-02-13T03:28:51.637811+0000 mon.c (mon.2) 335 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:52 smithi183 bash[20824]: cluster 2024-02-13T03:28:52.517740+0000 mon.a (mon.0) 1371 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:28:53.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: audit 2024-02-13T03:28:52.533512+0000 mon.a (mon.0) 1372 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-19", "mode": "writeback"}]': finished 2024-02-13T03:28:53.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: cluster 2024-02-13T03:28:52.534015+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-02-13T03:28:53.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: audit 2024-02-13T03:28:52.538098+0000 mon.c (mon.2) 336 : audit [INF] from='client.? v1:172.21.15.161:0/1692072800' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi161-61726-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:53.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: audit 2024-02-13T03:28:52.547399+0000 mon.a (mon.0) 1374 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:53.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: audit 2024-02-13T03:28:52.547933+0000 mon.a (mon.0) 1375 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi161-61726-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:53.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: audit 2024-02-13T03:28:52.566688+0000 mon.a (mon.0) 1376 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:53.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: audit 2024-02-13T03:28:52.569448+0000 mon.b (mon.1) 177 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:53.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: audit 2024-02-13T03:28:52.639281+0000 mon.c (mon.2) 337 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:53.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: cluster 2024-02-13T03:28:53.259107+0000 mgr.y (mgr.24473) 325 : cluster [DBG] pgmap v341: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:53.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: audit 2024-02-13T03:28:53.531427+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi161-61719-30"}]': finished 2024-02-13T03:28:53.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: audit 2024-02-13T03:28:53.531685+0000 mon.a (mon.0) 1378 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi161-61726-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:53.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: audit 2024-02-13T03:28:53.531987+0000 mon.a (mon.0) 1379 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:28:53.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: cluster 2024-02-13T03:28:53.532138+0000 mon.a (mon.0) 1380 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-02-13T03:28:53.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: audit 2024-02-13T03:28:53.537321+0000 mon.a (mon.0) 1381 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:53.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: audit 2024-02-13T03:28:53.537472+0000 mon.b (mon.1) 178 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19"}]: dispatch 2024-02-13T03:28:53.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[27879]: audit 2024-02-13T03:28:53.537776+0000 mon.a (mon.0) 1382 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19"}]: dispatch 2024-02-13T03:28:53.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: audit 2024-02-13T03:28:52.533512+0000 mon.a (mon.0) 1372 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-19", "mode": "writeback"}]': finished 2024-02-13T03:28:53.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: cluster 2024-02-13T03:28:52.534015+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-02-13T03:28:53.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: audit 2024-02-13T03:28:52.538098+0000 mon.c (mon.2) 336 : audit [INF] from='client.? v1:172.21.15.161:0/1692072800' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi161-61726-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:53.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: audit 2024-02-13T03:28:52.547399+0000 mon.a (mon.0) 1374 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:53.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: audit 2024-02-13T03:28:52.547933+0000 mon.a (mon.0) 1375 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi161-61726-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:53.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: audit 2024-02-13T03:28:52.566688+0000 mon.a (mon.0) 1376 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:53.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: audit 2024-02-13T03:28:52.569448+0000 mon.b (mon.1) 177 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:53.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: audit 2024-02-13T03:28:52.639281+0000 mon.c (mon.2) 337 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:53.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: cluster 2024-02-13T03:28:53.259107+0000 mgr.y (mgr.24473) 325 : cluster [DBG] pgmap v341: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:53.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: audit 2024-02-13T03:28:53.531427+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi161-61719-30"}]': finished 2024-02-13T03:28:53.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: audit 2024-02-13T03:28:53.531685+0000 mon.a (mon.0) 1378 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi161-61726-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:53.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: audit 2024-02-13T03:28:53.531987+0000 mon.a (mon.0) 1379 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:28:53.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: cluster 2024-02-13T03:28:53.532138+0000 mon.a (mon.0) 1380 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-02-13T03:28:53.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: audit 2024-02-13T03:28:53.537321+0000 mon.a (mon.0) 1381 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:53.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: audit 2024-02-13T03:28:53.537472+0000 mon.b (mon.1) 178 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19"}]: dispatch 2024-02-13T03:28:53.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:53 smithi161 bash[19769]: audit 2024-02-13T03:28:53.537776+0000 mon.a (mon.0) 1382 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19"}]: dispatch 2024-02-13T03:28:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: audit 2024-02-13T03:28:52.533512+0000 mon.a (mon.0) 1372 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-19", "mode": "writeback"}]': finished 2024-02-13T03:28:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: cluster 2024-02-13T03:28:52.534015+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-02-13T03:28:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: audit 2024-02-13T03:28:52.538098+0000 mon.c (mon.2) 336 : audit [INF] from='client.? v1:172.21.15.161:0/1692072800' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi161-61726-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: audit 2024-02-13T03:28:52.547399+0000 mon.a (mon.0) 1374 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: audit 2024-02-13T03:28:52.547933+0000 mon.a (mon.0) 1375 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi161-61726-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: audit 2024-02-13T03:28:52.566688+0000 mon.a (mon.0) 1376 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: audit 2024-02-13T03:28:52.569448+0000 mon.b (mon.1) 177 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:28:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: audit 2024-02-13T03:28:52.639281+0000 mon.c (mon.2) 337 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: cluster 2024-02-13T03:28:53.259107+0000 mgr.y (mgr.24473) 325 : cluster [DBG] pgmap v341: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:28:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: audit 2024-02-13T03:28:53.531427+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi161-61719-30"}]': finished 2024-02-13T03:28:53.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: audit 2024-02-13T03:28:53.531685+0000 mon.a (mon.0) 1378 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi161-61726-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:53.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: audit 2024-02-13T03:28:53.531987+0000 mon.a (mon.0) 1379 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:28:53.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: cluster 2024-02-13T03:28:53.532138+0000 mon.a (mon.0) 1380 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-02-13T03:28:53.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: audit 2024-02-13T03:28:53.537321+0000 mon.a (mon.0) 1381 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi161-61719-30"}]: dispatch 2024-02-13T03:28:53.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: audit 2024-02-13T03:28:53.537472+0000 mon.b (mon.1) 178 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19"}]: dispatch 2024-02-13T03:28:53.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:53 smithi183 bash[20824]: audit 2024-02-13T03:28:53.537776+0000 mon.a (mon.0) 1382 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19"}]: dispatch 2024-02-13T03:28:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:54 smithi161 bash[27879]: audit 2024-02-13T03:28:53.640753+0000 mon.c (mon.2) 338 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:54 smithi161 bash[27879]: cluster 2024-02-13T03:28:54.524443+0000 mon.a (mon.0) 1383 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:28:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:54 smithi161 bash[27879]: audit 2024-02-13T03:28:54.535754+0000 mon.a (mon.0) 1384 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi161-61719-30"}]': finished 2024-02-13T03:28:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:54 smithi161 bash[27879]: audit 2024-02-13T03:28:54.535925+0000 mon.a (mon.0) 1385 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19"}]': finished 2024-02-13T03:28:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:54 smithi161 bash[27879]: cluster 2024-02-13T03:28:54.536193+0000 mon.a (mon.0) 1386 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-02-13T03:28:54.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:54 smithi161 bash[19769]: audit 2024-02-13T03:28:53.640753+0000 mon.c (mon.2) 338 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:54.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:54 smithi161 bash[19769]: cluster 2024-02-13T03:28:54.524443+0000 mon.a (mon.0) 1383 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:28:54.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:54 smithi161 bash[19769]: audit 2024-02-13T03:28:54.535754+0000 mon.a (mon.0) 1384 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi161-61719-30"}]': finished 2024-02-13T03:28:54.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:54 smithi161 bash[19769]: audit 2024-02-13T03:28:54.535925+0000 mon.a (mon.0) 1385 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19"}]': finished 2024-02-13T03:28:54.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:54 smithi161 bash[19769]: cluster 2024-02-13T03:28:54.536193+0000 mon.a (mon.0) 1386 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-02-13T03:28:54.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:54 smithi183 bash[20824]: audit 2024-02-13T03:28:53.640753+0000 mon.c (mon.2) 338 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:54 smithi183 bash[20824]: cluster 2024-02-13T03:28:54.524443+0000 mon.a (mon.0) 1383 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:28:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:54 smithi183 bash[20824]: audit 2024-02-13T03:28:54.535754+0000 mon.a (mon.0) 1384 : audit [INF] from='client.? v1:172.21.15.161:0/593259387' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi161-61719-30"}]': finished 2024-02-13T03:28:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:54 smithi183 bash[20824]: audit 2024-02-13T03:28:54.535925+0000 mon.a (mon.0) 1385 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-19"}]': finished 2024-02-13T03:28:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:54 smithi183 bash[20824]: cluster 2024-02-13T03:28:54.536193+0000 mon.a (mon.0) 1386 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-02-13T03:28:55.897 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:55 smithi183 bash[20824]: audit 2024-02-13T03:28:54.573472+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:28:55.897 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:55 smithi183 bash[20824]: audit 2024-02-13T03:28:54.575630+0000 mon.a (mon.0) 1388 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:28:55.897 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:55 smithi183 bash[20824]: audit 2024-02-13T03:28:54.576210+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi161-61719-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:55.897 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:55 smithi183 bash[20824]: audit 2024-02-13T03:28:54.642320+0000 mon.c (mon.2) 339 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:55.897 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:55 smithi183 bash[20824]: cluster 2024-02-13T03:28:55.259943+0000 mgr.y (mgr.24473) 326 : cluster [DBG] pgmap v344: 260 pgs: 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:55.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:55 smithi161 bash[27879]: audit 2024-02-13T03:28:54.573472+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:28:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:55 smithi161 bash[27879]: audit 2024-02-13T03:28:54.575630+0000 mon.a (mon.0) 1388 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:28:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:55 smithi161 bash[27879]: audit 2024-02-13T03:28:54.576210+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi161-61719-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:55 smithi161 bash[27879]: audit 2024-02-13T03:28:54.642320+0000 mon.c (mon.2) 339 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:55 smithi161 bash[27879]: cluster 2024-02-13T03:28:55.259943+0000 mgr.y (mgr.24473) 326 : cluster [DBG] pgmap v344: 260 pgs: 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:55 smithi161 bash[19769]: audit 2024-02-13T03:28:54.573472+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:28:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:55 smithi161 bash[19769]: audit 2024-02-13T03:28:54.575630+0000 mon.a (mon.0) 1388 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:28:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:55 smithi161 bash[19769]: audit 2024-02-13T03:28:54.576210+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi161-61719-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:28:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:55 smithi161 bash[19769]: audit 2024-02-13T03:28:54.642320+0000 mon.c (mon.2) 339 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:55 smithi161 bash[19769]: cluster 2024-02-13T03:28:55.259943+0000 mgr.y (mgr.24473) 326 : cluster [DBG] pgmap v344: 260 pgs: 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:28:56.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:28:55 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:28:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[27879]: audit 2024-02-13T03:28:55.579004+0000 mon.a (mon.0) 1390 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi161-61719-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[27879]: cluster 2024-02-13T03:28:55.579493+0000 mon.a (mon.0) 1391 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-02-13T03:28:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[27879]: audit 2024-02-13T03:28:55.580769+0000 mon.a (mon.0) 1392 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi161-61719-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:28:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[27879]: audit 2024-02-13T03:28:55.588484+0000 mon.b (mon.1) 179 : audit [INF] from='client.? v1:172.21.15.161:0/657777876' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi161-61726-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[27879]: audit 2024-02-13T03:28:55.594169+0000 mon.a (mon.0) 1393 : audit [INF] from='client.48790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi161-61726-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[27879]: audit 2024-02-13T03:28:55.643610+0000 mon.c (mon.2) 340 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[27879]: audit 2024-02-13T03:28:55.644494+0000 mon.c (mon.2) 341 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]: dispatch 2024-02-13T03:28:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[27879]: audit 2024-02-13T03:28:55.894506+0000 mgr.y (mgr.24473) 327 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[27879]: audit 2024-02-13T03:28:56.014563+0000 mon.a (mon.0) 1394 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]: dispatch 2024-02-13T03:28:57.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[19769]: audit 2024-02-13T03:28:55.579004+0000 mon.a (mon.0) 1390 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi161-61719-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:57.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[19769]: cluster 2024-02-13T03:28:55.579493+0000 mon.a (mon.0) 1391 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-02-13T03:28:57.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[19769]: audit 2024-02-13T03:28:55.580769+0000 mon.a (mon.0) 1392 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi161-61719-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:28:57.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[19769]: audit 2024-02-13T03:28:55.588484+0000 mon.b (mon.1) 179 : audit [INF] from='client.? v1:172.21.15.161:0/657777876' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi161-61726-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:57.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[19769]: audit 2024-02-13T03:28:55.594169+0000 mon.a (mon.0) 1393 : audit [INF] from='client.48790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi161-61726-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:57.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[19769]: audit 2024-02-13T03:28:55.643610+0000 mon.c (mon.2) 340 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:57.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[19769]: audit 2024-02-13T03:28:55.644494+0000 mon.c (mon.2) 341 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]: dispatch 2024-02-13T03:28:57.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[19769]: audit 2024-02-13T03:28:55.894506+0000 mgr.y (mgr.24473) 327 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:57.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:57 smithi161 bash[19769]: audit 2024-02-13T03:28:56.014563+0000 mon.a (mon.0) 1394 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]: dispatch 2024-02-13T03:28:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:57 smithi183 bash[20824]: audit 2024-02-13T03:28:55.579004+0000 mon.a (mon.0) 1390 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi161-61719-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:28:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:57 smithi183 bash[20824]: cluster 2024-02-13T03:28:55.579493+0000 mon.a (mon.0) 1391 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-02-13T03:28:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:57 smithi183 bash[20824]: audit 2024-02-13T03:28:55.580769+0000 mon.a (mon.0) 1392 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi161-61719-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:28:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:57 smithi183 bash[20824]: audit 2024-02-13T03:28:55.588484+0000 mon.b (mon.1) 179 : audit [INF] from='client.? v1:172.21.15.161:0/657777876' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi161-61726-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:57 smithi183 bash[20824]: audit 2024-02-13T03:28:55.594169+0000 mon.a (mon.0) 1393 : audit [INF] from='client.48790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi161-61726-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:57 smithi183 bash[20824]: audit 2024-02-13T03:28:55.643610+0000 mon.c (mon.2) 340 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:57 smithi183 bash[20824]: audit 2024-02-13T03:28:55.644494+0000 mon.c (mon.2) 341 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]: dispatch 2024-02-13T03:28:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:57 smithi183 bash[20824]: audit 2024-02-13T03:28:55.894506+0000 mgr.y (mgr.24473) 327 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:28:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:57 smithi183 bash[20824]: audit 2024-02-13T03:28:56.014563+0000 mon.a (mon.0) 1394 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]: dispatch 2024-02-13T03:28:58.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.005245+0000 mon.a (mon.0) 1395 : audit [INF] from='client.48790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi161-61726-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:58.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.005665+0000 mon.a (mon.0) 1396 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]': finished 2024-02-13T03:28:58.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: cluster 2024-02-13T03:28:57.005910+0000 mon.a (mon.0) 1397 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-02-13T03:28:58.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.006778+0000 mon.c (mon.2) 342 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:58.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.008558+0000 mon.c (mon.2) 343 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:58.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.009390+0000 mon.c (mon.2) 344 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-13T03:28:58.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.010703+0000 mon.b (mon.1) 180 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:58.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.018219+0000 mon.a (mon.0) 1398 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-13T03:28:58.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.018783+0000 mon.a (mon.0) 1399 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:58.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: cluster 2024-02-13T03:28:57.260845+0000 mgr.y (mgr.24473) 328 : cluster [DBG] pgmap v347: 292 pgs: 32 unknown, 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:28:58.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.587607+0000 mon.a (mon.0) 1400 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi161-61719-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi161-61719-31"}]': finished 2024-02-13T03:28:58.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.587840+0000 mon.a (mon.0) 1401 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]': finished 2024-02-13T03:28:58.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.587943+0000 mon.a (mon.0) 1402 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:58.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: cluster 2024-02-13T03:28:57.588030+0000 mon.a (mon.0) 1403 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-02-13T03:28:58.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.589663+0000 mon.c (mon.2) 345 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.005245+0000 mon.a (mon.0) 1395 : audit [INF] from='client.48790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi161-61726-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.005665+0000 mon.a (mon.0) 1396 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]': finished 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: cluster 2024-02-13T03:28:57.005910+0000 mon.a (mon.0) 1397 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.006778+0000 mon.c (mon.2) 342 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.008558+0000 mon.c (mon.2) 343 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.009390+0000 mon.c (mon.2) 344 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.010703+0000 mon.b (mon.1) 180 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.018219+0000 mon.a (mon.0) 1398 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.018783+0000 mon.a (mon.0) 1399 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: cluster 2024-02-13T03:28:57.260845+0000 mgr.y (mgr.24473) 328 : cluster [DBG] pgmap v347: 292 pgs: 32 unknown, 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.587607+0000 mon.a (mon.0) 1400 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi161-61719-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi161-61719-31"}]': finished 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.587840+0000 mon.a (mon.0) 1401 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]': finished 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.587943+0000 mon.a (mon.0) 1402 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: cluster 2024-02-13T03:28:57.588030+0000 mon.a (mon.0) 1403 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-02-13T03:28:58.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.589663+0000 mon.c (mon.2) 345 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:58.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.620573+0000 mon.a (mon.0) 1404 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:58.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[27879]: audit 2024-02-13T03:28:57.623416+0000 mon.b (mon.1) 181 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:58.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.620573+0000 mon.a (mon.0) 1404 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:58.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:58 smithi161 bash[19769]: audit 2024-02-13T03:28:57.623416+0000 mon.b (mon.1) 181 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.005245+0000 mon.a (mon.0) 1395 : audit [INF] from='client.48790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi161-61726-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.005665+0000 mon.a (mon.0) 1396 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]': finished 2024-02-13T03:28:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: cluster 2024-02-13T03:28:57.005910+0000 mon.a (mon.0) 1397 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-02-13T03:28:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.006778+0000 mon.c (mon.2) 342 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.008558+0000 mon.c (mon.2) 343 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.009390+0000 mon.c (mon.2) 344 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-13T03:28:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.010703+0000 mon.b (mon.1) 180 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.018219+0000 mon.a (mon.0) 1398 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-13T03:28:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.018783+0000 mon.a (mon.0) 1399 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: cluster 2024-02-13T03:28:57.260845+0000 mgr.y (mgr.24473) 328 : cluster [DBG] pgmap v347: 292 pgs: 32 unknown, 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:28:58.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.587607+0000 mon.a (mon.0) 1400 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi161-61719-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi161-61719-31"}]': finished 2024-02-13T03:28:58.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.587840+0000 mon.a (mon.0) 1401 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]': finished 2024-02-13T03:28:58.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.587943+0000 mon.a (mon.0) 1402 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:28:58.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: cluster 2024-02-13T03:28:57.588030+0000 mon.a (mon.0) 1403 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-02-13T03:28:58.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.589663+0000 mon.c (mon.2) 345 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:58.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.620573+0000 mon.a (mon.0) 1404 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:58.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:58 smithi183 bash[20824]: audit 2024-02-13T03:28:57.623416+0000 mon.b (mon.1) 181 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:28:59.330 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:59 smithi183 bash[20824]: audit 2024-02-13T03:28:57.683494+0000 mon.c (mon.2) 346 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:59 smithi183 bash[20824]: audit 2024-02-13T03:28:57.683958+0000 mon.c (mon.2) 347 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]: dispatch 2024-02-13T03:28:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:59 smithi183 bash[20824]: audit 2024-02-13T03:28:58.068093+0000 mon.a (mon.0) 1405 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]: dispatch 2024-02-13T03:28:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:59 smithi183 bash[20824]: audit 2024-02-13T03:28:58.588904+0000 mon.a (mon.0) 1406 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:28:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:59 smithi183 bash[20824]: audit 2024-02-13T03:28:58.589193+0000 mon.a (mon.0) 1407 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]': finished 2024-02-13T03:28:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:59 smithi183 bash[20824]: cluster 2024-02-13T03:28:58.589360+0000 mon.a (mon.0) 1408 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-02-13T03:28:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:59 smithi183 bash[20824]: audit 2024-02-13T03:28:58.591128+0000 mon.c (mon.2) 348 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:59 smithi183 bash[20824]: audit 2024-02-13T03:28:58.598352+0000 mon.b (mon.1) 182 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-21"}]: dispatch 2024-02-13T03:28:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:59 smithi183 bash[20824]: audit 2024-02-13T03:28:58.598865+0000 mon.a (mon.0) 1409 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-21"}]: dispatch 2024-02-13T03:28:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:59 smithi183 bash[20824]: audit 2024-02-13T03:28:58.599474+0000 mon.b (mon.1) 183 : audit [INF] from='client.? v1:172.21.15.161:0/3987237085' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:28:59 smithi183 bash[20824]: audit 2024-02-13T03:28:58.601587+0000 mon.a (mon.0) 1410 : audit [INF] from='client.48796 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[27879]: audit 2024-02-13T03:28:57.683494+0000 mon.c (mon.2) 346 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[27879]: audit 2024-02-13T03:28:57.683958+0000 mon.c (mon.2) 347 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]: dispatch 2024-02-13T03:28:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[27879]: audit 2024-02-13T03:28:58.068093+0000 mon.a (mon.0) 1405 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]: dispatch 2024-02-13T03:28:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[27879]: audit 2024-02-13T03:28:58.588904+0000 mon.a (mon.0) 1406 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:28:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[27879]: audit 2024-02-13T03:28:58.589193+0000 mon.a (mon.0) 1407 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]': finished 2024-02-13T03:28:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[27879]: cluster 2024-02-13T03:28:58.589360+0000 mon.a (mon.0) 1408 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-02-13T03:28:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[27879]: audit 2024-02-13T03:28:58.591128+0000 mon.c (mon.2) 348 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[27879]: audit 2024-02-13T03:28:58.598352+0000 mon.b (mon.1) 182 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-21"}]: dispatch 2024-02-13T03:28:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[27879]: audit 2024-02-13T03:28:58.598865+0000 mon.a (mon.0) 1409 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-21"}]: dispatch 2024-02-13T03:28:59.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[27879]: audit 2024-02-13T03:28:58.599474+0000 mon.b (mon.1) 183 : audit [INF] from='client.? v1:172.21.15.161:0/3987237085' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:59.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[27879]: audit 2024-02-13T03:28:58.601587+0000 mon.a (mon.0) 1410 : audit [INF] from='client.48796 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[19769]: audit 2024-02-13T03:28:57.683494+0000 mon.c (mon.2) 346 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[19769]: audit 2024-02-13T03:28:57.683958+0000 mon.c (mon.2) 347 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]: dispatch 2024-02-13T03:28:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[19769]: audit 2024-02-13T03:28:58.068093+0000 mon.a (mon.0) 1405 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]: dispatch 2024-02-13T03:28:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[19769]: audit 2024-02-13T03:28:58.588904+0000 mon.a (mon.0) 1406 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:28:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[19769]: audit 2024-02-13T03:28:58.589193+0000 mon.a (mon.0) 1407 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pg_num","val":"11"}]': finished 2024-02-13T03:28:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[19769]: cluster 2024-02-13T03:28:58.589360+0000 mon.a (mon.0) 1408 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-02-13T03:28:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[19769]: audit 2024-02-13T03:28:58.591128+0000 mon.c (mon.2) 348 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:28:59.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[19769]: audit 2024-02-13T03:28:58.598352+0000 mon.b (mon.1) 182 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-21"}]: dispatch 2024-02-13T03:28:59.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[19769]: audit 2024-02-13T03:28:58.598865+0000 mon.a (mon.0) 1409 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-21"}]: dispatch 2024-02-13T03:28:59.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[19769]: audit 2024-02-13T03:28:58.599474+0000 mon.b (mon.1) 183 : audit [INF] from='client.? v1:172.21.15.161:0/3987237085' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:59.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:28:59 smithi161 bash[19769]: audit 2024-02-13T03:28:58.601587+0000 mon.a (mon.0) 1410 : audit [INF] from='client.48796 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:28:59.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:28:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:28:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:29:00.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[19769]: cluster 2024-02-13T03:28:59.261782+0000 mgr.y (mgr.24473) 329 : cluster [DBG] pgmap v350: 300 pgs: 72 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:00.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[19769]: audit 2024-02-13T03:28:59.590638+0000 mon.a (mon.0) 1411 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-21"}]': finished 2024-02-13T03:29:00.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[19769]: audit 2024-02-13T03:28:59.590825+0000 mon.a (mon.0) 1412 : audit [INF] from='client.48796 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:00.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[19769]: cluster 2024-02-13T03:28:59.591032+0000 mon.a (mon.0) 1413 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-02-13T03:29:00.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[19769]: audit 2024-02-13T03:28:59.592925+0000 mon.c (mon.2) 349 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:00.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[19769]: audit 2024-02-13T03:28:59.595975+0000 mon.b (mon.1) 184 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-21", "mode": "writeback"}]: dispatch 2024-02-13T03:29:00.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[19769]: audit 2024-02-13T03:28:59.596004+0000 mon.a (mon.0) 1414 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-21", "mode": "writeback"}]: dispatch 2024-02-13T03:29:00.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[19769]: audit 2024-02-13T03:28:59.596385+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:29:00.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[27879]: cluster 2024-02-13T03:28:59.261782+0000 mgr.y (mgr.24473) 329 : cluster [DBG] pgmap v350: 300 pgs: 72 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:00.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[27879]: audit 2024-02-13T03:28:59.590638+0000 mon.a (mon.0) 1411 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-21"}]': finished 2024-02-13T03:29:00.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[27879]: audit 2024-02-13T03:28:59.590825+0000 mon.a (mon.0) 1412 : audit [INF] from='client.48796 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:00.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[27879]: cluster 2024-02-13T03:28:59.591032+0000 mon.a (mon.0) 1413 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-02-13T03:29:00.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[27879]: audit 2024-02-13T03:28:59.592925+0000 mon.c (mon.2) 349 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:00.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[27879]: audit 2024-02-13T03:28:59.595975+0000 mon.b (mon.1) 184 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-21", "mode": "writeback"}]: dispatch 2024-02-13T03:29:00.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[27879]: audit 2024-02-13T03:28:59.596004+0000 mon.a (mon.0) 1414 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-21", "mode": "writeback"}]: dispatch 2024-02-13T03:29:00.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:00 smithi161 bash[27879]: audit 2024-02-13T03:28:59.596385+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:29:00.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:00 smithi183 bash[20824]: cluster 2024-02-13T03:28:59.261782+0000 mgr.y (mgr.24473) 329 : cluster [DBG] pgmap v350: 300 pgs: 72 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:00 smithi183 bash[20824]: audit 2024-02-13T03:28:59.590638+0000 mon.a (mon.0) 1411 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-21"}]': finished 2024-02-13T03:29:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:00 smithi183 bash[20824]: audit 2024-02-13T03:28:59.590825+0000 mon.a (mon.0) 1412 : audit [INF] from='client.48796 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:00 smithi183 bash[20824]: cluster 2024-02-13T03:28:59.591032+0000 mon.a (mon.0) 1413 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-02-13T03:29:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:00 smithi183 bash[20824]: audit 2024-02-13T03:28:59.592925+0000 mon.c (mon.2) 349 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:00 smithi183 bash[20824]: audit 2024-02-13T03:28:59.595975+0000 mon.b (mon.1) 184 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-21", "mode": "writeback"}]: dispatch 2024-02-13T03:29:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:00 smithi183 bash[20824]: audit 2024-02-13T03:28:59.596004+0000 mon.a (mon.0) 1414 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-21", "mode": "writeback"}]: dispatch 2024-02-13T03:29:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:00 smithi183 bash[20824]: audit 2024-02-13T03:28:59.596385+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:29:01.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:29:01] "GET /metrics HTTP/1.1" 200 189605 "" "Prometheus/2.33.4" 2024-02-13T03:29:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[27879]: cluster 2024-02-13T03:29:00.586489+0000 mon.a (mon.0) 1416 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[27879]: audit 2024-02-13T03:29:00.599080+0000 mon.a (mon.0) 1417 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-21", "mode": "writeback"}]': finished 2024-02-13T03:29:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[27879]: audit 2024-02-13T03:29:00.599278+0000 mon.a (mon.0) 1418 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi161-61719-31"}]': finished 2024-02-13T03:29:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[27879]: cluster 2024-02-13T03:29:00.600087+0000 mon.a (mon.0) 1419 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-02-13T03:29:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[27879]: audit 2024-02-13T03:29:00.602369+0000 mon.c (mon.2) 350 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[27879]: audit 2024-02-13T03:29:00.604003+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:29:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[27879]: audit 2024-02-13T03:29:00.604360+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? v1:172.21.15.161:0/2851588785' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:01.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[27879]: audit 2024-02-13T03:29:00.651736+0000 mon.a (mon.0) 1422 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:01.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[27879]: audit 2024-02-13T03:29:00.654551+0000 mon.b (mon.1) 185 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:01.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[19769]: cluster 2024-02-13T03:29:00.586489+0000 mon.a (mon.0) 1416 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:01.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[19769]: audit 2024-02-13T03:29:00.599080+0000 mon.a (mon.0) 1417 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-21", "mode": "writeback"}]': finished 2024-02-13T03:29:01.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[19769]: audit 2024-02-13T03:29:00.599278+0000 mon.a (mon.0) 1418 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi161-61719-31"}]': finished 2024-02-13T03:29:01.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[19769]: cluster 2024-02-13T03:29:00.600087+0000 mon.a (mon.0) 1419 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-02-13T03:29:01.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[19769]: audit 2024-02-13T03:29:00.602369+0000 mon.c (mon.2) 350 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:01.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[19769]: audit 2024-02-13T03:29:00.604003+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:29:01.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[19769]: audit 2024-02-13T03:29:00.604360+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? v1:172.21.15.161:0/2851588785' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:01.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[19769]: audit 2024-02-13T03:29:00.651736+0000 mon.a (mon.0) 1422 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:01.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:01 smithi161 bash[19769]: audit 2024-02-13T03:29:00.654551+0000 mon.b (mon.1) 185 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:01 smithi183 bash[20824]: cluster 2024-02-13T03:29:00.586489+0000 mon.a (mon.0) 1416 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:01 smithi183 bash[20824]: audit 2024-02-13T03:29:00.599080+0000 mon.a (mon.0) 1417 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-21", "mode": "writeback"}]': finished 2024-02-13T03:29:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:01 smithi183 bash[20824]: audit 2024-02-13T03:29:00.599278+0000 mon.a (mon.0) 1418 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi161-61719-31"}]': finished 2024-02-13T03:29:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:01 smithi183 bash[20824]: cluster 2024-02-13T03:29:00.600087+0000 mon.a (mon.0) 1419 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-02-13T03:29:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:01 smithi183 bash[20824]: audit 2024-02-13T03:29:00.602369+0000 mon.c (mon.2) 350 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:01 smithi183 bash[20824]: audit 2024-02-13T03:29:00.604003+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi161-61719-31"}]: dispatch 2024-02-13T03:29:01.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:01 smithi183 bash[20824]: audit 2024-02-13T03:29:00.604360+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? v1:172.21.15.161:0/2851588785' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:01.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:01 smithi183 bash[20824]: audit 2024-02-13T03:29:00.651736+0000 mon.a (mon.0) 1422 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:01.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:01 smithi183 bash[20824]: audit 2024-02-13T03:29:00.654551+0000 mon.b (mon.1) 185 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:02.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: cluster 2024-02-13T03:29:01.264536+0000 mgr.y (mgr.24473) 330 : cluster [DBG] pgmap v353: 324 pgs: 32 creating+peering, 292 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-13T03:29:02.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: audit 2024-02-13T03:29:01.602102+0000 mon.a (mon.0) 1423 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi161-61719-31"}]': finished 2024-02-13T03:29:02.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: audit 2024-02-13T03:29:01.602302+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? v1:172.21.15.161:0/2851588785' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:02.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: audit 2024-02-13T03:29:01.603026+0000 mon.a (mon.0) 1425 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:02.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: cluster 2024-02-13T03:29:01.603197+0000 mon.a (mon.0) 1426 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-02-13T03:29:02.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: audit 2024-02-13T03:29:01.603947+0000 mon.c (mon.2) 351 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:02.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: audit 2024-02-13T03:29:01.605860+0000 mon.a (mon.0) 1427 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21"}]: dispatch 2024-02-13T03:29:02.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: audit 2024-02-13T03:29:01.608064+0000 mon.b (mon.1) 186 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21"}]: dispatch 2024-02-13T03:29:02.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: audit 2024-02-13T03:29:01.636985+0000 mon.a (mon.0) 1428 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:02.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: audit 2024-02-13T03:29:01.638458+0000 mon.a (mon.0) 1429 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:02.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: audit 2024-02-13T03:29:01.639364+0000 mon.a (mon.0) 1430 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi161-61719-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:02.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: audit 2024-02-13T03:29:01.639795+0000 mon.b (mon.1) 187 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:02.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: audit 2024-02-13T03:29:01.641458+0000 mon.b (mon.1) 188 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:02.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[19769]: audit 2024-02-13T03:29:01.642261+0000 mon.b (mon.1) 189 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi161-61719-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:02.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: cluster 2024-02-13T03:29:01.264536+0000 mgr.y (mgr.24473) 330 : cluster [DBG] pgmap v353: 324 pgs: 32 creating+peering, 292 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-13T03:29:02.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: audit 2024-02-13T03:29:01.602102+0000 mon.a (mon.0) 1423 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi161-61719-31"}]': finished 2024-02-13T03:29:02.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: audit 2024-02-13T03:29:01.602302+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? v1:172.21.15.161:0/2851588785' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:02.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: audit 2024-02-13T03:29:01.603026+0000 mon.a (mon.0) 1425 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:02.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: cluster 2024-02-13T03:29:01.603197+0000 mon.a (mon.0) 1426 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-02-13T03:29:02.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: audit 2024-02-13T03:29:01.603947+0000 mon.c (mon.2) 351 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:02.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: audit 2024-02-13T03:29:01.605860+0000 mon.a (mon.0) 1427 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21"}]: dispatch 2024-02-13T03:29:02.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: audit 2024-02-13T03:29:01.608064+0000 mon.b (mon.1) 186 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21"}]: dispatch 2024-02-13T03:29:02.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: audit 2024-02-13T03:29:01.636985+0000 mon.a (mon.0) 1428 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:02.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: audit 2024-02-13T03:29:01.638458+0000 mon.a (mon.0) 1429 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:02.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: audit 2024-02-13T03:29:01.639364+0000 mon.a (mon.0) 1430 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi161-61719-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:02.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: audit 2024-02-13T03:29:01.639795+0000 mon.b (mon.1) 187 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:02.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: audit 2024-02-13T03:29:01.641458+0000 mon.b (mon.1) 188 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:02.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:02 smithi161 bash[27879]: audit 2024-02-13T03:29:01.642261+0000 mon.b (mon.1) 189 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi161-61719-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: cluster 2024-02-13T03:29:01.264536+0000 mgr.y (mgr.24473) 330 : cluster [DBG] pgmap v353: 324 pgs: 32 creating+peering, 292 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-13T03:29:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: audit 2024-02-13T03:29:01.602102+0000 mon.a (mon.0) 1423 : audit [INF] from='client.? v1:172.21.15.161:0/3359149189' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi161-61719-31"}]': finished 2024-02-13T03:29:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: audit 2024-02-13T03:29:01.602302+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? v1:172.21.15.161:0/2851588785' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: audit 2024-02-13T03:29:01.603026+0000 mon.a (mon.0) 1425 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: cluster 2024-02-13T03:29:01.603197+0000 mon.a (mon.0) 1426 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-02-13T03:29:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: audit 2024-02-13T03:29:01.603947+0000 mon.c (mon.2) 351 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: audit 2024-02-13T03:29:01.605860+0000 mon.a (mon.0) 1427 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21"}]: dispatch 2024-02-13T03:29:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: audit 2024-02-13T03:29:01.608064+0000 mon.b (mon.1) 186 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21"}]: dispatch 2024-02-13T03:29:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: audit 2024-02-13T03:29:01.636985+0000 mon.a (mon.0) 1428 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: audit 2024-02-13T03:29:01.638458+0000 mon.a (mon.0) 1429 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: audit 2024-02-13T03:29:01.639364+0000 mon.a (mon.0) 1430 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi161-61719-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: audit 2024-02-13T03:29:01.639795+0000 mon.b (mon.1) 187 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: audit 2024-02-13T03:29:01.641458+0000 mon.b (mon.1) 188 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:02 smithi183 bash[20824]: audit 2024-02-13T03:29:01.642261+0000 mon.b (mon.1) 189 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi161-61719-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:03.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[19769]: cluster 2024-02-13T03:29:02.593579+0000 mon.a (mon.0) 1431 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:03.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[19769]: audit 2024-02-13T03:29:02.606110+0000 mon.a (mon.0) 1432 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21"}]': finished 2024-02-13T03:29:03.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[19769]: audit 2024-02-13T03:29:02.606415+0000 mon.a (mon.0) 1433 : audit [INF] from='client.48808 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi161-61719-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:03.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[19769]: cluster 2024-02-13T03:29:02.606707+0000 mon.a (mon.0) 1434 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-02-13T03:29:03.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[19769]: audit 2024-02-13T03:29:02.607089+0000 mon.c (mon.2) 352 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:03.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[19769]: audit 2024-02-13T03:29:02.611695+0000 mon.b (mon.1) 190 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi161-61719-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:03.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[19769]: audit 2024-02-13T03:29:02.612861+0000 mon.c (mon.2) 353 : audit [INF] from='client.? v1:172.21.15.161:0/1764200715' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:03.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[19769]: audit 2024-02-13T03:29:02.619771+0000 mon.a (mon.0) 1435 : audit [INF] from='client.48140 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:03.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[19769]: audit 2024-02-13T03:29:02.620361+0000 mon.a (mon.0) 1436 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi161-61719-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:03.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[27879]: cluster 2024-02-13T03:29:02.593579+0000 mon.a (mon.0) 1431 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:03.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[27879]: audit 2024-02-13T03:29:02.606110+0000 mon.a (mon.0) 1432 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21"}]': finished 2024-02-13T03:29:03.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[27879]: audit 2024-02-13T03:29:02.606415+0000 mon.a (mon.0) 1433 : audit [INF] from='client.48808 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi161-61719-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:03.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[27879]: cluster 2024-02-13T03:29:02.606707+0000 mon.a (mon.0) 1434 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-02-13T03:29:03.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[27879]: audit 2024-02-13T03:29:02.607089+0000 mon.c (mon.2) 352 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:03.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[27879]: audit 2024-02-13T03:29:02.611695+0000 mon.b (mon.1) 190 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi161-61719-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:03.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[27879]: audit 2024-02-13T03:29:02.612861+0000 mon.c (mon.2) 353 : audit [INF] from='client.? v1:172.21.15.161:0/1764200715' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:03.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[27879]: audit 2024-02-13T03:29:02.619771+0000 mon.a (mon.0) 1435 : audit [INF] from='client.48140 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:03.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:03 smithi161 bash[27879]: audit 2024-02-13T03:29:02.620361+0000 mon.a (mon.0) 1436 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi161-61719-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:03.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:03 smithi183 bash[20824]: cluster 2024-02-13T03:29:02.593579+0000 mon.a (mon.0) 1431 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:03 smithi183 bash[20824]: audit 2024-02-13T03:29:02.606110+0000 mon.a (mon.0) 1432 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-21"}]': finished 2024-02-13T03:29:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:03 smithi183 bash[20824]: audit 2024-02-13T03:29:02.606415+0000 mon.a (mon.0) 1433 : audit [INF] from='client.48808 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi161-61719-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:03 smithi183 bash[20824]: cluster 2024-02-13T03:29:02.606707+0000 mon.a (mon.0) 1434 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-02-13T03:29:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:03 smithi183 bash[20824]: audit 2024-02-13T03:29:02.607089+0000 mon.c (mon.2) 352 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:03 smithi183 bash[20824]: audit 2024-02-13T03:29:02.611695+0000 mon.b (mon.1) 190 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi161-61719-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:03 smithi183 bash[20824]: audit 2024-02-13T03:29:02.612861+0000 mon.c (mon.2) 353 : audit [INF] from='client.? v1:172.21.15.161:0/1764200715' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:03 smithi183 bash[20824]: audit 2024-02-13T03:29:02.619771+0000 mon.a (mon.0) 1435 : audit [INF] from='client.48140 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:03 smithi183 bash[20824]: audit 2024-02-13T03:29:02.620361+0000 mon.a (mon.0) 1436 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi161-61719-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:04 smithi183 bash[20824]: cluster 2024-02-13T03:29:03.265512+0000 mgr.y (mgr.24473) 331 : cluster [DBG] pgmap v356: 356 pgs: 32 unknown, 32 creating+peering, 292 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-13T03:29:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:04 smithi183 bash[20824]: audit 2024-02-13T03:29:03.607306+0000 mon.a (mon.0) 1437 : audit [INF] from='client.48140 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:04 smithi183 bash[20824]: audit 2024-02-13T03:29:03.610242+0000 mon.c (mon.2) 354 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:04 smithi183 bash[20824]: cluster 2024-02-13T03:29:04.037171+0000 mon.a (mon.0) 1438 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-02-13T03:29:04.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:04 smithi161 bash[27879]: cluster 2024-02-13T03:29:03.265512+0000 mgr.y (mgr.24473) 331 : cluster [DBG] pgmap v356: 356 pgs: 32 unknown, 32 creating+peering, 292 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-13T03:29:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:04 smithi161 bash[27879]: audit 2024-02-13T03:29:03.607306+0000 mon.a (mon.0) 1437 : audit [INF] from='client.48140 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:04 smithi161 bash[27879]: audit 2024-02-13T03:29:03.610242+0000 mon.c (mon.2) 354 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:04 smithi161 bash[27879]: cluster 2024-02-13T03:29:04.037171+0000 mon.a (mon.0) 1438 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-02-13T03:29:04.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:04 smithi161 bash[19769]: cluster 2024-02-13T03:29:03.265512+0000 mgr.y (mgr.24473) 331 : cluster [DBG] pgmap v356: 356 pgs: 32 unknown, 32 creating+peering, 292 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-13T03:29:04.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:04 smithi161 bash[19769]: audit 2024-02-13T03:29:03.607306+0000 mon.a (mon.0) 1437 : audit [INF] from='client.48140 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi161-61726-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:04.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:04 smithi161 bash[19769]: audit 2024-02-13T03:29:03.610242+0000 mon.c (mon.2) 354 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:04.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:04 smithi161 bash[19769]: cluster 2024-02-13T03:29:04.037171+0000 mon.a (mon.0) 1438 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-02-13T03:29:05.908 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:05 smithi183 bash[20824]: audit 2024-02-13T03:29:04.610719+0000 mon.a (mon.0) 1439 : audit [INF] from='client.48808 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi161-61719-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi161-61719-32"}]': finished 2024-02-13T03:29:05.909 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:05 smithi183 bash[20824]: cluster 2024-02-13T03:29:04.611060+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-02-13T03:29:05.909 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:05 smithi183 bash[20824]: audit 2024-02-13T03:29:04.613621+0000 mon.c (mon.2) 355 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:05.909 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:05 smithi183 bash[20824]: audit 2024-02-13T03:29:04.623475+0000 mon.a (mon.0) 1441 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:05.909 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:05 smithi183 bash[20824]: audit 2024-02-13T03:29:04.624460+0000 mon.b (mon.1) 191 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:05.909 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:05 smithi183 bash[20824]: audit 2024-02-13T03:29:05.245718+0000 mon.c (mon.2) 356 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:29:05.909 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:05 smithi183 bash[20824]: cluster 2024-02-13T03:29:05.266300+0000 mgr.y (mgr.24473) 332 : cluster [DBG] pgmap v359: 332 pgs: 40 unknown, 32 creating+peering, 260 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:05.909 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:05 smithi183 bash[20824]: audit 2024-02-13T03:29:05.559712+0000 mon.c (mon.2) 357 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:29:05.909 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:05 smithi183 bash[20824]: audit 2024-02-13T03:29:05.561122+0000 mon.c (mon.2) 358 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:29:05.909 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:05 smithi183 bash[20824]: audit 2024-02-13T03:29:05.569760+0000 mon.a (mon.0) 1442 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:29:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[19769]: audit 2024-02-13T03:29:04.610719+0000 mon.a (mon.0) 1439 : audit [INF] from='client.48808 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi161-61719-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi161-61719-32"}]': finished 2024-02-13T03:29:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[19769]: cluster 2024-02-13T03:29:04.611060+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-02-13T03:29:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[19769]: audit 2024-02-13T03:29:04.613621+0000 mon.c (mon.2) 355 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[19769]: audit 2024-02-13T03:29:04.623475+0000 mon.a (mon.0) 1441 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[19769]: audit 2024-02-13T03:29:04.624460+0000 mon.b (mon.1) 191 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[19769]: audit 2024-02-13T03:29:05.245718+0000 mon.c (mon.2) 356 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:29:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[19769]: cluster 2024-02-13T03:29:05.266300+0000 mgr.y (mgr.24473) 332 : cluster [DBG] pgmap v359: 332 pgs: 40 unknown, 32 creating+peering, 260 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[19769]: audit 2024-02-13T03:29:05.559712+0000 mon.c (mon.2) 357 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:29:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[19769]: audit 2024-02-13T03:29:05.561122+0000 mon.c (mon.2) 358 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:29:05.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[19769]: audit 2024-02-13T03:29:05.569760+0000 mon.a (mon.0) 1442 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:29:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[27879]: audit 2024-02-13T03:29:04.610719+0000 mon.a (mon.0) 1439 : audit [INF] from='client.48808 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi161-61719-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi161-61719-32"}]': finished 2024-02-13T03:29:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[27879]: cluster 2024-02-13T03:29:04.611060+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-02-13T03:29:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[27879]: audit 2024-02-13T03:29:04.613621+0000 mon.c (mon.2) 355 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[27879]: audit 2024-02-13T03:29:04.623475+0000 mon.a (mon.0) 1441 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[27879]: audit 2024-02-13T03:29:04.624460+0000 mon.b (mon.1) 191 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[27879]: audit 2024-02-13T03:29:05.245718+0000 mon.c (mon.2) 356 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:29:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[27879]: cluster 2024-02-13T03:29:05.266300+0000 mgr.y (mgr.24473) 332 : cluster [DBG] pgmap v359: 332 pgs: 40 unknown, 32 creating+peering, 260 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[27879]: audit 2024-02-13T03:29:05.559712+0000 mon.c (mon.2) 357 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:29:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[27879]: audit 2024-02-13T03:29:05.561122+0000 mon.c (mon.2) 358 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:29:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:05 smithi161 bash[27879]: audit 2024-02-13T03:29:05.569760+0000 mon.a (mon.0) 1442 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:29:06.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:29:05 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:29:06.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:06 smithi161 bash[27879]: audit 2024-02-13T03:29:05.612410+0000 mon.a (mon.0) 1443 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:06.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:06 smithi161 bash[27879]: cluster 2024-02-13T03:29:05.612797+0000 mon.a (mon.0) 1444 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-02-13T03:29:06.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:06 smithi161 bash[27879]: audit 2024-02-13T03:29:05.616649+0000 mon.c (mon.2) 359 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:06.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:06 smithi161 bash[27879]: audit 2024-02-13T03:29:05.905888+0000 mgr.y (mgr.24473) 333 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:06.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:06 smithi161 bash[27879]: audit 2024-02-13T03:29:06.617944+0000 mon.c (mon.2) 360 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:06.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:06 smithi161 bash[19769]: audit 2024-02-13T03:29:05.612410+0000 mon.a (mon.0) 1443 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:06.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:06 smithi161 bash[19769]: cluster 2024-02-13T03:29:05.612797+0000 mon.a (mon.0) 1444 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-02-13T03:29:06.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:06 smithi161 bash[19769]: audit 2024-02-13T03:29:05.616649+0000 mon.c (mon.2) 359 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:06.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:06 smithi161 bash[19769]: audit 2024-02-13T03:29:05.905888+0000 mgr.y (mgr.24473) 333 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:06.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:06 smithi161 bash[19769]: audit 2024-02-13T03:29:06.617944+0000 mon.c (mon.2) 360 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:06.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:06 smithi183 bash[20824]: audit 2024-02-13T03:29:05.612410+0000 mon.a (mon.0) 1443 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:06.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:06 smithi183 bash[20824]: cluster 2024-02-13T03:29:05.612797+0000 mon.a (mon.0) 1444 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-02-13T03:29:06.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:06 smithi183 bash[20824]: audit 2024-02-13T03:29:05.616649+0000 mon.c (mon.2) 359 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:06.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:06 smithi183 bash[20824]: audit 2024-02-13T03:29:05.905888+0000 mgr.y (mgr.24473) 333 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:06.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:06 smithi183 bash[20824]: audit 2024-02-13T03:29:06.617944+0000 mon.c (mon.2) 360 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:07.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[27879]: cluster 2024-02-13T03:29:06.643525+0000 mon.a (mon.0) 1445 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-02-13T03:29:07.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[27879]: audit 2024-02-13T03:29:06.646886+0000 mon.a (mon.0) 1446 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:07.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[27879]: audit 2024-02-13T03:29:06.648850+0000 mon.b (mon.1) 192 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:07.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[27879]: audit 2024-02-13T03:29:06.673489+0000 mon.a (mon.0) 1447 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:07.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[27879]: audit 2024-02-13T03:29:06.676339+0000 mon.b (mon.1) 193 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:07.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[27879]: cluster 2024-02-13T03:29:07.267114+0000 mgr.y (mgr.24473) 334 : cluster [DBG] pgmap v362: 260 pgs: 7 creating+activating, 20 creating+peering, 233 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:29:07.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[27879]: audit 2024-02-13T03:29:07.619436+0000 mon.c (mon.2) 361 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:07.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[19769]: cluster 2024-02-13T03:29:06.643525+0000 mon.a (mon.0) 1445 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-02-13T03:29:07.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[19769]: audit 2024-02-13T03:29:06.646886+0000 mon.a (mon.0) 1446 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:07.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[19769]: audit 2024-02-13T03:29:06.648850+0000 mon.b (mon.1) 192 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:07.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[19769]: audit 2024-02-13T03:29:06.673489+0000 mon.a (mon.0) 1447 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:07.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[19769]: audit 2024-02-13T03:29:06.676339+0000 mon.b (mon.1) 193 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:07.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[19769]: cluster 2024-02-13T03:29:07.267114+0000 mgr.y (mgr.24473) 334 : cluster [DBG] pgmap v362: 260 pgs: 7 creating+activating, 20 creating+peering, 233 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:29:07.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:07 smithi161 bash[19769]: audit 2024-02-13T03:29:07.619436+0000 mon.c (mon.2) 361 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:07.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:07 smithi183 bash[20824]: cluster 2024-02-13T03:29:06.643525+0000 mon.a (mon.0) 1445 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-02-13T03:29:07.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:07 smithi183 bash[20824]: audit 2024-02-13T03:29:06.646886+0000 mon.a (mon.0) 1446 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:07.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:07 smithi183 bash[20824]: audit 2024-02-13T03:29:06.648850+0000 mon.b (mon.1) 192 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:07.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:07 smithi183 bash[20824]: audit 2024-02-13T03:29:06.673489+0000 mon.a (mon.0) 1447 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:07.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:07 smithi183 bash[20824]: audit 2024-02-13T03:29:06.676339+0000 mon.b (mon.1) 193 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:07.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:07 smithi183 bash[20824]: cluster 2024-02-13T03:29:07.267114+0000 mgr.y (mgr.24473) 334 : cluster [DBG] pgmap v362: 260 pgs: 7 creating+activating, 20 creating+peering, 233 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:29:07.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:07 smithi183 bash[20824]: audit 2024-02-13T03:29:07.619436+0000 mon.c (mon.2) 361 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:08.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:07.648536+0000 mon.a (mon.0) 1448 : audit [INF] from='client.48808 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]': finished 2024-02-13T03:29:08.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:07.648651+0000 mon.a (mon.0) 1449 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:08.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: cluster 2024-02-13T03:29:07.648788+0000 mon.a (mon.0) 1450 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-02-13T03:29:08.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:07.650604+0000 mon.c (mon.2) 362 : audit [INF] from='client.? v1:172.21.15.161:0/1166343282' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:08.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:07.653608+0000 mon.b (mon.1) 194 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:08.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:07.653997+0000 mon.b (mon.1) 195 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-23"}]: dispatch 2024-02-13T03:29:08.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:07.656291+0000 mon.a (mon.0) 1451 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:08.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:07.656816+0000 mon.a (mon.0) 1452 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:08.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:07.663265+0000 mon.a (mon.0) 1453 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-23"}]: dispatch 2024-02-13T03:29:08.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:08.620976+0000 mon.c (mon.2) 363 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:08.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:08.647317+0000 mon.a (mon.0) 1454 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:08.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:08.647699+0000 mon.a (mon.0) 1455 : audit [INF] from='client.48808 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]': finished 2024-02-13T03:29:08.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:08.647903+0000 mon.a (mon.0) 1456 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-23"}]': finished 2024-02-13T03:29:08.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: cluster 2024-02-13T03:29:08.648041+0000 mon.a (mon.0) 1457 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-02-13T03:29:08.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:08.654363+0000 mon.a (mon.0) 1458 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-23", "mode": "writeback"}]: dispatch 2024-02-13T03:29:08.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[27879]: audit 2024-02-13T03:29:08.656836+0000 mon.b (mon.1) 196 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-23", "mode": "writeback"}]: dispatch 2024-02-13T03:29:08.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:07.648536+0000 mon.a (mon.0) 1448 : audit [INF] from='client.48808 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]': finished 2024-02-13T03:29:08.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:07.648651+0000 mon.a (mon.0) 1449 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:08.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: cluster 2024-02-13T03:29:07.648788+0000 mon.a (mon.0) 1450 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-02-13T03:29:08.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:07.650604+0000 mon.c (mon.2) 362 : audit [INF] from='client.? v1:172.21.15.161:0/1166343282' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:08.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:07.653608+0000 mon.b (mon.1) 194 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:08.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:07.653997+0000 mon.b (mon.1) 195 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-23"}]: dispatch 2024-02-13T03:29:08.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:07.656291+0000 mon.a (mon.0) 1451 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:08.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:07.656816+0000 mon.a (mon.0) 1452 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:08.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:07.663265+0000 mon.a (mon.0) 1453 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-23"}]: dispatch 2024-02-13T03:29:08.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:08.620976+0000 mon.c (mon.2) 363 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:08.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:08.647317+0000 mon.a (mon.0) 1454 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:08.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:08.647699+0000 mon.a (mon.0) 1455 : audit [INF] from='client.48808 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]': finished 2024-02-13T03:29:08.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:08.647903+0000 mon.a (mon.0) 1456 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-23"}]': finished 2024-02-13T03:29:08.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: cluster 2024-02-13T03:29:08.648041+0000 mon.a (mon.0) 1457 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-02-13T03:29:08.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:08.654363+0000 mon.a (mon.0) 1458 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-23", "mode": "writeback"}]: dispatch 2024-02-13T03:29:08.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:08 smithi161 bash[19769]: audit 2024-02-13T03:29:08.656836+0000 mon.b (mon.1) 196 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-23", "mode": "writeback"}]: dispatch 2024-02-13T03:29:08.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:07.648536+0000 mon.a (mon.0) 1448 : audit [INF] from='client.48808 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi161-61719-32"}]': finished 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:07.648651+0000 mon.a (mon.0) 1449 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: cluster 2024-02-13T03:29:07.648788+0000 mon.a (mon.0) 1450 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:07.650604+0000 mon.c (mon.2) 362 : audit [INF] from='client.? v1:172.21.15.161:0/1166343282' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:07.653608+0000 mon.b (mon.1) 194 : audit [INF] from='client.? v1:172.21.15.161:0/4066725848' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:07.653997+0000 mon.b (mon.1) 195 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-23"}]: dispatch 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:07.656291+0000 mon.a (mon.0) 1451 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:07.656816+0000 mon.a (mon.0) 1452 : audit [INF] from='client.48808 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]: dispatch 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:07.663265+0000 mon.a (mon.0) 1453 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-23"}]: dispatch 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:08.620976+0000 mon.c (mon.2) 363 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:08.647317+0000 mon.a (mon.0) 1454 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:08.647699+0000 mon.a (mon.0) 1455 : audit [INF] from='client.48808 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi161-61719-32"}]': finished 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:08.647903+0000 mon.a (mon.0) 1456 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-23"}]': finished 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: cluster 2024-02-13T03:29:08.648041+0000 mon.a (mon.0) 1457 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-02-13T03:29:08.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:08.654363+0000 mon.a (mon.0) 1458 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-23", "mode": "writeback"}]: dispatch 2024-02-13T03:29:08.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:08 smithi183 bash[20824]: audit 2024-02-13T03:29:08.656836+0000 mon.b (mon.1) 196 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-23", "mode": "writeback"}]: dispatch 2024-02-13T03:29:09.672 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:29:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:29:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:29:09.672 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:09 smithi183 bash[20824]: audit 2024-02-13T03:29:08.686412+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:09.672 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:09 smithi183 bash[20824]: audit 2024-02-13T03:29:08.687785+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:09.672 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:09 smithi183 bash[20824]: audit 2024-02-13T03:29:08.688194+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi161-61719-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:09.672 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:09 smithi183 bash[20824]: cluster 2024-02-13T03:29:09.268019+0000 mgr.y (mgr.24473) 335 : cluster [DBG] pgmap v365: 292 pgs: 32 unknown, 7 creating+activating, 20 creating+peering, 233 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:29:09.672 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:09 smithi183 bash[20824]: audit 2024-02-13T03:29:09.622504+0000 mon.c (mon.2) 364 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:09.672 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:09 smithi183 bash[20824]: cluster 2024-02-13T03:29:09.645494+0000 mon.a (mon.0) 1462 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:09.673 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:09 smithi183 bash[20824]: audit 2024-02-13T03:29:09.657659+0000 mon.a (mon.0) 1463 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-23", "mode": "writeback"}]': finished 2024-02-13T03:29:09.673 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:09 smithi183 bash[20824]: audit 2024-02-13T03:29:09.657933+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi161-61719-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:09.673 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:09 smithi183 bash[20824]: cluster 2024-02-13T03:29:09.658257+0000 mon.a (mon.0) 1465 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-02-13T03:29:09.673 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:09 smithi183 bash[20824]: audit 2024-02-13T03:29:09.663880+0000 mon.c (mon.2) 365 : audit [INF] from='client.? v1:172.21.15.161:0/2382913754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:09.673 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:09 smithi183 bash[20824]: audit 2024-02-13T03:29:09.663949+0000 mon.a (mon.0) 1466 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi161-61719-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:09.673 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:09 smithi183 bash[20824]: audit 2024-02-13T03:29:09.665458+0000 mon.a (mon.0) 1467 : audit [INF] from='client.48158 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[27879]: audit 2024-02-13T03:29:08.686412+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[27879]: audit 2024-02-13T03:29:08.687785+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[27879]: audit 2024-02-13T03:29:08.688194+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi161-61719-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[27879]: cluster 2024-02-13T03:29:09.268019+0000 mgr.y (mgr.24473) 335 : cluster [DBG] pgmap v365: 292 pgs: 32 unknown, 7 creating+activating, 20 creating+peering, 233 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:29:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[27879]: audit 2024-02-13T03:29:09.622504+0000 mon.c (mon.2) 364 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[27879]: cluster 2024-02-13T03:29:09.645494+0000 mon.a (mon.0) 1462 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[27879]: audit 2024-02-13T03:29:09.657659+0000 mon.a (mon.0) 1463 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-23", "mode": "writeback"}]': finished 2024-02-13T03:29:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[27879]: audit 2024-02-13T03:29:09.657933+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi161-61719-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[27879]: cluster 2024-02-13T03:29:09.658257+0000 mon.a (mon.0) 1465 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-02-13T03:29:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[27879]: audit 2024-02-13T03:29:09.663880+0000 mon.c (mon.2) 365 : audit [INF] from='client.? v1:172.21.15.161:0/2382913754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[27879]: audit 2024-02-13T03:29:09.663949+0000 mon.a (mon.0) 1466 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi161-61719-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:09.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[27879]: audit 2024-02-13T03:29:09.665458+0000 mon.a (mon.0) 1467 : audit [INF] from='client.48158 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:09.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[19769]: audit 2024-02-13T03:29:08.686412+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:09.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[19769]: audit 2024-02-13T03:29:08.687785+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:09.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[19769]: audit 2024-02-13T03:29:08.688194+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi161-61719-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:09.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[19769]: cluster 2024-02-13T03:29:09.268019+0000 mgr.y (mgr.24473) 335 : cluster [DBG] pgmap v365: 292 pgs: 32 unknown, 7 creating+activating, 20 creating+peering, 233 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:29:09.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[19769]: audit 2024-02-13T03:29:09.622504+0000 mon.c (mon.2) 364 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:09.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[19769]: cluster 2024-02-13T03:29:09.645494+0000 mon.a (mon.0) 1462 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:09.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[19769]: audit 2024-02-13T03:29:09.657659+0000 mon.a (mon.0) 1463 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-23", "mode": "writeback"}]': finished 2024-02-13T03:29:09.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[19769]: audit 2024-02-13T03:29:09.657933+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi161-61719-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:09.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[19769]: cluster 2024-02-13T03:29:09.658257+0000 mon.a (mon.0) 1465 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-02-13T03:29:09.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[19769]: audit 2024-02-13T03:29:09.663880+0000 mon.c (mon.2) 365 : audit [INF] from='client.? v1:172.21.15.161:0/2382913754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:09.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[19769]: audit 2024-02-13T03:29:09.663949+0000 mon.a (mon.0) 1466 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi161-61719-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:09.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:09 smithi161 bash[19769]: audit 2024-02-13T03:29:09.665458+0000 mon.a (mon.0) 1467 : audit [INF] from='client.48158 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:11 smithi161 bash[27879]: audit 2024-02-13T03:29:10.624131+0000 mon.c (mon.2) 366 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:11.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:29:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:29:11] "GET /metrics HTTP/1.1" 200 194953 "" "Prometheus/2.33.4" 2024-02-13T03:29:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:11 smithi161 bash[19769]: audit 2024-02-13T03:29:10.624131+0000 mon.c (mon.2) 366 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:11.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:11 smithi183 bash[20824]: audit 2024-02-13T03:29:10.624131+0000 mon.c (mon.2) 366 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[27879]: audit 2024-02-13T03:29:11.077155+0000 mon.a (mon.0) 1468 : audit [INF] from='client.48158 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[27879]: cluster 2024-02-13T03:29:11.078167+0000 mon.a (mon.0) 1469 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-02-13T03:29:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[27879]: audit 2024-02-13T03:29:11.115996+0000 mon.b (mon.1) 197 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[27879]: cluster 2024-02-13T03:29:11.268907+0000 mgr.y (mgr.24473) 336 : cluster [DBG] pgmap v368: 324 pgs: 32 creating+peering, 292 active+clean; 9.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-13T03:29:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[27879]: audit 2024-02-13T03:29:11.542452+0000 mon.a (mon.0) 1470 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[27879]: audit 2024-02-13T03:29:11.625730+0000 mon.c (mon.2) 367 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[27879]: audit 2024-02-13T03:29:11.660888+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi161-61719-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi161-61719-33"}]': finished 2024-02-13T03:29:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[27879]: audit 2024-02-13T03:29:11.661163+0000 mon.a (mon.0) 1472 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:12.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[27879]: cluster 2024-02-13T03:29:11.661273+0000 mon.a (mon.0) 1473 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-02-13T03:29:12.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[27879]: audit 2024-02-13T03:29:11.665547+0000 mon.a (mon.0) 1474 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23"}]: dispatch 2024-02-13T03:29:12.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[27879]: audit 2024-02-13T03:29:11.666645+0000 mon.c (mon.2) 368 : audit [INF] from='client.? v1:172.21.15.161:0/819195127' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:12.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[27879]: audit 2024-02-13T03:29:11.667728+0000 mon.b (mon.1) 198 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23"}]: dispatch 2024-02-13T03:29:12.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[27879]: audit 2024-02-13T03:29:11.668679+0000 mon.a (mon.0) 1475 : audit [INF] from='client.48164 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:12.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[19769]: audit 2024-02-13T03:29:11.077155+0000 mon.a (mon.0) 1468 : audit [INF] from='client.48158 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:12.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[19769]: cluster 2024-02-13T03:29:11.078167+0000 mon.a (mon.0) 1469 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-02-13T03:29:12.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[19769]: audit 2024-02-13T03:29:11.115996+0000 mon.b (mon.1) 197 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:12.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[19769]: cluster 2024-02-13T03:29:11.268907+0000 mgr.y (mgr.24473) 336 : cluster [DBG] pgmap v368: 324 pgs: 32 creating+peering, 292 active+clean; 9.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-13T03:29:12.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[19769]: audit 2024-02-13T03:29:11.542452+0000 mon.a (mon.0) 1470 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[19769]: audit 2024-02-13T03:29:11.625730+0000 mon.c (mon.2) 367 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[19769]: audit 2024-02-13T03:29:11.660888+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi161-61719-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi161-61719-33"}]': finished 2024-02-13T03:29:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[19769]: audit 2024-02-13T03:29:11.661163+0000 mon.a (mon.0) 1472 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[19769]: cluster 2024-02-13T03:29:11.661273+0000 mon.a (mon.0) 1473 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-02-13T03:29:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[19769]: audit 2024-02-13T03:29:11.665547+0000 mon.a (mon.0) 1474 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23"}]: dispatch 2024-02-13T03:29:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[19769]: audit 2024-02-13T03:29:11.666645+0000 mon.c (mon.2) 368 : audit [INF] from='client.? v1:172.21.15.161:0/819195127' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[19769]: audit 2024-02-13T03:29:11.667728+0000 mon.b (mon.1) 198 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23"}]: dispatch 2024-02-13T03:29:12.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:12 smithi161 bash[19769]: audit 2024-02-13T03:29:11.668679+0000 mon.a (mon.0) 1475 : audit [INF] from='client.48164 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:12.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:12 smithi183 bash[20824]: audit 2024-02-13T03:29:11.077155+0000 mon.a (mon.0) 1468 : audit [INF] from='client.48158 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:12.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:12 smithi183 bash[20824]: cluster 2024-02-13T03:29:11.078167+0000 mon.a (mon.0) 1469 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-02-13T03:29:12.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:12 smithi183 bash[20824]: audit 2024-02-13T03:29:11.115996+0000 mon.b (mon.1) 197 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:12.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:12 smithi183 bash[20824]: cluster 2024-02-13T03:29:11.268907+0000 mgr.y (mgr.24473) 336 : cluster [DBG] pgmap v368: 324 pgs: 32 creating+peering, 292 active+clean; 9.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-13T03:29:12.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:12 smithi183 bash[20824]: audit 2024-02-13T03:29:11.542452+0000 mon.a (mon.0) 1470 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:12.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:12 smithi183 bash[20824]: audit 2024-02-13T03:29:11.625730+0000 mon.c (mon.2) 367 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:12.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:12 smithi183 bash[20824]: audit 2024-02-13T03:29:11.660888+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi161-61719-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi161-61719-33"}]': finished 2024-02-13T03:29:12.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:12 smithi183 bash[20824]: audit 2024-02-13T03:29:11.661163+0000 mon.a (mon.0) 1472 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:12.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:12 smithi183 bash[20824]: cluster 2024-02-13T03:29:11.661273+0000 mon.a (mon.0) 1473 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-02-13T03:29:12.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:12 smithi183 bash[20824]: audit 2024-02-13T03:29:11.665547+0000 mon.a (mon.0) 1474 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23"}]: dispatch 2024-02-13T03:29:12.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:12 smithi183 bash[20824]: audit 2024-02-13T03:29:11.666645+0000 mon.c (mon.2) 368 : audit [INF] from='client.? v1:172.21.15.161:0/819195127' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:12.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:12 smithi183 bash[20824]: audit 2024-02-13T03:29:11.667728+0000 mon.b (mon.1) 198 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23"}]: dispatch 2024-02-13T03:29:12.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:12 smithi183 bash[20824]: audit 2024-02-13T03:29:11.668679+0000 mon.a (mon.0) 1475 : audit [INF] from='client.48164 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:13.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:13 smithi161 bash[19769]: audit 2024-02-13T03:29:12.627391+0000 mon.c (mon.2) 369 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:13.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:13 smithi161 bash[19769]: cluster 2024-02-13T03:29:12.657196+0000 mon.a (mon.0) 1476 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:13.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:13 smithi161 bash[19769]: audit 2024-02-13T03:29:12.665610+0000 mon.a (mon.0) 1477 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23"}]': finished 2024-02-13T03:29:13.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:13 smithi161 bash[19769]: audit 2024-02-13T03:29:12.665718+0000 mon.a (mon.0) 1478 : audit [INF] from='client.48164 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:13.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:13 smithi161 bash[19769]: cluster 2024-02-13T03:29:12.665917+0000 mon.a (mon.0) 1479 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-02-13T03:29:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:13 smithi161 bash[27879]: audit 2024-02-13T03:29:12.627391+0000 mon.c (mon.2) 369 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:13 smithi161 bash[27879]: cluster 2024-02-13T03:29:12.657196+0000 mon.a (mon.0) 1476 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:13 smithi161 bash[27879]: audit 2024-02-13T03:29:12.665610+0000 mon.a (mon.0) 1477 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23"}]': finished 2024-02-13T03:29:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:13 smithi161 bash[27879]: audit 2024-02-13T03:29:12.665718+0000 mon.a (mon.0) 1478 : audit [INF] from='client.48164 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:13.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:13 smithi161 bash[27879]: cluster 2024-02-13T03:29:12.665917+0000 mon.a (mon.0) 1479 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-02-13T03:29:13.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:13 smithi183 bash[20824]: audit 2024-02-13T03:29:12.627391+0000 mon.c (mon.2) 369 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:13.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:13 smithi183 bash[20824]: cluster 2024-02-13T03:29:12.657196+0000 mon.a (mon.0) 1476 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:13.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:13 smithi183 bash[20824]: audit 2024-02-13T03:29:12.665610+0000 mon.a (mon.0) 1477 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-23"}]': finished 2024-02-13T03:29:13.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:13 smithi183 bash[20824]: audit 2024-02-13T03:29:12.665718+0000 mon.a (mon.0) 1478 : audit [INF] from='client.48164 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:13.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:13 smithi183 bash[20824]: cluster 2024-02-13T03:29:12.665917+0000 mon.a (mon.0) 1479 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-02-13T03:29:14.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:14 smithi161 bash[27879]: cluster 2024-02-13T03:29:13.269856+0000 mgr.y (mgr.24473) 337 : cluster [DBG] pgmap v371: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 9.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-13T03:29:14.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:14 smithi161 bash[27879]: audit 2024-02-13T03:29:13.628999+0000 mon.c (mon.2) 370 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:14.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:14 smithi161 bash[27879]: cluster 2024-02-13T03:29:13.672838+0000 mon.a (mon.0) 1480 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-02-13T03:29:14.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:14 smithi161 bash[27879]: audit 2024-02-13T03:29:13.678285+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:14.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:14 smithi161 bash[27879]: audit 2024-02-13T03:29:13.683897+0000 mon.c (mon.2) 371 : audit [INF] from='client.? v1:172.21.15.161:0/2710692966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:14.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:14 smithi161 bash[27879]: audit 2024-02-13T03:29:13.685920+0000 mon.a (mon.0) 1482 : audit [INF] from='client.48170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:14.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:14 smithi161 bash[19769]: cluster 2024-02-13T03:29:13.269856+0000 mgr.y (mgr.24473) 337 : cluster [DBG] pgmap v371: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 9.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-13T03:29:14.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:14 smithi161 bash[19769]: audit 2024-02-13T03:29:13.628999+0000 mon.c (mon.2) 370 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:14.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:14 smithi161 bash[19769]: cluster 2024-02-13T03:29:13.672838+0000 mon.a (mon.0) 1480 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-02-13T03:29:14.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:14 smithi161 bash[19769]: audit 2024-02-13T03:29:13.678285+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:14.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:14 smithi161 bash[19769]: audit 2024-02-13T03:29:13.683897+0000 mon.c (mon.2) 371 : audit [INF] from='client.? v1:172.21.15.161:0/2710692966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:14.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:14 smithi161 bash[19769]: audit 2024-02-13T03:29:13.685920+0000 mon.a (mon.0) 1482 : audit [INF] from='client.48170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:14.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:14 smithi183 bash[20824]: cluster 2024-02-13T03:29:13.269856+0000 mgr.y (mgr.24473) 337 : cluster [DBG] pgmap v371: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 9.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-13T03:29:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:14 smithi183 bash[20824]: audit 2024-02-13T03:29:13.628999+0000 mon.c (mon.2) 370 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:14 smithi183 bash[20824]: cluster 2024-02-13T03:29:13.672838+0000 mon.a (mon.0) 1480 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-02-13T03:29:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:14 smithi183 bash[20824]: audit 2024-02-13T03:29:13.678285+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:14 smithi183 bash[20824]: audit 2024-02-13T03:29:13.683897+0000 mon.c (mon.2) 371 : audit [INF] from='client.? v1:172.21.15.161:0/2710692966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:14 smithi183 bash[20824]: audit 2024-02-13T03:29:13.685920+0000 mon.a (mon.0) 1482 : audit [INF] from='client.48170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:15.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[27879]: audit 2024-02-13T03:29:14.630576+0000 mon.c (mon.2) 372 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:15.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[27879]: audit 2024-02-13T03:29:14.676564+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi161-61719-33"}]': finished 2024-02-13T03:29:15.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[27879]: audit 2024-02-13T03:29:14.676885+0000 mon.a (mon.0) 1484 : audit [INF] from='client.48170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:15.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[27879]: cluster 2024-02-13T03:29:14.677084+0000 mon.a (mon.0) 1485 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-02-13T03:29:15.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[27879]: audit 2024-02-13T03:29:14.684435+0000 mon.a (mon.0) 1486 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:15.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[27879]: audit 2024-02-13T03:29:14.685271+0000 mon.a (mon.0) 1487 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:15.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[27879]: audit 2024-02-13T03:29:14.686506+0000 mon.b (mon.1) 199 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:15.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[19769]: audit 2024-02-13T03:29:14.630576+0000 mon.c (mon.2) 372 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:15.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[19769]: audit 2024-02-13T03:29:14.676564+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi161-61719-33"}]': finished 2024-02-13T03:29:15.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[19769]: audit 2024-02-13T03:29:14.676885+0000 mon.a (mon.0) 1484 : audit [INF] from='client.48170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:15.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[19769]: cluster 2024-02-13T03:29:14.677084+0000 mon.a (mon.0) 1485 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-02-13T03:29:15.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[19769]: audit 2024-02-13T03:29:14.684435+0000 mon.a (mon.0) 1486 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:15.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[19769]: audit 2024-02-13T03:29:14.685271+0000 mon.a (mon.0) 1487 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:15.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:15 smithi161 bash[19769]: audit 2024-02-13T03:29:14.686506+0000 mon.b (mon.1) 199 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:15 smithi183 bash[20824]: audit 2024-02-13T03:29:14.630576+0000 mon.c (mon.2) 372 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:15 smithi183 bash[20824]: audit 2024-02-13T03:29:14.676564+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi161-61719-33"}]': finished 2024-02-13T03:29:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:15 smithi183 bash[20824]: audit 2024-02-13T03:29:14.676885+0000 mon.a (mon.0) 1484 : audit [INF] from='client.48170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:15 smithi183 bash[20824]: cluster 2024-02-13T03:29:14.677084+0000 mon.a (mon.0) 1485 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-02-13T03:29:15.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:15 smithi183 bash[20824]: audit 2024-02-13T03:29:14.684435+0000 mon.a (mon.0) 1486 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi161-61719-33"}]: dispatch 2024-02-13T03:29:15.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:15 smithi183 bash[20824]: audit 2024-02-13T03:29:14.685271+0000 mon.a (mon.0) 1487 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:15.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:15 smithi183 bash[20824]: audit 2024-02-13T03:29:14.686506+0000 mon.b (mon.1) 199 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:16.180 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:29:15 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:29:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:16 smithi183 bash[20824]: cluster 2024-02-13T03:29:15.270735+0000 mgr.y (mgr.24473) 338 : cluster [DBG] pgmap v374: 388 pgs: 96 unknown, 32 creating+peering, 260 active+clean; 9.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:16 smithi183 bash[20824]: audit 2024-02-13T03:29:15.632196+0000 mon.c (mon.2) 373 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:16 smithi183 bash[20824]: audit 2024-02-13T03:29:15.683960+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi161-61719-33"}]': finished 2024-02-13T03:29:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:16 smithi183 bash[20824]: audit 2024-02-13T03:29:15.684639+0000 mon.a (mon.0) 1489 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:16 smithi183 bash[20824]: cluster 2024-02-13T03:29:15.685029+0000 mon.a (mon.0) 1490 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-02-13T03:29:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:16 smithi183 bash[20824]: audit 2024-02-13T03:29:15.687547+0000 mon.c (mon.2) 374 : audit [INF] from='client.? v1:172.21.15.161:0/1834211144' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:16 smithi183 bash[20824]: audit 2024-02-13T03:29:15.688907+0000 mon.a (mon.0) 1491 : audit [INF] from='client.48176 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:16 smithi183 bash[20824]: audit 2024-02-13T03:29:15.717741+0000 mon.c (mon.2) 375 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:16 smithi183 bash[20824]: audit 2024-02-13T03:29:15.718140+0000 mon.a (mon.0) 1492 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:16 smithi183 bash[20824]: audit 2024-02-13T03:29:15.719170+0000 mon.c (mon.2) 376 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:16.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:16 smithi183 bash[20824]: audit 2024-02-13T03:29:15.719431+0000 mon.a (mon.0) 1493 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:16.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:16 smithi183 bash[20824]: audit 2024-02-13T03:29:15.719872+0000 mon.c (mon.2) 377 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi161-61719-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:16.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:16 smithi183 bash[20824]: audit 2024-02-13T03:29:15.720118+0000 mon.a (mon.0) 1494 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi161-61719-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:16.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[27879]: cluster 2024-02-13T03:29:15.270735+0000 mgr.y (mgr.24473) 338 : cluster [DBG] pgmap v374: 388 pgs: 96 unknown, 32 creating+peering, 260 active+clean; 9.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:16.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[27879]: audit 2024-02-13T03:29:15.632196+0000 mon.c (mon.2) 373 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:16.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[27879]: audit 2024-02-13T03:29:15.683960+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi161-61719-33"}]': finished 2024-02-13T03:29:16.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[27879]: audit 2024-02-13T03:29:15.684639+0000 mon.a (mon.0) 1489 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:16.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[27879]: cluster 2024-02-13T03:29:15.685029+0000 mon.a (mon.0) 1490 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-02-13T03:29:16.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[27879]: audit 2024-02-13T03:29:15.687547+0000 mon.c (mon.2) 374 : audit [INF] from='client.? v1:172.21.15.161:0/1834211144' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:16.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[27879]: audit 2024-02-13T03:29:15.688907+0000 mon.a (mon.0) 1491 : audit [INF] from='client.48176 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:16.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[27879]: audit 2024-02-13T03:29:15.717741+0000 mon.c (mon.2) 375 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:16.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[27879]: audit 2024-02-13T03:29:15.718140+0000 mon.a (mon.0) 1492 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:16.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[27879]: audit 2024-02-13T03:29:15.719170+0000 mon.c (mon.2) 376 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:16.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[27879]: audit 2024-02-13T03:29:15.719431+0000 mon.a (mon.0) 1493 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:16.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[27879]: audit 2024-02-13T03:29:15.719872+0000 mon.c (mon.2) 377 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi161-61719-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:16.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[27879]: audit 2024-02-13T03:29:15.720118+0000 mon.a (mon.0) 1494 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi161-61719-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[19769]: cluster 2024-02-13T03:29:15.270735+0000 mgr.y (mgr.24473) 338 : cluster [DBG] pgmap v374: 388 pgs: 96 unknown, 32 creating+peering, 260 active+clean; 9.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[19769]: audit 2024-02-13T03:29:15.632196+0000 mon.c (mon.2) 373 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[19769]: audit 2024-02-13T03:29:15.683960+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? v1:172.21.15.161:0/416925892' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi161-61719-33"}]': finished 2024-02-13T03:29:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[19769]: audit 2024-02-13T03:29:15.684639+0000 mon.a (mon.0) 1489 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[19769]: cluster 2024-02-13T03:29:15.685029+0000 mon.a (mon.0) 1490 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-02-13T03:29:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[19769]: audit 2024-02-13T03:29:15.687547+0000 mon.c (mon.2) 374 : audit [INF] from='client.? v1:172.21.15.161:0/1834211144' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[19769]: audit 2024-02-13T03:29:15.688907+0000 mon.a (mon.0) 1491 : audit [INF] from='client.48176 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:16.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[19769]: audit 2024-02-13T03:29:15.717741+0000 mon.c (mon.2) 375 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:16.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[19769]: audit 2024-02-13T03:29:15.718140+0000 mon.a (mon.0) 1492 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:16.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[19769]: audit 2024-02-13T03:29:15.719170+0000 mon.c (mon.2) 376 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:16.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[19769]: audit 2024-02-13T03:29:15.719431+0000 mon.a (mon.0) 1493 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:16.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[19769]: audit 2024-02-13T03:29:15.719872+0000 mon.c (mon.2) 377 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi161-61719-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:16.679 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:16 smithi161 bash[19769]: audit 2024-02-13T03:29:15.720118+0000 mon.a (mon.0) 1494 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi161-61719-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:17 smithi183 bash[20824]: audit 2024-02-13T03:29:15.917325+0000 mgr.y (mgr.24473) 339 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:17.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:17 smithi183 bash[20824]: audit 2024-02-13T03:29:16.633857+0000 mon.c (mon.2) 378 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:17.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:17 smithi183 bash[20824]: audit 2024-02-13T03:29:16.682541+0000 mon.a (mon.0) 1495 : audit [INF] from='client.48176 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:17.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:17 smithi183 bash[20824]: audit 2024-02-13T03:29:16.682767+0000 mon.a (mon.0) 1496 : audit [INF] from='client.48182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi161-61719-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:17.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:17 smithi183 bash[20824]: cluster 2024-02-13T03:29:16.682858+0000 mon.a (mon.0) 1497 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-02-13T03:29:17.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:17 smithi183 bash[20824]: audit 2024-02-13T03:29:16.686422+0000 mon.c (mon.2) 379 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi161-61719-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:17.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:17 smithi183 bash[20824]: audit 2024-02-13T03:29:16.687730+0000 mon.a (mon.0) 1498 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi161-61719-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:17.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:17 smithi183 bash[20824]: audit 2024-02-13T03:29:16.713365+0000 mon.a (mon.0) 1499 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:17.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:17 smithi183 bash[20824]: audit 2024-02-13T03:29:16.716257+0000 mon.b (mon.1) 200 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[19769]: audit 2024-02-13T03:29:15.917325+0000 mgr.y (mgr.24473) 339 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[19769]: audit 2024-02-13T03:29:16.633857+0000 mon.c (mon.2) 378 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[19769]: audit 2024-02-13T03:29:16.682541+0000 mon.a (mon.0) 1495 : audit [INF] from='client.48176 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[19769]: audit 2024-02-13T03:29:16.682767+0000 mon.a (mon.0) 1496 : audit [INF] from='client.48182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi161-61719-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[19769]: cluster 2024-02-13T03:29:16.682858+0000 mon.a (mon.0) 1497 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-02-13T03:29:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[19769]: audit 2024-02-13T03:29:16.686422+0000 mon.c (mon.2) 379 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi161-61719-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[19769]: audit 2024-02-13T03:29:16.687730+0000 mon.a (mon.0) 1498 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi161-61719-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[19769]: audit 2024-02-13T03:29:16.713365+0000 mon.a (mon.0) 1499 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[19769]: audit 2024-02-13T03:29:16.716257+0000 mon.b (mon.1) 200 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:17.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[27879]: audit 2024-02-13T03:29:15.917325+0000 mgr.y (mgr.24473) 339 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:17.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[27879]: audit 2024-02-13T03:29:16.633857+0000 mon.c (mon.2) 378 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:17.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[27879]: audit 2024-02-13T03:29:16.682541+0000 mon.a (mon.0) 1495 : audit [INF] from='client.48176 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi161-61726-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:17.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[27879]: audit 2024-02-13T03:29:16.682767+0000 mon.a (mon.0) 1496 : audit [INF] from='client.48182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi161-61719-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:17.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[27879]: cluster 2024-02-13T03:29:16.682858+0000 mon.a (mon.0) 1497 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-02-13T03:29:17.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[27879]: audit 2024-02-13T03:29:16.686422+0000 mon.c (mon.2) 379 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi161-61719-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:17.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[27879]: audit 2024-02-13T03:29:16.687730+0000 mon.a (mon.0) 1498 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi161-61719-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:17.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[27879]: audit 2024-02-13T03:29:16.713365+0000 mon.a (mon.0) 1499 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:17.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:17 smithi161 bash[27879]: audit 2024-02-13T03:29:16.716257+0000 mon.b (mon.1) 200 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:18.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:18 smithi183 bash[20824]: cluster 2024-02-13T03:29:17.271786+0000 mgr.y (mgr.24473) 340 : cluster [DBG] pgmap v377: 420 pgs: 32 unknown, 32 creating+peering, 356 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:18.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:18 smithi183 bash[20824]: audit 2024-02-13T03:29:17.635379+0000 mon.c (mon.2) 380 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:18.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:18 smithi183 bash[20824]: audit 2024-02-13T03:29:18.108682+0000 mon.a (mon.0) 1500 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:18.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:18 smithi183 bash[20824]: cluster 2024-02-13T03:29:18.108872+0000 mon.a (mon.0) 1501 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-02-13T03:29:18.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:18 smithi183 bash[20824]: audit 2024-02-13T03:29:18.111045+0000 mon.a (mon.0) 1502 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-25"}]: dispatch 2024-02-13T03:29:18.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:18 smithi183 bash[20824]: audit 2024-02-13T03:29:18.113540+0000 mon.b (mon.1) 201 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-25"}]: dispatch 2024-02-13T03:29:18.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:18 smithi161 bash[27879]: cluster 2024-02-13T03:29:17.271786+0000 mgr.y (mgr.24473) 340 : cluster [DBG] pgmap v377: 420 pgs: 32 unknown, 32 creating+peering, 356 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:18.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:18 smithi161 bash[27879]: audit 2024-02-13T03:29:17.635379+0000 mon.c (mon.2) 380 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:18.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:18 smithi161 bash[27879]: audit 2024-02-13T03:29:18.108682+0000 mon.a (mon.0) 1500 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:18.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:18 smithi161 bash[27879]: cluster 2024-02-13T03:29:18.108872+0000 mon.a (mon.0) 1501 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-02-13T03:29:18.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:18 smithi161 bash[27879]: audit 2024-02-13T03:29:18.111045+0000 mon.a (mon.0) 1502 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-25"}]: dispatch 2024-02-13T03:29:18.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:18 smithi161 bash[27879]: audit 2024-02-13T03:29:18.113540+0000 mon.b (mon.1) 201 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-25"}]: dispatch 2024-02-13T03:29:18.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:18 smithi161 bash[19769]: cluster 2024-02-13T03:29:17.271786+0000 mgr.y (mgr.24473) 340 : cluster [DBG] pgmap v377: 420 pgs: 32 unknown, 32 creating+peering, 356 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:18.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:18 smithi161 bash[19769]: audit 2024-02-13T03:29:17.635379+0000 mon.c (mon.2) 380 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:18.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:18 smithi161 bash[19769]: audit 2024-02-13T03:29:18.108682+0000 mon.a (mon.0) 1500 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:18.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:18 smithi161 bash[19769]: cluster 2024-02-13T03:29:18.108872+0000 mon.a (mon.0) 1501 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-02-13T03:29:18.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:18 smithi161 bash[19769]: audit 2024-02-13T03:29:18.111045+0000 mon.a (mon.0) 1502 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-25"}]: dispatch 2024-02-13T03:29:18.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:18 smithi161 bash[19769]: audit 2024-02-13T03:29:18.113540+0000 mon.b (mon.1) 201 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-25"}]: dispatch 2024-02-13T03:29:19.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:29:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:29:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:29:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:19 smithi183 bash[20824]: audit 2024-02-13T03:29:18.510363+0000 mon.a (mon.0) 1503 : audit [INF] from='client.48182 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi161-61719-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi161-61719-34"}]': finished 2024-02-13T03:29:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:19 smithi183 bash[20824]: audit 2024-02-13T03:29:18.510535+0000 mon.a (mon.0) 1504 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-25"}]': finished 2024-02-13T03:29:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:19 smithi183 bash[20824]: cluster 2024-02-13T03:29:18.510745+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-02-13T03:29:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:19 smithi183 bash[20824]: audit 2024-02-13T03:29:18.517819+0000 mon.b (mon.1) 202 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-25", "mode": "writeback"}]: dispatch 2024-02-13T03:29:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:19 smithi183 bash[20824]: audit 2024-02-13T03:29:18.524087+0000 mon.a (mon.0) 1506 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-25", "mode": "writeback"}]: dispatch 2024-02-13T03:29:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:19 smithi183 bash[20824]: audit 2024-02-13T03:29:18.637010+0000 mon.c (mon.2) 381 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:19 smithi183 bash[20824]: cluster 2024-02-13T03:29:19.272797+0000 mgr.y (mgr.24473) 341 : cluster [DBG] pgmap v380: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:29:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[27879]: audit 2024-02-13T03:29:18.510363+0000 mon.a (mon.0) 1503 : audit [INF] from='client.48182 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi161-61719-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi161-61719-34"}]': finished 2024-02-13T03:29:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[27879]: audit 2024-02-13T03:29:18.510535+0000 mon.a (mon.0) 1504 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-25"}]': finished 2024-02-13T03:29:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[27879]: cluster 2024-02-13T03:29:18.510745+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-02-13T03:29:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[27879]: audit 2024-02-13T03:29:18.517819+0000 mon.b (mon.1) 202 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-25", "mode": "writeback"}]: dispatch 2024-02-13T03:29:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[27879]: audit 2024-02-13T03:29:18.524087+0000 mon.a (mon.0) 1506 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-25", "mode": "writeback"}]: dispatch 2024-02-13T03:29:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[27879]: audit 2024-02-13T03:29:18.637010+0000 mon.c (mon.2) 381 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[27879]: cluster 2024-02-13T03:29:19.272797+0000 mgr.y (mgr.24473) 341 : cluster [DBG] pgmap v380: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:29:19.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[19769]: audit 2024-02-13T03:29:18.510363+0000 mon.a (mon.0) 1503 : audit [INF] from='client.48182 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi161-61719-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi161-61719-34"}]': finished 2024-02-13T03:29:19.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[19769]: audit 2024-02-13T03:29:18.510535+0000 mon.a (mon.0) 1504 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-25"}]': finished 2024-02-13T03:29:19.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[19769]: cluster 2024-02-13T03:29:18.510745+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-02-13T03:29:19.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[19769]: audit 2024-02-13T03:29:18.517819+0000 mon.b (mon.1) 202 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-25", "mode": "writeback"}]: dispatch 2024-02-13T03:29:19.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[19769]: audit 2024-02-13T03:29:18.524087+0000 mon.a (mon.0) 1506 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-25", "mode": "writeback"}]: dispatch 2024-02-13T03:29:19.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[19769]: audit 2024-02-13T03:29:18.637010+0000 mon.c (mon.2) 381 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:19.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:19 smithi161 bash[19769]: cluster 2024-02-13T03:29:19.272797+0000 mgr.y (mgr.24473) 341 : cluster [DBG] pgmap v380: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:29:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[27879]: cluster 2024-02-13T03:29:19.504787+0000 mon.a (mon.0) 1507 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[27879]: audit 2024-02-13T03:29:19.514876+0000 mon.a (mon.0) 1508 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-25", "mode": "writeback"}]': finished 2024-02-13T03:29:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[27879]: cluster 2024-02-13T03:29:19.515764+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-02-13T03:29:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[27879]: audit 2024-02-13T03:29:19.565099+0000 mon.a (mon.0) 1510 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[27879]: audit 2024-02-13T03:29:19.567892+0000 mon.b (mon.1) 203 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[27879]: audit 2024-02-13T03:29:19.638888+0000 mon.c (mon.2) 382 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[27879]: audit 2024-02-13T03:29:19.780539+0000 mon.a (mon.0) 1511 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:29:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[19769]: cluster 2024-02-13T03:29:19.504787+0000 mon.a (mon.0) 1507 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[19769]: audit 2024-02-13T03:29:19.514876+0000 mon.a (mon.0) 1508 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-25", "mode": "writeback"}]': finished 2024-02-13T03:29:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[19769]: cluster 2024-02-13T03:29:19.515764+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-02-13T03:29:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[19769]: audit 2024-02-13T03:29:19.565099+0000 mon.a (mon.0) 1510 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[19769]: audit 2024-02-13T03:29:19.567892+0000 mon.b (mon.1) 203 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[19769]: audit 2024-02-13T03:29:19.638888+0000 mon.c (mon.2) 382 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:20 smithi161 bash[19769]: audit 2024-02-13T03:29:19.780539+0000 mon.a (mon.0) 1511 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:29:20.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:20 smithi183 bash[20824]: cluster 2024-02-13T03:29:19.504787+0000 mon.a (mon.0) 1507 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:20.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:20 smithi183 bash[20824]: audit 2024-02-13T03:29:19.514876+0000 mon.a (mon.0) 1508 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-25", "mode": "writeback"}]': finished 2024-02-13T03:29:20.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:20 smithi183 bash[20824]: cluster 2024-02-13T03:29:19.515764+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-02-13T03:29:20.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:20 smithi183 bash[20824]: audit 2024-02-13T03:29:19.565099+0000 mon.a (mon.0) 1510 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:20.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:20 smithi183 bash[20824]: audit 2024-02-13T03:29:19.567892+0000 mon.b (mon.1) 203 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:20 smithi183 bash[20824]: audit 2024-02-13T03:29:19.638888+0000 mon.c (mon.2) 382 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:20 smithi183 bash[20824]: audit 2024-02-13T03:29:19.780539+0000 mon.a (mon.0) 1511 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:29:21.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:29:21] "GET /metrics HTTP/1.1" 200 195410 "" "Prometheus/2.33.4" 2024-02-13T03:29:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[27879]: audit 2024-02-13T03:29:20.538786+0000 mon.a (mon.0) 1512 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[27879]: cluster 2024-02-13T03:29:20.538870+0000 mon.a (mon.0) 1513 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-02-13T03:29:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[27879]: audit 2024-02-13T03:29:20.543081+0000 mon.c (mon.2) 383 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[27879]: audit 2024-02-13T03:29:20.543157+0000 mon.a (mon.0) 1514 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25"}]: dispatch 2024-02-13T03:29:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[27879]: audit 2024-02-13T03:29:20.543722+0000 mon.a (mon.0) 1515 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[27879]: audit 2024-02-13T03:29:20.545837+0000 mon.b (mon.1) 204 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25"}]: dispatch 2024-02-13T03:29:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[27879]: audit 2024-02-13T03:29:20.640583+0000 mon.c (mon.2) 384 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[27879]: cluster 2024-02-13T03:29:21.275141+0000 mgr.y (mgr.24473) 342 : cluster [DBG] pgmap v383: 292 pgs: 292 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-13T03:29:21.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[27879]: audit 2024-02-13T03:29:21.275663+0000 mon.c (mon.2) 385 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi161-61759-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-13T03:29:21.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[27879]: audit 2024-02-13T03:29:21.277570+0000 mon.a (mon.0) 1516 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi161-61759-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-13T03:29:21.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[19769]: audit 2024-02-13T03:29:20.538786+0000 mon.a (mon.0) 1512 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:21.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[19769]: cluster 2024-02-13T03:29:20.538870+0000 mon.a (mon.0) 1513 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-02-13T03:29:21.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[19769]: audit 2024-02-13T03:29:20.543081+0000 mon.c (mon.2) 383 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:21.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[19769]: audit 2024-02-13T03:29:20.543157+0000 mon.a (mon.0) 1514 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25"}]: dispatch 2024-02-13T03:29:21.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[19769]: audit 2024-02-13T03:29:20.543722+0000 mon.a (mon.0) 1515 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:21.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[19769]: audit 2024-02-13T03:29:20.545837+0000 mon.b (mon.1) 204 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25"}]: dispatch 2024-02-13T03:29:21.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[19769]: audit 2024-02-13T03:29:20.640583+0000 mon.c (mon.2) 384 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:21.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[19769]: cluster 2024-02-13T03:29:21.275141+0000 mgr.y (mgr.24473) 342 : cluster [DBG] pgmap v383: 292 pgs: 292 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-13T03:29:21.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[19769]: audit 2024-02-13T03:29:21.275663+0000 mon.c (mon.2) 385 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi161-61759-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-13T03:29:21.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:21 smithi161 bash[19769]: audit 2024-02-13T03:29:21.277570+0000 mon.a (mon.0) 1516 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi161-61759-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-13T03:29:21.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:21 smithi183 bash[20824]: audit 2024-02-13T03:29:20.538786+0000 mon.a (mon.0) 1512 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:21 smithi183 bash[20824]: cluster 2024-02-13T03:29:20.538870+0000 mon.a (mon.0) 1513 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-02-13T03:29:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:21 smithi183 bash[20824]: audit 2024-02-13T03:29:20.543081+0000 mon.c (mon.2) 383 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:21 smithi183 bash[20824]: audit 2024-02-13T03:29:20.543157+0000 mon.a (mon.0) 1514 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25"}]: dispatch 2024-02-13T03:29:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:21 smithi183 bash[20824]: audit 2024-02-13T03:29:20.543722+0000 mon.a (mon.0) 1515 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:21 smithi183 bash[20824]: audit 2024-02-13T03:29:20.545837+0000 mon.b (mon.1) 204 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25"}]: dispatch 2024-02-13T03:29:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:21 smithi183 bash[20824]: audit 2024-02-13T03:29:20.640583+0000 mon.c (mon.2) 384 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:21 smithi183 bash[20824]: cluster 2024-02-13T03:29:21.275141+0000 mgr.y (mgr.24473) 342 : cluster [DBG] pgmap v383: 292 pgs: 292 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-13T03:29:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:21 smithi183 bash[20824]: audit 2024-02-13T03:29:21.275663+0000 mon.c (mon.2) 385 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi161-61759-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-13T03:29:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:21 smithi183 bash[20824]: audit 2024-02-13T03:29:21.277570+0000 mon.a (mon.0) 1516 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi161-61759-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-13T03:29:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: cluster 2024-02-13T03:29:21.534150+0000 mon.a (mon.0) 1517 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: audit 2024-02-13T03:29:21.546255+0000 mon.a (mon.0) 1518 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25"}]': finished 2024-02-13T03:29:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: audit 2024-02-13T03:29:21.546475+0000 mon.a (mon.0) 1519 : audit [INF] from='client.48182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]': finished 2024-02-13T03:29:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: audit 2024-02-13T03:29:21.546715+0000 mon.a (mon.0) 1520 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi161-61759-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-13T03:29:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: cluster 2024-02-13T03:29:21.546899+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-02-13T03:29:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: audit 2024-02-13T03:29:21.548210+0000 mon.c (mon.2) 386 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: audit 2024-02-13T03:29:21.550804+0000 mon.a (mon.0) 1522 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: audit 2024-02-13T03:29:21.642090+0000 mon.c (mon.2) 387 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: audit 2024-02-13T03:29:21.643022+0000 mon.c (mon.2) 388 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: audit 2024-02-13T03:29:21.643879+0000 mon.c (mon.2) 389 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-13T03:29:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: audit 2024-02-13T03:29:22.012142+0000 mon.a (mon.0) 1523 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-13T03:29:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: audit 2024-02-13T03:29:22.549765+0000 mon.a (mon.0) 1524 : audit [INF] from='client.48182 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]': finished 2024-02-13T03:29:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: audit 2024-02-13T03:29:22.550474+0000 mon.a (mon.0) 1525 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]': finished 2024-02-13T03:29:23.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[19769]: cluster 2024-02-13T03:29:22.550639+0000 mon.a (mon.0) 1526 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-02-13T03:29:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: cluster 2024-02-13T03:29:21.534150+0000 mon.a (mon.0) 1517 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: audit 2024-02-13T03:29:21.546255+0000 mon.a (mon.0) 1518 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25"}]': finished 2024-02-13T03:29:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: audit 2024-02-13T03:29:21.546475+0000 mon.a (mon.0) 1519 : audit [INF] from='client.48182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]': finished 2024-02-13T03:29:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: audit 2024-02-13T03:29:21.546715+0000 mon.a (mon.0) 1520 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi161-61759-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-13T03:29:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: cluster 2024-02-13T03:29:21.546899+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-02-13T03:29:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: audit 2024-02-13T03:29:21.548210+0000 mon.c (mon.2) 386 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: audit 2024-02-13T03:29:21.550804+0000 mon.a (mon.0) 1522 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: audit 2024-02-13T03:29:21.642090+0000 mon.c (mon.2) 387 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: audit 2024-02-13T03:29:21.643022+0000 mon.c (mon.2) 388 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: audit 2024-02-13T03:29:21.643879+0000 mon.c (mon.2) 389 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-13T03:29:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: audit 2024-02-13T03:29:22.012142+0000 mon.a (mon.0) 1523 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-13T03:29:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: audit 2024-02-13T03:29:22.549765+0000 mon.a (mon.0) 1524 : audit [INF] from='client.48182 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]': finished 2024-02-13T03:29:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: audit 2024-02-13T03:29:22.550474+0000 mon.a (mon.0) 1525 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]': finished 2024-02-13T03:29:23.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:23 smithi161 bash[27879]: cluster 2024-02-13T03:29:22.550639+0000 mon.a (mon.0) 1526 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-02-13T03:29:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: cluster 2024-02-13T03:29:21.534150+0000 mon.a (mon.0) 1517 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: audit 2024-02-13T03:29:21.546255+0000 mon.a (mon.0) 1518 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-25"}]': finished 2024-02-13T03:29:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: audit 2024-02-13T03:29:21.546475+0000 mon.a (mon.0) 1519 : audit [INF] from='client.48182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi161-61719-34"}]': finished 2024-02-13T03:29:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: audit 2024-02-13T03:29:21.546715+0000 mon.a (mon.0) 1520 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi161-61759-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-13T03:29:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: cluster 2024-02-13T03:29:21.546899+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-02-13T03:29:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: audit 2024-02-13T03:29:21.548210+0000 mon.c (mon.2) 386 : audit [INF] from='client.? v1:172.21.15.161:0/1830674168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: audit 2024-02-13T03:29:21.550804+0000 mon.a (mon.0) 1522 : audit [INF] from='client.48182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]: dispatch 2024-02-13T03:29:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: audit 2024-02-13T03:29:21.642090+0000 mon.c (mon.2) 387 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: audit 2024-02-13T03:29:21.643022+0000 mon.c (mon.2) 388 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: audit 2024-02-13T03:29:21.643879+0000 mon.c (mon.2) 389 : audit [INF] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-13T03:29:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: audit 2024-02-13T03:29:22.012142+0000 mon.a (mon.0) 1523 : audit [INF] from='client.24722 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-13T03:29:23.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: audit 2024-02-13T03:29:22.549765+0000 mon.a (mon.0) 1524 : audit [INF] from='client.48182 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi161-61719-34"}]': finished 2024-02-13T03:29:23.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: audit 2024-02-13T03:29:22.550474+0000 mon.a (mon.0) 1525 : audit [INF] from='client.24722 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi161-61759-1","var":"pgp_num","val":"11"}]': finished 2024-02-13T03:29:23.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:23 smithi183 bash[20824]: cluster 2024-02-13T03:29:22.550639+0000 mon.a (mon.0) 1526 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-02-13T03:29:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:22.551658+0000 mon.c (mon.2) 390 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:22.554131+0000 mon.c (mon.2) 391 : audit [INF] from='client.? v1:172.21.15.161:0/3123652187' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi161-61726-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:22.582402+0000 mon.c (mon.2) 392 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.001092+0000 mon.a (mon.0) 1527 : audit [INF] from='client.48188 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi161-61726-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.001718+0000 mon.a (mon.0) 1528 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.009549+0000 mon.c (mon.2) 393 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.009997+0000 mon.a (mon.0) 1529 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.010679+0000 mon.c (mon.2) 394 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi161-61719-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:24.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.011108+0000 mon.a (mon.0) 1530 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi161-61719-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:24.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: cluster 2024-02-13T03:29:23.276114+0000 mgr.y (mgr.24473) 343 : cluster [DBG] pgmap v386: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:24.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.551015+0000 mon.a (mon.0) 1531 : audit [INF] from='client.48188 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi161-61726-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:24.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.551546+0000 mon.a (mon.0) 1532 : audit [INF] from='client.48194 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi161-61719-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:24.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: cluster 2024-02-13T03:29:23.551686+0000 mon.a (mon.0) 1533 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-02-13T03:29:24.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.555467+0000 mon.c (mon.2) 395 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi161-61719-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:22.551658+0000 mon.c (mon.2) 390 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:24.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:22.554131+0000 mon.c (mon.2) 391 : audit [INF] from='client.? v1:172.21.15.161:0/3123652187' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi161-61726-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:24.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:22.582402+0000 mon.c (mon.2) 392 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.001092+0000 mon.a (mon.0) 1527 : audit [INF] from='client.48188 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi161-61726-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:24.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.001718+0000 mon.a (mon.0) 1528 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.009549+0000 mon.c (mon.2) 393 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.009997+0000 mon.a (mon.0) 1529 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.010679+0000 mon.c (mon.2) 394 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi161-61719-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:24.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.011108+0000 mon.a (mon.0) 1530 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi161-61719-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:24.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: cluster 2024-02-13T03:29:23.276114+0000 mgr.y (mgr.24473) 343 : cluster [DBG] pgmap v386: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:24.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.551015+0000 mon.a (mon.0) 1531 : audit [INF] from='client.48188 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi161-61726-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:24.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.551546+0000 mon.a (mon.0) 1532 : audit [INF] from='client.48194 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi161-61719-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:24.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: cluster 2024-02-13T03:29:23.551686+0000 mon.a (mon.0) 1533 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-02-13T03:29:24.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.555467+0000 mon.c (mon.2) 395 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi161-61719-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.559500+0000 mon.b (mon.1) 205 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:24.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.570140+0000 mon.a (mon.0) 1534 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi161-61719-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.571055+0000 mon.a (mon.0) 1535 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:24.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.596508+0000 mon.c (mon.2) 396 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:24.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.596824+0000 mon.a (mon.0) 1536 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:24.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.597952+0000 mon.c (mon.2) 397 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:24.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.598220+0000 mon.a (mon.0) 1537 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:24.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.598813+0000 mon.c (mon.2) 398 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi161-61759-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:24.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[27879]: audit 2024-02-13T03:29:23.599181+0000 mon.a (mon.0) 1538 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi161-61759-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:24.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.559500+0000 mon.b (mon.1) 205 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:24.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.570140+0000 mon.a (mon.0) 1534 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi161-61719-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.571055+0000 mon.a (mon.0) 1535 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:24.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.596508+0000 mon.c (mon.2) 396 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:24.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.596824+0000 mon.a (mon.0) 1536 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:24.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.597952+0000 mon.c (mon.2) 397 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:24.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.598220+0000 mon.a (mon.0) 1537 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:24.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.598813+0000 mon.c (mon.2) 398 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi161-61759-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:24.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:24 smithi161 bash[19769]: audit 2024-02-13T03:29:23.599181+0000 mon.a (mon.0) 1538 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi161-61759-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:22.551658+0000 mon.c (mon.2) 390 : audit [DBG] from='client.? v1:172.21.15.161:0/1638945847' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-13T03:29:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:22.554131+0000 mon.c (mon.2) 391 : audit [INF] from='client.? v1:172.21.15.161:0/3123652187' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi161-61726-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:22.582402+0000 mon.c (mon.2) 392 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.001092+0000 mon.a (mon.0) 1527 : audit [INF] from='client.48188 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi161-61726-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.001718+0000 mon.a (mon.0) 1528 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.009549+0000 mon.c (mon.2) 393 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.009997+0000 mon.a (mon.0) 1529 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.010679+0000 mon.c (mon.2) 394 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi161-61719-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:24.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.011108+0000 mon.a (mon.0) 1530 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi161-61719-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:24.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: cluster 2024-02-13T03:29:23.276114+0000 mgr.y (mgr.24473) 343 : cluster [DBG] pgmap v386: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:24.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.551015+0000 mon.a (mon.0) 1531 : audit [INF] from='client.48188 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi161-61726-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:24.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.551546+0000 mon.a (mon.0) 1532 : audit [INF] from='client.48194 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi161-61719-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:24.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: cluster 2024-02-13T03:29:23.551686+0000 mon.a (mon.0) 1533 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-02-13T03:29:24.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.555467+0000 mon.c (mon.2) 395 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi161-61719-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.559500+0000 mon.b (mon.1) 205 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:24.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.570140+0000 mon.a (mon.0) 1534 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi161-61719-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:24.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.571055+0000 mon.a (mon.0) 1535 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:24.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.596508+0000 mon.c (mon.2) 396 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:24.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.596824+0000 mon.a (mon.0) 1536 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:24.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.597952+0000 mon.c (mon.2) 397 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:24.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.598220+0000 mon.a (mon.0) 1537 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:24.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.598813+0000 mon.c (mon.2) 398 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi161-61759-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:24.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:24 smithi183 bash[20824]: audit 2024-02-13T03:29:23.599181+0000 mon.a (mon.0) 1538 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi161-61759-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:25.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:25 smithi161 bash[27879]: audit 2024-02-13T03:29:24.975753+0000 mon.a (mon.0) 1539 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:25 smithi161 bash[27879]: audit 2024-02-13T03:29:24.976326+0000 mon.a (mon.0) 1540 : audit [INF] from='client.48200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi161-61759-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:25 smithi161 bash[27879]: cluster 2024-02-13T03:29:24.976579+0000 mon.a (mon.0) 1541 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-02-13T03:29:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:25 smithi161 bash[27879]: audit 2024-02-13T03:29:24.977756+0000 mon.c (mon.2) 399 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi161-61759-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:25 smithi161 bash[27879]: audit 2024-02-13T03:29:24.979788+0000 mon.a (mon.0) 1542 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi161-61759-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:25 smithi161 bash[27879]: cluster 2024-02-13T03:29:25.276904+0000 mgr.y (mgr.24473) 344 : cluster [DBG] pgmap v389: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:25 smithi161 bash[19769]: audit 2024-02-13T03:29:24.975753+0000 mon.a (mon.0) 1539 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:25 smithi161 bash[19769]: audit 2024-02-13T03:29:24.976326+0000 mon.a (mon.0) 1540 : audit [INF] from='client.48200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi161-61759-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:25 smithi161 bash[19769]: cluster 2024-02-13T03:29:24.976579+0000 mon.a (mon.0) 1541 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-02-13T03:29:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:25 smithi161 bash[19769]: audit 2024-02-13T03:29:24.977756+0000 mon.c (mon.2) 399 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi161-61759-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:25 smithi161 bash[19769]: audit 2024-02-13T03:29:24.979788+0000 mon.a (mon.0) 1542 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi161-61759-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:25 smithi161 bash[19769]: cluster 2024-02-13T03:29:25.276904+0000 mgr.y (mgr.24473) 344 : cluster [DBG] pgmap v389: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:25.931 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:25 smithi183 bash[20824]: audit 2024-02-13T03:29:24.975753+0000 mon.a (mon.0) 1539 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:25.931 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:25 smithi183 bash[20824]: audit 2024-02-13T03:29:24.976326+0000 mon.a (mon.0) 1540 : audit [INF] from='client.48200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi161-61759-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:25.931 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:25 smithi183 bash[20824]: cluster 2024-02-13T03:29:24.976579+0000 mon.a (mon.0) 1541 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-02-13T03:29:25.931 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:25 smithi183 bash[20824]: audit 2024-02-13T03:29:24.977756+0000 mon.c (mon.2) 399 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi161-61759-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:25.931 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:25 smithi183 bash[20824]: audit 2024-02-13T03:29:24.979788+0000 mon.a (mon.0) 1542 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi161-61759-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:25.931 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:25 smithi183 bash[20824]: cluster 2024-02-13T03:29:25.276904+0000 mgr.y (mgr.24473) 344 : cluster [DBG] pgmap v389: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:26.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:29:25 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:29:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[19769]: audit 2024-02-13T03:29:25.571547+0000 mon.a (mon.0) 1543 : audit [INF] from='client.48194 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi161-61719-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi161-61719-35"}]': finished 2024-02-13T03:29:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[19769]: audit 2024-02-13T03:29:25.928581+0000 mgr.y (mgr.24473) 345 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[19769]: cluster 2024-02-13T03:29:25.977429+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-02-13T03:29:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[19769]: audit 2024-02-13T03:29:25.979240+0000 mon.c (mon.2) 400 : audit [INF] from='client.? v1:172.21.15.161:0/2785947240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi161-61726-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[19769]: audit 2024-02-13T03:29:25.981312+0000 mon.a (mon.0) 1545 : audit [INF] from='client.48206 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi161-61726-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[19769]: audit 2024-02-13T03:29:25.990045+0000 mon.a (mon.0) 1546 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[19769]: audit 2024-02-13T03:29:25.992810+0000 mon.b (mon.1) 206 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[27879]: audit 2024-02-13T03:29:25.571547+0000 mon.a (mon.0) 1543 : audit [INF] from='client.48194 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi161-61719-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi161-61719-35"}]': finished 2024-02-13T03:29:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[27879]: audit 2024-02-13T03:29:25.928581+0000 mgr.y (mgr.24473) 345 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[27879]: cluster 2024-02-13T03:29:25.977429+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-02-13T03:29:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[27879]: audit 2024-02-13T03:29:25.979240+0000 mon.c (mon.2) 400 : audit [INF] from='client.? v1:172.21.15.161:0/2785947240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi161-61726-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[27879]: audit 2024-02-13T03:29:25.981312+0000 mon.a (mon.0) 1545 : audit [INF] from='client.48206 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi161-61726-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[27879]: audit 2024-02-13T03:29:25.990045+0000 mon.a (mon.0) 1546 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:26 smithi161 bash[27879]: audit 2024-02-13T03:29:25.992810+0000 mon.b (mon.1) 206 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:26.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:26 smithi183 bash[20824]: audit 2024-02-13T03:29:25.571547+0000 mon.a (mon.0) 1543 : audit [INF] from='client.48194 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi161-61719-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi161-61719-35"}]': finished 2024-02-13T03:29:26.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:26 smithi183 bash[20824]: audit 2024-02-13T03:29:25.928581+0000 mgr.y (mgr.24473) 345 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:26.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:26 smithi183 bash[20824]: cluster 2024-02-13T03:29:25.977429+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-02-13T03:29:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:26 smithi183 bash[20824]: audit 2024-02-13T03:29:25.979240+0000 mon.c (mon.2) 400 : audit [INF] from='client.? v1:172.21.15.161:0/2785947240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi161-61726-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:26 smithi183 bash[20824]: audit 2024-02-13T03:29:25.981312+0000 mon.a (mon.0) 1545 : audit [INF] from='client.48206 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi161-61726-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:26 smithi183 bash[20824]: audit 2024-02-13T03:29:25.990045+0000 mon.a (mon.0) 1546 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:26.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:26 smithi183 bash[20824]: audit 2024-02-13T03:29:25.992810+0000 mon.b (mon.1) 206 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:27.592 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[19769]: audit 2024-02-13T03:29:26.573729+0000 mon.a (mon.0) 1547 : audit [INF] from='client.48200 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi161-61759-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi161-61759-2"}]': finished 2024-02-13T03:29:27.592 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[19769]: audit 2024-02-13T03:29:26.573948+0000 mon.a (mon.0) 1548 : audit [INF] from='client.48206 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi161-61726-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:27.592 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[19769]: audit 2024-02-13T03:29:26.574182+0000 mon.a (mon.0) 1549 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:27.593 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[19769]: cluster 2024-02-13T03:29:26.574294+0000 mon.a (mon.0) 1550 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-02-13T03:29:27.593 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[19769]: audit 2024-02-13T03:29:26.579556+0000 mon.b (mon.1) 207 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-27"}]: dispatch 2024-02-13T03:29:27.593 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[19769]: audit 2024-02-13T03:29:26.585235+0000 mon.a (mon.0) 1551 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-27"}]: dispatch 2024-02-13T03:29:27.593 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[19769]: cluster 2024-02-13T03:29:27.277947+0000 mgr.y (mgr.24473) 346 : cluster [DBG] pgmap v392: 276 pgs: 25 creating+peering, 23 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:27.593 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[19769]: audit 2024-02-13T03:29:27.578147+0000 mon.a (mon.0) 1552 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-27"}]': finished 2024-02-13T03:29:27.593 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[19769]: cluster 2024-02-13T03:29:27.578357+0000 mon.a (mon.0) 1553 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-02-13T03:29:27.593 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[19769]: audit 2024-02-13T03:29:27.584126+0000 mon.b (mon.1) 208 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-27", "mode": "writeback"}]: dispatch 2024-02-13T03:29:27.593 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[19769]: audit 2024-02-13T03:29:27.584625+0000 mon.c (mon.2) 401 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:27.838 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAid=5 2024-02-13T03:29:27.839 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:5c6b0b28:::7:head 2024-02-13T03:29:27.839 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:5c6b0b28:::7:head expected=13:5c6b0b28:::7:head 2024-02-13T03:29:27.839 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:5c6b0b28:::7:head -> 7 2024-02-13T03:29:27.839 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=7 expected=7 2024-02-13T03:29:27.839 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:62a1935d:::14:head 2024-02-13T03:29:27.839 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:62a1935d:::14:head expected=13:62a1935d:::14:head 2024-02-13T03:29:27.839 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:62a1935d:::14:head -> 14 2024-02-13T03:29:27.839 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=14 expected=14 2024-02-13T03:29:27.839 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:52ea6a34:::10:head 2024-02-13T03:29:27.839 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:52ea6a34:::10:head expected=13:52ea6a34:::10:head 2024-02-13T03:29:27.839 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:52ea6a34:::10:head -> 10 2024-02-13T03:29:27.839 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=10 expected=10 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:02547ec2:::1:head 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:02547ec2:::1:head expected=13:02547ec2:::1:head 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:02547ec2:::1:head -> 1 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=1 expected=1 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:566253c9:::13:head 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:566253c9:::13:head expected=13:566253c9:::13:head 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:566253c9:::13:head -> 13 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=13 expected=13 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:89d3ae78:::11:head 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:89d3ae78:::11:head expected=13:89d3ae78:::11:head 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:89d3ae78:::11:head -> 11 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=11 expected=11 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:863748b0:::15:head 2024-02-13T03:29:27.840 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:863748b0:::15:head expected=13:863748b0:::15:head 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:863748b0:::15:head -> 15 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=15 expected=15 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : seek to 13:6cac518f:::0:head 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : cursor()=13:6cac518f:::0:head expected=13:6cac518f:::0:head 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: > 13:6cac518f:::0:head -> 0 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: : entry=0 expected=0 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (237 ms) 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (119250 ms) 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: split 0/5 -> MIN 13:33333333::::head 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: split 1/5 -> 13:33333333::::head 13:66666666::::head 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: split 2/5 -> 13:66666666::::head 13:99999999::::head 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: split 3/5 -> 13:99999999::::head 13:cccccccc::::head 2024-02-13T03:29:27.841 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: split 4/5 -> 13:cccccccc::::head MAX 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (24958 ms) 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [----------] 7 tests from LibRadosList (144951 ms total) 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1225 ms) 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: myset foo1,foo2,foo3 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo1 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo2 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo3 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: myset foo1,foo4,foo5 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo4 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo5 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo1 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: myset foo6,foo7 2024-02-13T03:29:27.842 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo7 2024-02-13T03:29:27.843 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: foo6 2024-02-13T03:29:27.843 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-13T03:29:27.843 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: ns1:foo4 2024-02-13T03:29:27.843 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: ns1:foo5 2024-02-13T03:29:27.843 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: ns2:foo7 2024-02-13T03:29:27.843 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: ns2:foo6 2024-02-13T03:29:27.843 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: ns1:foo1 2024-02-13T03:29:27.843 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: :foo1 2024-02-13T03:29:27.843 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: :foo2 2024-02-13T03:29:27.843 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: :foo3 2024-02-13T03:29:27.843 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (36 ms) 2024-02-13T03:29:27.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[27879]: audit 2024-02-13T03:29:26.573729+0000 mon.a (mon.0) 1547 : audit [INF] from='client.48200 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi161-61759-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi161-61759-2"}]': finished 2024-02-13T03:29:27.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[27879]: audit 2024-02-13T03:29:26.573948+0000 mon.a (mon.0) 1548 : audit [INF] from='client.48206 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi161-61726-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:27.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[27879]: audit 2024-02-13T03:29:26.574182+0000 mon.a (mon.0) 1549 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:27.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[27879]: cluster 2024-02-13T03:29:26.574294+0000 mon.a (mon.0) 1550 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-02-13T03:29:27.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[27879]: audit 2024-02-13T03:29:26.579556+0000 mon.b (mon.1) 207 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-27"}]: dispatch 2024-02-13T03:29:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[27879]: audit 2024-02-13T03:29:26.585235+0000 mon.a (mon.0) 1551 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-27"}]: dispatch 2024-02-13T03:29:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[27879]: cluster 2024-02-13T03:29:27.277947+0000 mgr.y (mgr.24473) 346 : cluster [DBG] pgmap v392: 276 pgs: 25 creating+peering, 23 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[27879]: audit 2024-02-13T03:29:27.578147+0000 mon.a (mon.0) 1552 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-27"}]': finished 2024-02-13T03:29:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[27879]: cluster 2024-02-13T03:29:27.578357+0000 mon.a (mon.0) 1553 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-02-13T03:29:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[27879]: audit 2024-02-13T03:29:27.584126+0000 mon.b (mon.1) 208 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-27", "mode": "writeback"}]: dispatch 2024-02-13T03:29:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:27 smithi161 bash[27879]: audit 2024-02-13T03:29:27.584625+0000 mon.c (mon.2) 401 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:27.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:27 smithi183 bash[20824]: audit 2024-02-13T03:29:26.573729+0000 mon.a (mon.0) 1547 : audit [INF] from='client.48200 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi161-61759-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi161-61759-2"}]': finished 2024-02-13T03:29:27.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:27 smithi183 bash[20824]: audit 2024-02-13T03:29:26.573948+0000 mon.a (mon.0) 1548 : audit [INF] from='client.48206 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi161-61726-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:27 smithi183 bash[20824]: audit 2024-02-13T03:29:26.574182+0000 mon.a (mon.0) 1549 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:27 smithi183 bash[20824]: cluster 2024-02-13T03:29:26.574294+0000 mon.a (mon.0) 1550 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-02-13T03:29:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:27 smithi183 bash[20824]: audit 2024-02-13T03:29:26.579556+0000 mon.b (mon.1) 207 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-27"}]: dispatch 2024-02-13T03:29:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:27 smithi183 bash[20824]: audit 2024-02-13T03:29:26.585235+0000 mon.a (mon.0) 1551 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-27"}]: dispatch 2024-02-13T03:29:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:27 smithi183 bash[20824]: cluster 2024-02-13T03:29:27.277947+0000 mgr.y (mgr.24473) 346 : cluster [DBG] pgmap v392: 276 pgs: 25 creating+peering, 23 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:27 smithi183 bash[20824]: audit 2024-02-13T03:29:27.578147+0000 mon.a (mon.0) 1552 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-27"}]': finished 2024-02-13T03:29:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:27 smithi183 bash[20824]: cluster 2024-02-13T03:29:27.578357+0000 mon.a (mon.0) 1553 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-02-13T03:29:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:27 smithi183 bash[20824]: audit 2024-02-13T03:29:27.584126+0000 mon.b (mon.1) 208 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-27", "mode": "writeback"}]: dispatch 2024-02-13T03:29:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:27 smithi183 bash[20824]: audit 2024-02-13T03:29:27.584625+0000 mon.c (mon.2) 401 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[27879]: audit 2024-02-13T03:29:27.583313+0000 mon.a (mon.0) 1554 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-27", "mode": "writeback"}]: dispatch 2024-02-13T03:29:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[27879]: audit 2024-02-13T03:29:27.585329+0000 mon.a (mon.0) 1555 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[27879]: audit 2024-02-13T03:29:27.610702+0000 mon.c (mon.2) 402 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[27879]: audit 2024-02-13T03:29:27.611216+0000 mon.a (mon.0) 1556 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[27879]: audit 2024-02-13T03:29:27.611856+0000 mon.c (mon.2) 403 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[27879]: audit 2024-02-13T03:29:27.612128+0000 mon.a (mon.0) 1557 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[27879]: audit 2024-02-13T03:29:27.612549+0000 mon.c (mon.2) 404 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[27879]: audit 2024-02-13T03:29:27.612822+0000 mon.a (mon.0) 1558 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[27879]: cluster 2024-02-13T03:29:28.570106+0000 mon.a (mon.0) 1559 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[27879]: audit 2024-02-13T03:29:28.581950+0000 mon.a (mon.0) 1560 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-27", "mode": "writeback"}]': finished 2024-02-13T03:29:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[27879]: audit 2024-02-13T03:29:28.582169+0000 mon.a (mon.0) 1561 : audit [INF] from='client.48194 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]': finished 2024-02-13T03:29:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[27879]: audit 2024-02-13T03:29:28.582290+0000 mon.a (mon.0) 1562 : audit [INF] from='client.48212 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[27879]: cluster 2024-02-13T03:29:28.582438+0000 mon.a (mon.0) 1563 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-02-13T03:29:28.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[19769]: audit 2024-02-13T03:29:27.583313+0000 mon.a (mon.0) 1554 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-27", "mode": "writeback"}]: dispatch 2024-02-13T03:29:28.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[19769]: audit 2024-02-13T03:29:27.585329+0000 mon.a (mon.0) 1555 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:28.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[19769]: audit 2024-02-13T03:29:27.610702+0000 mon.c (mon.2) 402 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:28.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[19769]: audit 2024-02-13T03:29:27.611216+0000 mon.a (mon.0) 1556 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:28.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[19769]: audit 2024-02-13T03:29:27.611856+0000 mon.c (mon.2) 403 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:28.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[19769]: audit 2024-02-13T03:29:27.612128+0000 mon.a (mon.0) 1557 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:28.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[19769]: audit 2024-02-13T03:29:27.612549+0000 mon.c (mon.2) 404 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:28.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[19769]: audit 2024-02-13T03:29:27.612822+0000 mon.a (mon.0) 1558 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:28.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[19769]: cluster 2024-02-13T03:29:28.570106+0000 mon.a (mon.0) 1559 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:28.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[19769]: audit 2024-02-13T03:29:28.581950+0000 mon.a (mon.0) 1560 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-27", "mode": "writeback"}]': finished 2024-02-13T03:29:28.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[19769]: audit 2024-02-13T03:29:28.582169+0000 mon.a (mon.0) 1561 : audit [INF] from='client.48194 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]': finished 2024-02-13T03:29:28.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[19769]: audit 2024-02-13T03:29:28.582290+0000 mon.a (mon.0) 1562 : audit [INF] from='client.48212 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:28.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:28 smithi161 bash[19769]: cluster 2024-02-13T03:29:28.582438+0000 mon.a (mon.0) 1563 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-02-13T03:29:28.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:28 smithi183 bash[20824]: audit 2024-02-13T03:29:27.583313+0000 mon.a (mon.0) 1554 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-27", "mode": "writeback"}]: dispatch 2024-02-13T03:29:28.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:28 smithi183 bash[20824]: audit 2024-02-13T03:29:27.585329+0000 mon.a (mon.0) 1555 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:28.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:28 smithi183 bash[20824]: audit 2024-02-13T03:29:27.610702+0000 mon.c (mon.2) 402 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:28.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:28 smithi183 bash[20824]: audit 2024-02-13T03:29:27.611216+0000 mon.a (mon.0) 1556 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:28.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:28 smithi183 bash[20824]: audit 2024-02-13T03:29:27.611856+0000 mon.c (mon.2) 403 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:28.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:28 smithi183 bash[20824]: audit 2024-02-13T03:29:27.612128+0000 mon.a (mon.0) 1557 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:28.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:28 smithi183 bash[20824]: audit 2024-02-13T03:29:27.612549+0000 mon.c (mon.2) 404 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:28.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:28 smithi183 bash[20824]: audit 2024-02-13T03:29:27.612822+0000 mon.a (mon.0) 1558 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:28.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:28 smithi183 bash[20824]: cluster 2024-02-13T03:29:28.570106+0000 mon.a (mon.0) 1559 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:28.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:28 smithi183 bash[20824]: audit 2024-02-13T03:29:28.581950+0000 mon.a (mon.0) 1560 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-27", "mode": "writeback"}]': finished 2024-02-13T03:29:28.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:28 smithi183 bash[20824]: audit 2024-02-13T03:29:28.582169+0000 mon.a (mon.0) 1561 : audit [INF] from='client.48194 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi161-61719-35"}]': finished 2024-02-13T03:29:28.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:28 smithi183 bash[20824]: audit 2024-02-13T03:29:28.582290+0000 mon.a (mon.0) 1562 : audit [INF] from='client.48212 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:28.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:28 smithi183 bash[20824]: cluster 2024-02-13T03:29:28.582438+0000 mon.a (mon.0) 1563 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-02-13T03:29:29.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:29:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:29:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:29:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[27879]: audit 2024-02-13T03:29:28.587371+0000 mon.c (mon.2) 405 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[27879]: audit 2024-02-13T03:29:28.587989+0000 mon.c (mon.2) 406 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[27879]: audit 2024-02-13T03:29:28.588091+0000 mon.a (mon.0) 1564 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[27879]: audit 2024-02-13T03:29:28.588605+0000 mon.c (mon.2) 407 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[27879]: audit 2024-02-13T03:29:28.589497+0000 mon.a (mon.0) 1565 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[27879]: audit 2024-02-13T03:29:28.590288+0000 mon.a (mon.0) 1566 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:30.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[27879]: cluster 2024-02-13T03:29:29.278834+0000 mgr.y (mgr.24473) 347 : cluster [DBG] pgmap v395: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-13T03:29:30.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[27879]: audit 2024-02-13T03:29:29.583749+0000 mon.a (mon.0) 1567 : audit [INF] from='client.48194 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]': finished 2024-02-13T03:29:30.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[19769]: audit 2024-02-13T03:29:28.587371+0000 mon.c (mon.2) 405 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:30.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[19769]: audit 2024-02-13T03:29:28.587989+0000 mon.c (mon.2) 406 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:30.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[19769]: audit 2024-02-13T03:29:28.588091+0000 mon.a (mon.0) 1564 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:30.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[19769]: audit 2024-02-13T03:29:28.588605+0000 mon.c (mon.2) 407 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:30.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[19769]: audit 2024-02-13T03:29:28.589497+0000 mon.a (mon.0) 1565 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:30.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[19769]: audit 2024-02-13T03:29:28.590288+0000 mon.a (mon.0) 1566 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:30.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[19769]: cluster 2024-02-13T03:29:29.278834+0000 mgr.y (mgr.24473) 347 : cluster [DBG] pgmap v395: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-13T03:29:30.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:30 smithi161 bash[19769]: audit 2024-02-13T03:29:29.583749+0000 mon.a (mon.0) 1567 : audit [INF] from='client.48194 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]': finished 2024-02-13T03:29:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:30 smithi183 bash[20824]: audit 2024-02-13T03:29:28.587371+0000 mon.c (mon.2) 405 : audit [INF] from='client.? v1:172.21.15.161:0/1555678974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:30 smithi183 bash[20824]: audit 2024-02-13T03:29:28.587989+0000 mon.c (mon.2) 406 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:30 smithi183 bash[20824]: audit 2024-02-13T03:29:28.588091+0000 mon.a (mon.0) 1564 : audit [INF] from='client.48194 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]: dispatch 2024-02-13T03:29:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:30 smithi183 bash[20824]: audit 2024-02-13T03:29:28.588605+0000 mon.c (mon.2) 407 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:30 smithi183 bash[20824]: audit 2024-02-13T03:29:28.589497+0000 mon.a (mon.0) 1565 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:30 smithi183 bash[20824]: audit 2024-02-13T03:29:28.590288+0000 mon.a (mon.0) 1566 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:30 smithi183 bash[20824]: cluster 2024-02-13T03:29:29.278834+0000 mgr.y (mgr.24473) 347 : cluster [DBG] pgmap v395: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-13T03:29:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:30 smithi183 bash[20824]: audit 2024-02-13T03:29:29.583749+0000 mon.a (mon.0) 1567 : audit [INF] from='client.48194 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi161-61719-35"}]': finished 2024-02-13T03:29:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:29.613037+0000 mon.c (mon.2) 408 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.012222+0000 mon.a (mon.0) 1568 : audit [INF] from='client.48200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]': finished 2024-02-13T03:29:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: cluster 2024-02-13T03:29:30.012465+0000 mon.a (mon.0) 1569 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-02-13T03:29:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.013147+0000 mon.c (mon.2) 409 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.013926+0000 mon.a (mon.0) 1570 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.014372+0000 mon.a (mon.0) 1571 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.015228+0000 mon.c (mon.2) 410 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.023034+0000 mon.a (mon.0) 1572 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.023999+0000 mon.c (mon.2) 411 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.029757+0000 mon.a (mon.0) 1573 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.036360+0000 mon.a (mon.0) 1574 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.039111+0000 mon.b (mon.1) 209 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.589495+0000 mon.a (mon.0) 1575 : audit [INF] from='client.48212 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-45"}]': finished 2024-02-13T03:29:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.589840+0000 mon.a (mon.0) 1576 : audit [INF] from='client.48200 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]': finished 2024-02-13T03:29:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.590059+0000 mon.a (mon.0) 1577 : audit [INF] from='client.48218 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.590226+0000 mon.a (mon.0) 1578 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: cluster 2024-02-13T03:29:30.590368+0000 mon.a (mon.0) 1579 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-13T03:29:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.593462+0000 mon.a (mon.0) 1580 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27"}]: dispatch 2024-02-13T03:29:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.595543+0000 mon.b (mon.1) 210 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27"}]: dispatch 2024-02-13T03:29:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.600055+0000 mon.c (mon.2) 412 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi161-61719-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[19769]: audit 2024-02-13T03:29:30.600601+0000 mon.a (mon.0) 1581 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi161-61719-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.430 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:29:31] "GET /metrics HTTP/1.1" 200 195410 "" "Prometheus/2.33.4" 2024-02-13T03:29:31.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:29.613037+0000 mon.c (mon.2) 408 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.012222+0000 mon.a (mon.0) 1568 : audit [INF] from='client.48200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]': finished 2024-02-13T03:29:31.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: cluster 2024-02-13T03:29:30.012465+0000 mon.a (mon.0) 1569 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-02-13T03:29:31.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.013147+0000 mon.c (mon.2) 409 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:31.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.013926+0000 mon.a (mon.0) 1570 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.014372+0000 mon.a (mon.0) 1571 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:31.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.015228+0000 mon.c (mon.2) 410 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.023034+0000 mon.a (mon.0) 1572 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.023999+0000 mon.c (mon.2) 411 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.029757+0000 mon.a (mon.0) 1573 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.036360+0000 mon.a (mon.0) 1574 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.039111+0000 mon.b (mon.1) 209 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.589495+0000 mon.a (mon.0) 1575 : audit [INF] from='client.48212 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-45"}]': finished 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.589840+0000 mon.a (mon.0) 1576 : audit [INF] from='client.48200 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]': finished 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.590059+0000 mon.a (mon.0) 1577 : audit [INF] from='client.48218 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.590226+0000 mon.a (mon.0) 1578 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: cluster 2024-02-13T03:29:30.590368+0000 mon.a (mon.0) 1579 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.593462+0000 mon.a (mon.0) 1580 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27"}]: dispatch 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.595543+0000 mon.b (mon.1) 210 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27"}]: dispatch 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.600055+0000 mon.c (mon.2) 412 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi161-61719-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:31 smithi161 bash[27879]: audit 2024-02-13T03:29:30.600601+0000 mon.a (mon.0) 1581 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi161-61719-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:29.613037+0000 mon.c (mon.2) 408 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.012222+0000 mon.a (mon.0) 1568 : audit [INF] from='client.48200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi161-61759-2"}]': finished 2024-02-13T03:29:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: cluster 2024-02-13T03:29:30.012465+0000 mon.a (mon.0) 1569 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-02-13T03:29:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.013147+0000 mon.c (mon.2) 409 : audit [INF] from='client.? v1:172.21.15.161:0/593520976' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.013926+0000 mon.a (mon.0) 1570 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.014372+0000 mon.a (mon.0) 1571 : audit [INF] from='client.48200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]: dispatch 2024-02-13T03:29:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.015228+0000 mon.c (mon.2) 410 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.023034+0000 mon.a (mon.0) 1572 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.023999+0000 mon.c (mon.2) 411 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.029757+0000 mon.a (mon.0) 1573 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.036360+0000 mon.a (mon.0) 1574 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:31.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.039111+0000 mon.b (mon.1) 209 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:31.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.589495+0000 mon.a (mon.0) 1575 : audit [INF] from='client.48212 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-45"}]': finished 2024-02-13T03:29:31.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.589840+0000 mon.a (mon.0) 1576 : audit [INF] from='client.48200 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi161-61759-2"}]': finished 2024-02-13T03:29:31.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.590059+0000 mon.a (mon.0) 1577 : audit [INF] from='client.48218 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:31.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.590226+0000 mon.a (mon.0) 1578 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:31.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: cluster 2024-02-13T03:29:30.590368+0000 mon.a (mon.0) 1579 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-13T03:29:31.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.593462+0000 mon.a (mon.0) 1580 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27"}]: dispatch 2024-02-13T03:29:31.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.595543+0000 mon.b (mon.1) 210 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27"}]: dispatch 2024-02-13T03:29:31.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.600055+0000 mon.c (mon.2) 412 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi161-61719-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:31.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:31 smithi183 bash[20824]: audit 2024-02-13T03:29:30.600601+0000 mon.a (mon.0) 1581 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi161-61719-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:32.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:32 smithi161 bash[27879]: cluster 2024-02-13T03:29:31.281142+0000 mgr.y (mgr.24473) 348 : cluster [DBG] pgmap v398: 236 pgs: 8 creating+peering, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-13T03:29:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:32 smithi161 bash[27879]: cluster 2024-02-13T03:29:31.583244+0000 mon.a (mon.0) 1582 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:32 smithi161 bash[27879]: audit 2024-02-13T03:29:31.597532+0000 mon.a (mon.0) 1583 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27"}]': finished 2024-02-13T03:29:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:32 smithi161 bash[27879]: cluster 2024-02-13T03:29:32.011988+0000 mon.a (mon.0) 1584 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-13T03:29:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:32 smithi161 bash[27879]: audit 2024-02-13T03:29:32.018057+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? v1:172.21.15.161:0/1417036247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61759-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:32 smithi161 bash[19769]: cluster 2024-02-13T03:29:31.281142+0000 mgr.y (mgr.24473) 348 : cluster [DBG] pgmap v398: 236 pgs: 8 creating+peering, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-13T03:29:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:32 smithi161 bash[19769]: cluster 2024-02-13T03:29:31.583244+0000 mon.a (mon.0) 1582 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:32 smithi161 bash[19769]: audit 2024-02-13T03:29:31.597532+0000 mon.a (mon.0) 1583 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27"}]': finished 2024-02-13T03:29:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:32 smithi161 bash[19769]: cluster 2024-02-13T03:29:32.011988+0000 mon.a (mon.0) 1584 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-13T03:29:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:32 smithi161 bash[19769]: audit 2024-02-13T03:29:32.018057+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? v1:172.21.15.161:0/1417036247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61759-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:32.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:32 smithi183 bash[20824]: cluster 2024-02-13T03:29:31.281142+0000 mgr.y (mgr.24473) 348 : cluster [DBG] pgmap v398: 236 pgs: 8 creating+peering, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-13T03:29:32.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:32 smithi183 bash[20824]: cluster 2024-02-13T03:29:31.583244+0000 mon.a (mon.0) 1582 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:32.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:32 smithi183 bash[20824]: audit 2024-02-13T03:29:31.597532+0000 mon.a (mon.0) 1583 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-27"}]': finished 2024-02-13T03:29:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:32 smithi183 bash[20824]: cluster 2024-02-13T03:29:32.011988+0000 mon.a (mon.0) 1584 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-13T03:29:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:32 smithi183 bash[20824]: audit 2024-02-13T03:29:32.018057+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? v1:172.21.15.161:0/1417036247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61759-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:33.540 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-02-13T03:29:33.540 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 1 0 2024-02-13T03:29:33.541 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 10 0 2024-02-13T03:29:33.541 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 13 0 2024-02-13T03:29:33.541 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 7 0 2024-02-13T03:29:33.541 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 14 0 2024-02-13T03:29:33.541 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 0 0 2024-02-13T03:29:33.541 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 15 0 2024-02-13T03:29:33.541 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 11 0 2024-02-13T03:29:33.541 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 5 0 2024-02-13T03:29:33.541 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 8 0 2024-02-13T03:29:33.541 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 6 0 2024-02-13T03:29:33.541 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 3 0 2024-02-13T03:29:33.541 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 4 0 2024-02-13T03:29:33.541 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 12 0 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 9 0 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 2 0 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (50 ms) 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [----------] 3 tests from LibRadosListEC (1311 ms total) 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (2934 ms) 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [----------] 1 test from LibRadosListNP (2934 ms total) 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [----------] Global test environment tear-down 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [==========] 11 tests from 3 test suites ran. (167304 ms total) 2024-02-13T03:29:33.542 INFO:tasks.workunit.client.0.smithi161.stdout: api_list: [ PASSED ] 11 tests. 2024-02-13T03:29:33.543 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:29:33.543 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62051 2024-02-13T03:29:33.543 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 62051 2024-02-13T03:29:33.543 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:29:33.543 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62020 2024-02-13T03:29:33.543 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 62020 2024-02-13T03:29:33.543 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:29:33.543 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62207 2024-02-13T03:29:33.543 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 62207 2024-02-13T03:29:33.543 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:29:33.543 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61722 2024-02-13T03:29:33.543 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61722 2024-02-13T03:29:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: audit 2024-02-13T03:29:32.595386+0000 mon.a (mon.0) 1586 : audit [INF] from='client.48218 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi161-61719-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi161-61719-36"}]': finished 2024-02-13T03:29:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: audit 2024-02-13T03:29:32.595747+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? v1:172.21.15.161:0/1417036247' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61759-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: cluster 2024-02-13T03:29:32.596102+0000 mon.a (mon.0) 1588 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-13T03:29:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: audit 2024-02-13T03:29:32.598501+0000 mon.c (mon.2) 413 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: audit 2024-02-13T03:29:32.603227+0000 mon.a (mon.0) 1589 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: audit 2024-02-13T03:29:33.038701+0000 mon.a (mon.0) 1590 : audit [INF] from='client.? v1:172.21.15.161:0/1417036247' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi161-61759-3","pool2":"test-rados-api-smithi161-61759-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-13T03:29:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: cluster 2024-02-13T03:29:33.282030+0000 mgr.y (mgr.24473) 349 : cluster [DBG] pgmap v401: 236 pgs: 40 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-13T03:29:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: audit 2024-02-13T03:29:33.520344+0000 mon.a (mon.0) 1591 : audit [INF] from='client.48212 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]': finished 2024-02-13T03:29:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: audit 2024-02-13T03:29:33.520522+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? v1:172.21.15.161:0/1417036247' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi161-61759-3","pool2":"test-rados-api-smithi161-61759-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-13T03:29:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: cluster 2024-02-13T03:29:33.520656+0000 mon.a (mon.0) 1593 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-13T03:29:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: audit 2024-02-13T03:29:33.521613+0000 mon.c (mon.2) 414 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:33.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: audit 2024-02-13T03:29:33.530635+0000 mon.b (mon.1) 211 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:33.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: audit 2024-02-13T03:29:33.534160+0000 mon.a (mon.0) 1594 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:33.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[27879]: audit 2024-02-13T03:29:33.534985+0000 mon.a (mon.0) 1595 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: audit 2024-02-13T03:29:32.595386+0000 mon.a (mon.0) 1586 : audit [INF] from='client.48218 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi161-61719-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi161-61719-36"}]': finished 2024-02-13T03:29:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: audit 2024-02-13T03:29:32.595747+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? v1:172.21.15.161:0/1417036247' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61759-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: cluster 2024-02-13T03:29:32.596102+0000 mon.a (mon.0) 1588 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-13T03:29:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: audit 2024-02-13T03:29:32.598501+0000 mon.c (mon.2) 413 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: audit 2024-02-13T03:29:32.603227+0000 mon.a (mon.0) 1589 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: audit 2024-02-13T03:29:33.038701+0000 mon.a (mon.0) 1590 : audit [INF] from='client.? v1:172.21.15.161:0/1417036247' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi161-61759-3","pool2":"test-rados-api-smithi161-61759-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-13T03:29:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: cluster 2024-02-13T03:29:33.282030+0000 mgr.y (mgr.24473) 349 : cluster [DBG] pgmap v401: 236 pgs: 40 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-13T03:29:33.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: audit 2024-02-13T03:29:33.520344+0000 mon.a (mon.0) 1591 : audit [INF] from='client.48212 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]': finished 2024-02-13T03:29:33.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: audit 2024-02-13T03:29:33.520522+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? v1:172.21.15.161:0/1417036247' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi161-61759-3","pool2":"test-rados-api-smithi161-61759-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-13T03:29:33.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: cluster 2024-02-13T03:29:33.520656+0000 mon.a (mon.0) 1593 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-13T03:29:33.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: audit 2024-02-13T03:29:33.521613+0000 mon.c (mon.2) 414 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:33.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: audit 2024-02-13T03:29:33.530635+0000 mon.b (mon.1) 211 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:33.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: audit 2024-02-13T03:29:33.534160+0000 mon.a (mon.0) 1594 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:33.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:33 smithi161 bash[19769]: audit 2024-02-13T03:29:33.534985+0000 mon.a (mon.0) 1595 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:33.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: audit 2024-02-13T03:29:32.595386+0000 mon.a (mon.0) 1586 : audit [INF] from='client.48218 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi161-61719-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi161-61719-36"}]': finished 2024-02-13T03:29:33.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: audit 2024-02-13T03:29:32.595747+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? v1:172.21.15.161:0/1417036247' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61759-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: cluster 2024-02-13T03:29:32.596102+0000 mon.a (mon.0) 1588 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-13T03:29:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: audit 2024-02-13T03:29:32.598501+0000 mon.c (mon.2) 413 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: audit 2024-02-13T03:29:32.603227+0000 mon.a (mon.0) 1589 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: audit 2024-02-13T03:29:33.038701+0000 mon.a (mon.0) 1590 : audit [INF] from='client.? v1:172.21.15.161:0/1417036247' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi161-61759-3","pool2":"test-rados-api-smithi161-61759-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-13T03:29:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: cluster 2024-02-13T03:29:33.282030+0000 mgr.y (mgr.24473) 349 : cluster [DBG] pgmap v401: 236 pgs: 40 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-13T03:29:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: audit 2024-02-13T03:29:33.520344+0000 mon.a (mon.0) 1591 : audit [INF] from='client.48212 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-45"}]': finished 2024-02-13T03:29:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: audit 2024-02-13T03:29:33.520522+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? v1:172.21.15.161:0/1417036247' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi161-61759-3","pool2":"test-rados-api-smithi161-61759-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-13T03:29:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: cluster 2024-02-13T03:29:33.520656+0000 mon.a (mon.0) 1593 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-13T03:29:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: audit 2024-02-13T03:29:33.521613+0000 mon.c (mon.2) 414 : audit [INF] from='client.? v1:172.21.15.161:0/3304980015' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: audit 2024-02-13T03:29:33.530635+0000 mon.b (mon.1) 211 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: audit 2024-02-13T03:29:33.534160+0000 mon.a (mon.0) 1594 : audit [INF] from='client.48212 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]: dispatch 2024-02-13T03:29:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:33 smithi183 bash[20824]: audit 2024-02-13T03:29:33.534985+0000 mon.a (mon.0) 1595 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[27879]: audit 2024-02-13T03:29:34.523735+0000 mon.a (mon.0) 1596 : audit [INF] from='client.48212 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]': finished 2024-02-13T03:29:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[27879]: audit 2024-02-13T03:29:34.524075+0000 mon.a (mon.0) 1597 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[27879]: cluster 2024-02-13T03:29:34.524243+0000 mon.a (mon.0) 1598 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-13T03:29:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[27879]: audit 2024-02-13T03:29:34.526775+0000 mon.c (mon.2) 415 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[27879]: audit 2024-02-13T03:29:34.528479+0000 mon.a (mon.0) 1599 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[27879]: audit 2024-02-13T03:29:34.528940+0000 mon.a (mon.0) 1600 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[27879]: audit 2024-02-13T03:29:34.529315+0000 mon.b (mon.1) 212 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[27879]: audit 2024-02-13T03:29:34.554997+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[27879]: audit 2024-02-13T03:29:34.556238+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:35.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[27879]: audit 2024-02-13T03:29:34.556644+0000 mon.a (mon.0) 1603 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:35.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[27879]: audit 2024-02-13T03:29:34.780738+0000 mon.a (mon.0) 1604 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:29:35.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[27879]: cluster 2024-02-13T03:29:35.282837+0000 mgr.y (mgr.24473) 350 : cluster [DBG] pgmap v404: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[19769]: audit 2024-02-13T03:29:34.523735+0000 mon.a (mon.0) 1596 : audit [INF] from='client.48212 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]': finished 2024-02-13T03:29:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[19769]: audit 2024-02-13T03:29:34.524075+0000 mon.a (mon.0) 1597 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[19769]: cluster 2024-02-13T03:29:34.524243+0000 mon.a (mon.0) 1598 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-13T03:29:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[19769]: audit 2024-02-13T03:29:34.526775+0000 mon.c (mon.2) 415 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[19769]: audit 2024-02-13T03:29:34.528479+0000 mon.a (mon.0) 1599 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[19769]: audit 2024-02-13T03:29:34.528940+0000 mon.a (mon.0) 1600 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[19769]: audit 2024-02-13T03:29:34.529315+0000 mon.b (mon.1) 212 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[19769]: audit 2024-02-13T03:29:34.554997+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:35.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[19769]: audit 2024-02-13T03:29:34.556238+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:35.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[19769]: audit 2024-02-13T03:29:34.556644+0000 mon.a (mon.0) 1603 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:35.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[19769]: audit 2024-02-13T03:29:34.780738+0000 mon.a (mon.0) 1604 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:29:35.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:35 smithi161 bash[19769]: cluster 2024-02-13T03:29:35.282837+0000 mgr.y (mgr.24473) 350 : cluster [DBG] pgmap v404: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:35.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:35 smithi183 bash[20824]: audit 2024-02-13T03:29:34.523735+0000 mon.a (mon.0) 1596 : audit [INF] from='client.48212 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-45"}]': finished 2024-02-13T03:29:35.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:35 smithi183 bash[20824]: audit 2024-02-13T03:29:34.524075+0000 mon.a (mon.0) 1597 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:35.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:35 smithi183 bash[20824]: cluster 2024-02-13T03:29:34.524243+0000 mon.a (mon.0) 1598 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-13T03:29:35.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:35 smithi183 bash[20824]: audit 2024-02-13T03:29:34.526775+0000 mon.c (mon.2) 415 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:35.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:35 smithi183 bash[20824]: audit 2024-02-13T03:29:34.528479+0000 mon.a (mon.0) 1599 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:35.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:35 smithi183 bash[20824]: audit 2024-02-13T03:29:34.528940+0000 mon.a (mon.0) 1600 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:35.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:35 smithi183 bash[20824]: audit 2024-02-13T03:29:34.529315+0000 mon.b (mon.1) 212 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:35.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:35 smithi183 bash[20824]: audit 2024-02-13T03:29:34.554997+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:35.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:35 smithi183 bash[20824]: audit 2024-02-13T03:29:34.556238+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:35.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:35 smithi183 bash[20824]: audit 2024-02-13T03:29:34.556644+0000 mon.a (mon.0) 1603 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:35.943 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:35 smithi183 bash[20824]: audit 2024-02-13T03:29:34.780738+0000 mon.a (mon.0) 1604 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:29:35.943 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:35 smithi183 bash[20824]: cluster 2024-02-13T03:29:35.282837+0000 mgr.y (mgr.24473) 350 : cluster [DBG] pgmap v404: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:36.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:29:35 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:29:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:36 smithi183 bash[20824]: audit 2024-02-13T03:29:35.529020+0000 mon.a (mon.0) 1605 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:36 smithi183 bash[20824]: audit 2024-02-13T03:29:35.529419+0000 mon.a (mon.0) 1606 : audit [INF] from='client.48218 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]': finished 2024-02-13T03:29:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:36 smithi183 bash[20824]: audit 2024-02-13T03:29:35.529635+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:36 smithi183 bash[20824]: cluster 2024-02-13T03:29:35.529928+0000 mon.a (mon.0) 1608 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-13T03:29:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:36 smithi183 bash[20824]: audit 2024-02-13T03:29:35.530877+0000 mon.c (mon.2) 416 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:36 smithi183 bash[20824]: audit 2024-02-13T03:29:35.531382+0000 mon.a (mon.0) 1609 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:36 smithi183 bash[20824]: audit 2024-02-13T03:29:35.531830+0000 mon.a (mon.0) 1610 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-29"}]: dispatch 2024-02-13T03:29:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:36 smithi183 bash[20824]: audit 2024-02-13T03:29:35.532014+0000 mon.a (mon.0) 1611 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:36 smithi183 bash[20824]: audit 2024-02-13T03:29:35.534248+0000 mon.b (mon.1) 213 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-29"}]: dispatch 2024-02-13T03:29:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:36 smithi183 bash[20824]: audit 2024-02-13T03:29:35.939836+0000 mgr.y (mgr.24473) 351 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:37.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[27879]: audit 2024-02-13T03:29:35.529020+0000 mon.a (mon.0) 1605 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:37.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[27879]: audit 2024-02-13T03:29:35.529419+0000 mon.a (mon.0) 1606 : audit [INF] from='client.48218 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]': finished 2024-02-13T03:29:37.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[27879]: audit 2024-02-13T03:29:35.529635+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:37.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[27879]: cluster 2024-02-13T03:29:35.529928+0000 mon.a (mon.0) 1608 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-13T03:29:37.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[27879]: audit 2024-02-13T03:29:35.530877+0000 mon.c (mon.2) 416 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:37.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[27879]: audit 2024-02-13T03:29:35.531382+0000 mon.a (mon.0) 1609 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:37.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[27879]: audit 2024-02-13T03:29:35.531830+0000 mon.a (mon.0) 1610 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-29"}]: dispatch 2024-02-13T03:29:37.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[27879]: audit 2024-02-13T03:29:35.532014+0000 mon.a (mon.0) 1611 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:37.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[27879]: audit 2024-02-13T03:29:35.534248+0000 mon.b (mon.1) 213 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-29"}]: dispatch 2024-02-13T03:29:37.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[27879]: audit 2024-02-13T03:29:35.939836+0000 mgr.y (mgr.24473) 351 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[19769]: audit 2024-02-13T03:29:35.529020+0000 mon.a (mon.0) 1605 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[19769]: audit 2024-02-13T03:29:35.529419+0000 mon.a (mon.0) 1606 : audit [INF] from='client.48218 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi161-61719-36"}]': finished 2024-02-13T03:29:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[19769]: audit 2024-02-13T03:29:35.529635+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi161-61726-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[19769]: cluster 2024-02-13T03:29:35.529928+0000 mon.a (mon.0) 1608 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-13T03:29:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[19769]: audit 2024-02-13T03:29:35.530877+0000 mon.c (mon.2) 416 : audit [INF] from='client.? v1:172.21.15.161:0/3079190378' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[19769]: audit 2024-02-13T03:29:35.531382+0000 mon.a (mon.0) 1609 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[19769]: audit 2024-02-13T03:29:35.531830+0000 mon.a (mon.0) 1610 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-29"}]: dispatch 2024-02-13T03:29:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[19769]: audit 2024-02-13T03:29:35.532014+0000 mon.a (mon.0) 1611 : audit [INF] from='client.48218 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]: dispatch 2024-02-13T03:29:37.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[19769]: audit 2024-02-13T03:29:35.534248+0000 mon.b (mon.1) 213 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-29"}]: dispatch 2024-02-13T03:29:37.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:36 smithi161 bash[19769]: audit 2024-02-13T03:29:35.939836+0000 mgr.y (mgr.24473) 351 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:36.954800+0000 mon.a (mon.0) 1612 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-29"}]': finished 2024-02-13T03:29:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:36.955072+0000 mon.a (mon.0) 1613 : audit [INF] from='client.48218 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]': finished 2024-02-13T03:29:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: cluster 2024-02-13T03:29:36.955283+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-13T03:29:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:36.957825+0000 mon.a (mon.0) 1615 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-29", "mode": "writeback"}]: dispatch 2024-02-13T03:29:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:36.959374+0000 mon.b (mon.1) 214 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-29", "mode": "writeback"}]: dispatch 2024-02-13T03:29:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:36.979947+0000 mon.c (mon.2) 417 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:36.980308+0000 mon.a (mon.0) 1616 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:36.981690+0000 mon.c (mon.2) 418 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:36.982035+0000 mon.a (mon.0) 1617 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:36.982601+0000 mon.c (mon.2) 419 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi161-61719-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:36.982957+0000 mon.a (mon.0) 1618 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi161-61719-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: cluster 2024-02-13T03:29:37.283678+0000 mgr.y (mgr.24473) 352 : cluster [DBG] pgmap v407: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: cluster 2024-02-13T03:29:37.522245+0000 mon.a (mon.0) 1619 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:37.530167+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-46"}]': finished 2024-02-13T03:29:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:37.530334+0000 mon.a (mon.0) 1621 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-29", "mode": "writeback"}]': finished 2024-02-13T03:29:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:37.530496+0000 mon.a (mon.0) 1622 : audit [INF] from='client.48236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi161-61719-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: cluster 2024-02-13T03:29:37.530603+0000 mon.a (mon.0) 1623 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-13T03:29:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:37.534471+0000 mon.c (mon.2) 420 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi161-61719-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:37.541469+0000 mon.a (mon.0) 1624 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi161-61719-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:37.641417+0000 mon.a (mon.0) 1625 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:38.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:37 smithi183 bash[20824]: audit 2024-02-13T03:29:37.643693+0000 mon.b (mon.1) 215 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:38.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:36.954800+0000 mon.a (mon.0) 1612 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-29"}]': finished 2024-02-13T03:29:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:36.955072+0000 mon.a (mon.0) 1613 : audit [INF] from='client.48218 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]': finished 2024-02-13T03:29:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: cluster 2024-02-13T03:29:36.955283+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-13T03:29:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:36.957825+0000 mon.a (mon.0) 1615 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-29", "mode": "writeback"}]: dispatch 2024-02-13T03:29:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:36.959374+0000 mon.b (mon.1) 214 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-29", "mode": "writeback"}]: dispatch 2024-02-13T03:29:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:36.979947+0000 mon.c (mon.2) 417 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:36.980308+0000 mon.a (mon.0) 1616 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:36.981690+0000 mon.c (mon.2) 418 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:36.982035+0000 mon.a (mon.0) 1617 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:36.982601+0000 mon.c (mon.2) 419 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi161-61719-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:36.982957+0000 mon.a (mon.0) 1618 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi161-61719-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: cluster 2024-02-13T03:29:37.283678+0000 mgr.y (mgr.24473) 352 : cluster [DBG] pgmap v407: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:38.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: cluster 2024-02-13T03:29:37.522245+0000 mon.a (mon.0) 1619 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:38.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:37.530167+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-46"}]': finished 2024-02-13T03:29:38.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:37.530334+0000 mon.a (mon.0) 1621 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-29", "mode": "writeback"}]': finished 2024-02-13T03:29:38.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:36.954800+0000 mon.a (mon.0) 1612 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-29"}]': finished 2024-02-13T03:29:38.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:36.955072+0000 mon.a (mon.0) 1613 : audit [INF] from='client.48218 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi161-61719-36"}]': finished 2024-02-13T03:29:38.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: cluster 2024-02-13T03:29:36.955283+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-13T03:29:38.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:36.957825+0000 mon.a (mon.0) 1615 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-29", "mode": "writeback"}]: dispatch 2024-02-13T03:29:38.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:36.959374+0000 mon.b (mon.1) 214 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-29", "mode": "writeback"}]: dispatch 2024-02-13T03:29:38.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:36.979947+0000 mon.c (mon.2) 417 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:36.980308+0000 mon.a (mon.0) 1616 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:36.981690+0000 mon.c (mon.2) 418 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:36.982035+0000 mon.a (mon.0) 1617 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:36.982601+0000 mon.c (mon.2) 419 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi161-61719-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:38.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:36.982957+0000 mon.a (mon.0) 1618 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi161-61719-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:38.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: cluster 2024-02-13T03:29:37.283678+0000 mgr.y (mgr.24473) 352 : cluster [DBG] pgmap v407: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:38.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: cluster 2024-02-13T03:29:37.522245+0000 mon.a (mon.0) 1619 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:38.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:37.530167+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi161-61726-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi161-61726-46"}]': finished 2024-02-13T03:29:38.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:37.530334+0000 mon.a (mon.0) 1621 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-29", "mode": "writeback"}]': finished 2024-02-13T03:29:38.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:37.530496+0000 mon.a (mon.0) 1622 : audit [INF] from='client.48236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi161-61719-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:38.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: cluster 2024-02-13T03:29:37.530603+0000 mon.a (mon.0) 1623 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-13T03:29:38.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:37.534471+0000 mon.c (mon.2) 420 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi161-61719-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:37.541469+0000 mon.a (mon.0) 1624 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi161-61719-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:37.641417+0000 mon.a (mon.0) 1625 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:38.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[27879]: audit 2024-02-13T03:29:37.643693+0000 mon.b (mon.1) 215 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:38.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:37.530496+0000 mon.a (mon.0) 1622 : audit [INF] from='client.48236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi161-61719-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:38.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: cluster 2024-02-13T03:29:37.530603+0000 mon.a (mon.0) 1623 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-13T03:29:38.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:37.534471+0000 mon.c (mon.2) 420 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi161-61719-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:37.541469+0000 mon.a (mon.0) 1624 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi161-61719-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:38.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:37.641417+0000 mon.a (mon.0) 1625 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:38.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:37 smithi161 bash[19769]: audit 2024-02-13T03:29:37.643693+0000 mon.b (mon.1) 215 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:39.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:39 smithi183 bash[20824]: audit 2024-02-13T03:29:38.961351+0000 mon.a (mon.0) 1626 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:39.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:39 smithi183 bash[20824]: cluster 2024-02-13T03:29:38.961563+0000 mon.a (mon.0) 1627 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-13T03:29:39.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:39 smithi183 bash[20824]: audit 2024-02-13T03:29:38.963356+0000 mon.a (mon.0) 1628 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29"}]: dispatch 2024-02-13T03:29:39.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:39 smithi183 bash[20824]: audit 2024-02-13T03:29:38.965977+0000 mon.b (mon.1) 216 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29"}]: dispatch 2024-02-13T03:29:39.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:39 smithi183 bash[20824]: cluster 2024-02-13T03:29:39.284537+0000 mgr.y (mgr.24473) 353 : cluster [DBG] pgmap v410: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:39.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:29:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:29:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:29:39.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:39 smithi161 bash[27879]: audit 2024-02-13T03:29:38.961351+0000 mon.a (mon.0) 1626 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:39.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:39 smithi161 bash[27879]: cluster 2024-02-13T03:29:38.961563+0000 mon.a (mon.0) 1627 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-13T03:29:39.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:39 smithi161 bash[27879]: audit 2024-02-13T03:29:38.963356+0000 mon.a (mon.0) 1628 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29"}]: dispatch 2024-02-13T03:29:39.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:39 smithi161 bash[27879]: audit 2024-02-13T03:29:38.965977+0000 mon.b (mon.1) 216 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29"}]: dispatch 2024-02-13T03:29:39.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:39 smithi161 bash[27879]: cluster 2024-02-13T03:29:39.284537+0000 mgr.y (mgr.24473) 353 : cluster [DBG] pgmap v410: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:39.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:39 smithi161 bash[19769]: audit 2024-02-13T03:29:38.961351+0000 mon.a (mon.0) 1626 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:39.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:39 smithi161 bash[19769]: cluster 2024-02-13T03:29:38.961563+0000 mon.a (mon.0) 1627 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-13T03:29:39.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:39 smithi161 bash[19769]: audit 2024-02-13T03:29:38.963356+0000 mon.a (mon.0) 1628 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29"}]: dispatch 2024-02-13T03:29:39.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:39 smithi161 bash[19769]: audit 2024-02-13T03:29:38.965977+0000 mon.b (mon.1) 216 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29"}]: dispatch 2024-02-13T03:29:39.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:39 smithi161 bash[19769]: cluster 2024-02-13T03:29:39.284537+0000 mgr.y (mgr.24473) 353 : cluster [DBG] pgmap v410: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:40.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:40 smithi161 bash[27879]: cluster 2024-02-13T03:29:39.529963+0000 mon.a (mon.0) 1629 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:40.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:40 smithi161 bash[27879]: audit 2024-02-13T03:29:39.549529+0000 mon.a (mon.0) 1630 : audit [INF] from='client.48236 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi161-61719-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi161-61719-37"}]': finished 2024-02-13T03:29:40.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:40 smithi161 bash[27879]: audit 2024-02-13T03:29:39.549750+0000 mon.a (mon.0) 1631 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29"}]': finished 2024-02-13T03:29:40.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:40 smithi161 bash[27879]: cluster 2024-02-13T03:29:39.549857+0000 mon.a (mon.0) 1632 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-13T03:29:40.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:40 smithi161 bash[27879]: audit 2024-02-13T03:29:39.553447+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:40.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:40 smithi161 bash[19769]: cluster 2024-02-13T03:29:39.529963+0000 mon.a (mon.0) 1629 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:40.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:40 smithi161 bash[19769]: audit 2024-02-13T03:29:39.549529+0000 mon.a (mon.0) 1630 : audit [INF] from='client.48236 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi161-61719-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi161-61719-37"}]': finished 2024-02-13T03:29:40.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:40 smithi161 bash[19769]: audit 2024-02-13T03:29:39.549750+0000 mon.a (mon.0) 1631 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29"}]': finished 2024-02-13T03:29:40.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:40 smithi161 bash[19769]: cluster 2024-02-13T03:29:39.549857+0000 mon.a (mon.0) 1632 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-13T03:29:40.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:40 smithi161 bash[19769]: audit 2024-02-13T03:29:39.553447+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:40.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:40 smithi183 bash[20824]: cluster 2024-02-13T03:29:39.529963+0000 mon.a (mon.0) 1629 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:40.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:40 smithi183 bash[20824]: audit 2024-02-13T03:29:39.549529+0000 mon.a (mon.0) 1630 : audit [INF] from='client.48236 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi161-61719-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi161-61719-37"}]': finished 2024-02-13T03:29:40.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:40 smithi183 bash[20824]: audit 2024-02-13T03:29:39.549750+0000 mon.a (mon.0) 1631 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-29"}]': finished 2024-02-13T03:29:40.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:40 smithi183 bash[20824]: cluster 2024-02-13T03:29:39.549857+0000 mon.a (mon.0) 1632 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-13T03:29:40.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:40 smithi183 bash[20824]: audit 2024-02-13T03:29:39.553447+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:41.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:29:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:29:41] "GET /metrics HTTP/1.1" 200 185854 "" "Prometheus/2.33.4" 2024-02-13T03:29:41.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:41 smithi161 bash[27879]: audit 2024-02-13T03:29:40.547998+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-46"}]': finished 2024-02-13T03:29:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:41 smithi161 bash[27879]: cluster 2024-02-13T03:29:40.548169+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-13T03:29:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:41 smithi161 bash[27879]: audit 2024-02-13T03:29:40.549510+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:41 smithi161 bash[27879]: cluster 2024-02-13T03:29:41.286655+0000 mgr.y (mgr.24473) 354 : cluster [DBG] pgmap v413: 204 pgs: 4 creating+activating, 200 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:41.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:41 smithi161 bash[19769]: audit 2024-02-13T03:29:40.547998+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-46"}]': finished 2024-02-13T03:29:41.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:41 smithi161 bash[19769]: cluster 2024-02-13T03:29:40.548169+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-13T03:29:41.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:41 smithi161 bash[19769]: audit 2024-02-13T03:29:40.549510+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:41.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:41 smithi161 bash[19769]: cluster 2024-02-13T03:29:41.286655+0000 mgr.y (mgr.24473) 354 : cluster [DBG] pgmap v413: 204 pgs: 4 creating+activating, 200 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:41.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:41 smithi183 bash[20824]: audit 2024-02-13T03:29:40.547998+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi161-61726-46"}]': finished 2024-02-13T03:29:41.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:41 smithi183 bash[20824]: cluster 2024-02-13T03:29:40.548169+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-13T03:29:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:41 smithi183 bash[20824]: audit 2024-02-13T03:29:40.549510+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-46"}]: dispatch 2024-02-13T03:29:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:41 smithi183 bash[20824]: cluster 2024-02-13T03:29:41.286655+0000 mgr.y (mgr.24473) 354 : cluster [DBG] pgmap v413: 204 pgs: 4 creating+activating, 200 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[27879]: audit 2024-02-13T03:29:41.568575+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-46"}]': finished 2024-02-13T03:29:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[27879]: cluster 2024-02-13T03:29:41.568790+0000 mon.a (mon.0) 1638 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-13T03:29:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[27879]: audit 2024-02-13T03:29:41.573757+0000 mon.c (mon.2) 421 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[27879]: audit 2024-02-13T03:29:41.576247+0000 mon.b (mon.1) 217 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[27879]: audit 2024-02-13T03:29:41.577869+0000 mon.a (mon.0) 1639 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[27879]: audit 2024-02-13T03:29:41.578348+0000 mon.a (mon.0) 1640 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[27879]: audit 2024-02-13T03:29:41.598392+0000 mon.a (mon.0) 1641 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[27879]: audit 2024-02-13T03:29:41.599742+0000 mon.a (mon.0) 1642 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[27879]: audit 2024-02-13T03:29:41.600101+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi161-61726-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:42.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[19769]: audit 2024-02-13T03:29:41.568575+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-46"}]': finished 2024-02-13T03:29:42.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[19769]: cluster 2024-02-13T03:29:41.568790+0000 mon.a (mon.0) 1638 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-13T03:29:42.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[19769]: audit 2024-02-13T03:29:41.573757+0000 mon.c (mon.2) 421 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:42.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[19769]: audit 2024-02-13T03:29:41.576247+0000 mon.b (mon.1) 217 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:42.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[19769]: audit 2024-02-13T03:29:41.577869+0000 mon.a (mon.0) 1639 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:42.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[19769]: audit 2024-02-13T03:29:41.578348+0000 mon.a (mon.0) 1640 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:42.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[19769]: audit 2024-02-13T03:29:41.598392+0000 mon.a (mon.0) 1641 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:42.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[19769]: audit 2024-02-13T03:29:41.599742+0000 mon.a (mon.0) 1642 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:42.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:42 smithi161 bash[19769]: audit 2024-02-13T03:29:41.600101+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi161-61726-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:42.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:42 smithi183 bash[20824]: audit 2024-02-13T03:29:41.568575+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? v1:172.21.15.161:0/820951927' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi161-61726-46"}]': finished 2024-02-13T03:29:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:42 smithi183 bash[20824]: cluster 2024-02-13T03:29:41.568790+0000 mon.a (mon.0) 1638 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-13T03:29:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:42 smithi183 bash[20824]: audit 2024-02-13T03:29:41.573757+0000 mon.c (mon.2) 421 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:42 smithi183 bash[20824]: audit 2024-02-13T03:29:41.576247+0000 mon.b (mon.1) 217 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:42 smithi183 bash[20824]: audit 2024-02-13T03:29:41.577869+0000 mon.a (mon.0) 1639 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:42 smithi183 bash[20824]: audit 2024-02-13T03:29:41.578348+0000 mon.a (mon.0) 1640 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:42 smithi183 bash[20824]: audit 2024-02-13T03:29:41.598392+0000 mon.a (mon.0) 1641 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:42 smithi183 bash[20824]: audit 2024-02-13T03:29:41.599742+0000 mon.a (mon.0) 1642 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:42 smithi183 bash[20824]: audit 2024-02-13T03:29:41.600101+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi161-61726-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:44.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[19769]: audit 2024-02-13T03:29:42.569149+0000 mon.a (mon.0) 1644 : audit [INF] from='client.48236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]': finished 2024-02-13T03:29:44.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[19769]: audit 2024-02-13T03:29:42.569325+0000 mon.a (mon.0) 1645 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:44.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[19769]: audit 2024-02-13T03:29:42.570058+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi161-61726-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:44.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[19769]: cluster 2024-02-13T03:29:42.570174+0000 mon.a (mon.0) 1647 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-13T03:29:44.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[19769]: audit 2024-02-13T03:29:42.571444+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi161-61726-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:44.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[19769]: audit 2024-02-13T03:29:42.574627+0000 mon.c (mon.2) 422 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:44.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[19769]: audit 2024-02-13T03:29:42.586202+0000 mon.a (mon.0) 1649 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:44.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[19769]: audit 2024-02-13T03:29:42.586799+0000 mon.a (mon.0) 1650 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:44.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[19769]: audit 2024-02-13T03:29:42.588932+0000 mon.b (mon.1) 218 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:44.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[19769]: cluster 2024-02-13T03:29:43.287588+0000 mgr.y (mgr.24473) 355 : cluster [DBG] pgmap v416: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[27879]: audit 2024-02-13T03:29:42.569149+0000 mon.a (mon.0) 1644 : audit [INF] from='client.48236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]': finished 2024-02-13T03:29:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[27879]: audit 2024-02-13T03:29:42.569325+0000 mon.a (mon.0) 1645 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[27879]: audit 2024-02-13T03:29:42.570058+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi161-61726-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[27879]: cluster 2024-02-13T03:29:42.570174+0000 mon.a (mon.0) 1647 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-13T03:29:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[27879]: audit 2024-02-13T03:29:42.571444+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi161-61726-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[27879]: audit 2024-02-13T03:29:42.574627+0000 mon.c (mon.2) 422 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[27879]: audit 2024-02-13T03:29:42.586202+0000 mon.a (mon.0) 1649 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[27879]: audit 2024-02-13T03:29:42.586799+0000 mon.a (mon.0) 1650 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[27879]: audit 2024-02-13T03:29:42.588932+0000 mon.b (mon.1) 218 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:44.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:44 smithi161 bash[27879]: cluster 2024-02-13T03:29:43.287588+0000 mgr.y (mgr.24473) 355 : cluster [DBG] pgmap v416: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:44.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:44 smithi183 bash[20824]: audit 2024-02-13T03:29:42.569149+0000 mon.a (mon.0) 1644 : audit [INF] from='client.48236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi161-61719-37"}]': finished 2024-02-13T03:29:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:44 smithi183 bash[20824]: audit 2024-02-13T03:29:42.569325+0000 mon.a (mon.0) 1645 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:44 smithi183 bash[20824]: audit 2024-02-13T03:29:42.570058+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi161-61726-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:44 smithi183 bash[20824]: cluster 2024-02-13T03:29:42.570174+0000 mon.a (mon.0) 1647 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-13T03:29:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:44 smithi183 bash[20824]: audit 2024-02-13T03:29:42.571444+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi161-61726-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:44 smithi183 bash[20824]: audit 2024-02-13T03:29:42.574627+0000 mon.c (mon.2) 422 : audit [INF] from='client.? v1:172.21.15.161:0/2143715694' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:44 smithi183 bash[20824]: audit 2024-02-13T03:29:42.586202+0000 mon.a (mon.0) 1649 : audit [INF] from='client.48236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]: dispatch 2024-02-13T03:29:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:44 smithi183 bash[20824]: audit 2024-02-13T03:29:42.586799+0000 mon.a (mon.0) 1650 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:44 smithi183 bash[20824]: audit 2024-02-13T03:29:42.588932+0000 mon.b (mon.1) 218 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:44 smithi183 bash[20824]: cluster 2024-02-13T03:29:43.287588+0000 mgr.y (mgr.24473) 355 : cluster [DBG] pgmap v416: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.001683+0000 mon.a (mon.0) 1651 : audit [INF] from='client.48236 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]': finished 2024-02-13T03:29:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.001895+0000 mon.a (mon.0) 1652 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: cluster 2024-02-13T03:29:44.002162+0000 mon.a (mon.0) 1653 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-13T03:29:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.003800+0000 mon.a (mon.0) 1654 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-31"}]: dispatch 2024-02-13T03:29:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.006388+0000 mon.b (mon.1) 219 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-31"}]: dispatch 2024-02-13T03:29:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.020402+0000 mon.c (mon.2) 423 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.020830+0000 mon.a (mon.0) 1655 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.021876+0000 mon.c (mon.2) 424 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.022126+0000 mon.a (mon.0) 1656 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.022605+0000 mon.c (mon.2) 425 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi161-61719-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:45.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.022877+0000 mon.a (mon.0) 1657 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi161-61719-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:45.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.576035+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi161-61726-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi161-61726-47"}]': finished 2024-02-13T03:29:45.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.576333+0000 mon.a (mon.0) 1659 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-31"}]': finished 2024-02-13T03:29:45.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.576465+0000 mon.a (mon.0) 1660 : audit [INF] from='client.48248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi161-61719-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:45.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.001683+0000 mon.a (mon.0) 1651 : audit [INF] from='client.48236 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]': finished 2024-02-13T03:29:45.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.001895+0000 mon.a (mon.0) 1652 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:45.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: cluster 2024-02-13T03:29:44.002162+0000 mon.a (mon.0) 1653 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-13T03:29:45.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.003800+0000 mon.a (mon.0) 1654 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-31"}]: dispatch 2024-02-13T03:29:45.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.006388+0000 mon.b (mon.1) 219 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-31"}]: dispatch 2024-02-13T03:29:45.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.020402+0000 mon.c (mon.2) 423 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.020830+0000 mon.a (mon.0) 1655 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.021876+0000 mon.c (mon.2) 424 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.022126+0000 mon.a (mon.0) 1656 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.022605+0000 mon.c (mon.2) 425 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi161-61719-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:45.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.022877+0000 mon.a (mon.0) 1657 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi161-61719-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:45.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.576035+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi161-61726-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi161-61726-47"}]': finished 2024-02-13T03:29:45.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.576333+0000 mon.a (mon.0) 1659 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-31"}]': finished 2024-02-13T03:29:45.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.576465+0000 mon.a (mon.0) 1660 : audit [INF] from='client.48248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi161-61719-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:45.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: cluster 2024-02-13T03:29:44.576607+0000 mon.a (mon.0) 1661 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-13T03:29:45.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.580010+0000 mon.a (mon.0) 1662 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-31", "mode": "writeback"}]: dispatch 2024-02-13T03:29:45.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.580589+0000 mon.c (mon.2) 426 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi161-61719-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.581552+0000 mon.a (mon.0) 1663 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi161-61719-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[19769]: audit 2024-02-13T03:29:44.582427+0000 mon.b (mon.1) 220 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-31", "mode": "writeback"}]: dispatch 2024-02-13T03:29:45.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: cluster 2024-02-13T03:29:44.576607+0000 mon.a (mon.0) 1661 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-13T03:29:45.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.580010+0000 mon.a (mon.0) 1662 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-31", "mode": "writeback"}]: dispatch 2024-02-13T03:29:45.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.580589+0000 mon.c (mon.2) 426 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi161-61719-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.581552+0000 mon.a (mon.0) 1663 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi161-61719-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:45 smithi161 bash[27879]: audit 2024-02-13T03:29:44.582427+0000 mon.b (mon.1) 220 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-31", "mode": "writeback"}]: dispatch 2024-02-13T03:29:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.001683+0000 mon.a (mon.0) 1651 : audit [INF] from='client.48236 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi161-61719-37"}]': finished 2024-02-13T03:29:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.001895+0000 mon.a (mon.0) 1652 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: cluster 2024-02-13T03:29:44.002162+0000 mon.a (mon.0) 1653 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-13T03:29:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.003800+0000 mon.a (mon.0) 1654 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-31"}]: dispatch 2024-02-13T03:29:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.006388+0000 mon.b (mon.1) 219 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-31"}]: dispatch 2024-02-13T03:29:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.020402+0000 mon.c (mon.2) 423 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.020830+0000 mon.a (mon.0) 1655 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.021876+0000 mon.c (mon.2) 424 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.022126+0000 mon.a (mon.0) 1656 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.022605+0000 mon.c (mon.2) 425 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi161-61719-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:45.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.022877+0000 mon.a (mon.0) 1657 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi161-61719-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:45.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.576035+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi161-61726-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi161-61726-47"}]': finished 2024-02-13T03:29:45.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.576333+0000 mon.a (mon.0) 1659 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-31"}]': finished 2024-02-13T03:29:45.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.576465+0000 mon.a (mon.0) 1660 : audit [INF] from='client.48248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi161-61719-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:45.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: cluster 2024-02-13T03:29:44.576607+0000 mon.a (mon.0) 1661 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-13T03:29:45.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.580010+0000 mon.a (mon.0) 1662 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-31", "mode": "writeback"}]: dispatch 2024-02-13T03:29:45.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.580589+0000 mon.c (mon.2) 426 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi161-61719-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.581552+0000 mon.a (mon.0) 1663 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi161-61719-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:45.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[20824]: audit 2024-02-13T03:29:44.582427+0000 mon.b (mon.1) 220 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-31", "mode": "writeback"}]: dispatch 2024-02-13T03:29:46.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:29:45 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:29:46.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:46 smithi183 bash[20824]: cluster 2024-02-13T03:29:45.288417+0000 mgr.y (mgr.24473) 356 : cluster [DBG] pgmap v419: 236 pgs: 40 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:46.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:46 smithi183 bash[20824]: cluster 2024-02-13T03:29:45.568871+0000 mon.a (mon.0) 1664 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:46.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:46 smithi161 bash[27879]: cluster 2024-02-13T03:29:45.288417+0000 mgr.y (mgr.24473) 356 : cluster [DBG] pgmap v419: 236 pgs: 40 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:46.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:46 smithi161 bash[27879]: cluster 2024-02-13T03:29:45.568871+0000 mon.a (mon.0) 1664 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:46.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:46 smithi161 bash[19769]: cluster 2024-02-13T03:29:45.288417+0000 mgr.y (mgr.24473) 356 : cluster [DBG] pgmap v419: 236 pgs: 40 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:46.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:46 smithi161 bash[19769]: cluster 2024-02-13T03:29:45.568871+0000 mon.a (mon.0) 1664 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[27879]: audit 2024-02-13T03:29:45.579995+0000 mon.a (mon.0) 1665 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-31", "mode": "writeback"}]': finished 2024-02-13T03:29:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[27879]: audit 2024-02-13T03:29:45.950830+0000 mgr.y (mgr.24473) 357 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[27879]: audit 2024-02-13T03:29:45.955975+0000 mon.b (mon.1) 221 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[27879]: cluster 2024-02-13T03:29:46.012975+0000 mon.a (mon.0) 1666 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-02-13T03:29:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[27879]: audit 2024-02-13T03:29:46.022046+0000 mon.a (mon.0) 1667 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[27879]: audit 2024-02-13T03:29:46.581536+0000 mon.a (mon.0) 1668 : audit [INF] from='client.48248 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi161-61719-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi161-61719-38"}]': finished 2024-02-13T03:29:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[27879]: audit 2024-02-13T03:29:46.581888+0000 mon.a (mon.0) 1669 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[27879]: cluster 2024-02-13T03:29:46.582045+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-02-13T03:29:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[27879]: audit 2024-02-13T03:29:46.584205+0000 mon.a (mon.0) 1671 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31"}]: dispatch 2024-02-13T03:29:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[27879]: audit 2024-02-13T03:29:46.584480+0000 mon.a (mon.0) 1672 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:47.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[27879]: audit 2024-02-13T03:29:46.586115+0000 mon.b (mon.1) 222 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31"}]: dispatch 2024-02-13T03:29:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[19769]: audit 2024-02-13T03:29:45.579995+0000 mon.a (mon.0) 1665 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-31", "mode": "writeback"}]': finished 2024-02-13T03:29:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[19769]: audit 2024-02-13T03:29:45.950830+0000 mgr.y (mgr.24473) 357 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[19769]: audit 2024-02-13T03:29:45.955975+0000 mon.b (mon.1) 221 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[19769]: cluster 2024-02-13T03:29:46.012975+0000 mon.a (mon.0) 1666 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-02-13T03:29:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[19769]: audit 2024-02-13T03:29:46.022046+0000 mon.a (mon.0) 1667 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[19769]: audit 2024-02-13T03:29:46.581536+0000 mon.a (mon.0) 1668 : audit [INF] from='client.48248 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi161-61719-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi161-61719-38"}]': finished 2024-02-13T03:29:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[19769]: audit 2024-02-13T03:29:46.581888+0000 mon.a (mon.0) 1669 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[19769]: cluster 2024-02-13T03:29:46.582045+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-02-13T03:29:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[19769]: audit 2024-02-13T03:29:46.584205+0000 mon.a (mon.0) 1671 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31"}]: dispatch 2024-02-13T03:29:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[19769]: audit 2024-02-13T03:29:46.584480+0000 mon.a (mon.0) 1672 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:47.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:47 smithi161 bash[19769]: audit 2024-02-13T03:29:46.586115+0000 mon.b (mon.1) 222 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31"}]: dispatch 2024-02-13T03:29:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:47 smithi183 bash[20824]: audit 2024-02-13T03:29:45.579995+0000 mon.a (mon.0) 1665 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-31", "mode": "writeback"}]': finished 2024-02-13T03:29:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:47 smithi183 bash[20824]: audit 2024-02-13T03:29:45.950830+0000 mgr.y (mgr.24473) 357 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:47 smithi183 bash[20824]: audit 2024-02-13T03:29:45.955975+0000 mon.b (mon.1) 221 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:47 smithi183 bash[20824]: cluster 2024-02-13T03:29:46.012975+0000 mon.a (mon.0) 1666 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-02-13T03:29:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:47 smithi183 bash[20824]: audit 2024-02-13T03:29:46.022046+0000 mon.a (mon.0) 1667 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:47 smithi183 bash[20824]: audit 2024-02-13T03:29:46.581536+0000 mon.a (mon.0) 1668 : audit [INF] from='client.48248 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi161-61719-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi161-61719-38"}]': finished 2024-02-13T03:29:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:47 smithi183 bash[20824]: audit 2024-02-13T03:29:46.581888+0000 mon.a (mon.0) 1669 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:47 smithi183 bash[20824]: cluster 2024-02-13T03:29:46.582045+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-02-13T03:29:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:47 smithi183 bash[20824]: audit 2024-02-13T03:29:46.584205+0000 mon.a (mon.0) 1671 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31"}]: dispatch 2024-02-13T03:29:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:47 smithi183 bash[20824]: audit 2024-02-13T03:29:46.584480+0000 mon.a (mon.0) 1672 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:47 smithi183 bash[20824]: audit 2024-02-13T03:29:46.586115+0000 mon.b (mon.1) 222 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31"}]: dispatch 2024-02-13T03:29:48.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:48 smithi161 bash[27879]: cluster 2024-02-13T03:29:47.289405+0000 mgr.y (mgr.24473) 358 : cluster [DBG] pgmap v422: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:48 smithi161 bash[27879]: cluster 2024-02-13T03:29:47.574596+0000 mon.a (mon.0) 1673 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:48 smithi161 bash[27879]: audit 2024-02-13T03:29:47.588499+0000 mon.a (mon.0) 1674 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31"}]': finished 2024-02-13T03:29:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:48 smithi161 bash[27879]: audit 2024-02-13T03:29:47.588780+0000 mon.a (mon.0) 1675 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi161-61726-47"}]': finished 2024-02-13T03:29:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:48 smithi161 bash[27879]: cluster 2024-02-13T03:29:47.589068+0000 mon.a (mon.0) 1676 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-02-13T03:29:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:48 smithi161 bash[27879]: audit 2024-02-13T03:29:47.590767+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:48 smithi161 bash[19769]: cluster 2024-02-13T03:29:47.289405+0000 mgr.y (mgr.24473) 358 : cluster [DBG] pgmap v422: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:48 smithi161 bash[19769]: cluster 2024-02-13T03:29:47.574596+0000 mon.a (mon.0) 1673 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:48 smithi161 bash[19769]: audit 2024-02-13T03:29:47.588499+0000 mon.a (mon.0) 1674 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31"}]': finished 2024-02-13T03:29:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:48 smithi161 bash[19769]: audit 2024-02-13T03:29:47.588780+0000 mon.a (mon.0) 1675 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi161-61726-47"}]': finished 2024-02-13T03:29:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:48 smithi161 bash[19769]: cluster 2024-02-13T03:29:47.589068+0000 mon.a (mon.0) 1676 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-02-13T03:29:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:48 smithi161 bash[19769]: audit 2024-02-13T03:29:47.590767+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:48 smithi183 bash[20824]: cluster 2024-02-13T03:29:47.289405+0000 mgr.y (mgr.24473) 358 : cluster [DBG] pgmap v422: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:48 smithi183 bash[20824]: cluster 2024-02-13T03:29:47.574596+0000 mon.a (mon.0) 1673 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:29:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:48 smithi183 bash[20824]: audit 2024-02-13T03:29:47.588499+0000 mon.a (mon.0) 1674 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-31"}]': finished 2024-02-13T03:29:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:48 smithi183 bash[20824]: audit 2024-02-13T03:29:47.588780+0000 mon.a (mon.0) 1675 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi161-61726-47"}]': finished 2024-02-13T03:29:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:48 smithi183 bash[20824]: cluster 2024-02-13T03:29:47.589068+0000 mon.a (mon.0) 1676 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-02-13T03:29:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:48 smithi183 bash[20824]: audit 2024-02-13T03:29:47.590767+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi161-61726-47"}]: dispatch 2024-02-13T03:29:49.586 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:29:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:29:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:29:49.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[27879]: audit 2024-02-13T03:29:48.587402+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi161-61726-47"}]': finished 2024-02-13T03:29:49.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[27879]: cluster 2024-02-13T03:29:48.587725+0000 mon.a (mon.0) 1679 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-02-13T03:29:49.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[27879]: audit 2024-02-13T03:29:48.589404+0000 mon.c (mon.2) 427 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:49.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[27879]: audit 2024-02-13T03:29:49.024736+0000 mon.a (mon.0) 1680 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:49.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[27879]: audit 2024-02-13T03:29:49.052637+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:49.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[27879]: audit 2024-02-13T03:29:49.054355+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:49.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[27879]: audit 2024-02-13T03:29:49.054882+0000 mon.a (mon.0) 1683 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi161-61726-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:49.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[27879]: cluster 2024-02-13T03:29:49.290363+0000 mgr.y (mgr.24473) 359 : cluster [DBG] pgmap v425: 196 pgs: 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:49.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[19769]: audit 2024-02-13T03:29:48.587402+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi161-61726-47"}]': finished 2024-02-13T03:29:49.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[19769]: cluster 2024-02-13T03:29:48.587725+0000 mon.a (mon.0) 1679 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-02-13T03:29:49.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[19769]: audit 2024-02-13T03:29:48.589404+0000 mon.c (mon.2) 427 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:49.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[19769]: audit 2024-02-13T03:29:49.024736+0000 mon.a (mon.0) 1680 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:49.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[19769]: audit 2024-02-13T03:29:49.052637+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:49.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[19769]: audit 2024-02-13T03:29:49.054355+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:49.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[19769]: audit 2024-02-13T03:29:49.054882+0000 mon.a (mon.0) 1683 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi161-61726-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:49.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:49 smithi161 bash[19769]: cluster 2024-02-13T03:29:49.290363+0000 mgr.y (mgr.24473) 359 : cluster [DBG] pgmap v425: 196 pgs: 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:49.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:49 smithi183 bash[20824]: audit 2024-02-13T03:29:48.587402+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? v1:172.21.15.161:0/3248024899' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi161-61726-47"}]': finished 2024-02-13T03:29:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:49 smithi183 bash[20824]: cluster 2024-02-13T03:29:48.587725+0000 mon.a (mon.0) 1679 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-02-13T03:29:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:49 smithi183 bash[20824]: audit 2024-02-13T03:29:48.589404+0000 mon.c (mon.2) 427 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:49 smithi183 bash[20824]: audit 2024-02-13T03:29:49.024736+0000 mon.a (mon.0) 1680 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:49 smithi183 bash[20824]: audit 2024-02-13T03:29:49.052637+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:49 smithi183 bash[20824]: audit 2024-02-13T03:29:49.054355+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:49 smithi183 bash[20824]: audit 2024-02-13T03:29:49.054882+0000 mon.a (mon.0) 1683 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi161-61726-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:49 smithi183 bash[20824]: cluster 2024-02-13T03:29:49.290363+0000 mgr.y (mgr.24473) 359 : cluster [DBG] pgmap v425: 196 pgs: 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:29:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[19769]: audit 2024-02-13T03:29:49.602871+0000 mon.a (mon.0) 1684 : audit [INF] from='client.48248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]': finished 2024-02-13T03:29:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[19769]: audit 2024-02-13T03:29:49.603080+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi161-61726-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[19769]: cluster 2024-02-13T03:29:49.603238+0000 mon.a (mon.0) 1686 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-02-13T03:29:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[19769]: audit 2024-02-13T03:29:49.606559+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi161-61726-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[19769]: audit 2024-02-13T03:29:49.607546+0000 mon.a (mon.0) 1688 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[19769]: audit 2024-02-13T03:29:49.609318+0000 mon.b (mon.1) 223 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[19769]: audit 2024-02-13T03:29:49.612104+0000 mon.c (mon.2) 428 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[19769]: audit 2024-02-13T03:29:49.612801+0000 mon.a (mon.0) 1689 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[27879]: audit 2024-02-13T03:29:49.602871+0000 mon.a (mon.0) 1684 : audit [INF] from='client.48248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]': finished 2024-02-13T03:29:51.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[27879]: audit 2024-02-13T03:29:49.603080+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi161-61726-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:51.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[27879]: cluster 2024-02-13T03:29:49.603238+0000 mon.a (mon.0) 1686 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-02-13T03:29:51.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[27879]: audit 2024-02-13T03:29:49.606559+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi161-61726-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:51.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[27879]: audit 2024-02-13T03:29:49.607546+0000 mon.a (mon.0) 1688 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:51.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[27879]: audit 2024-02-13T03:29:49.609318+0000 mon.b (mon.1) 223 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:51.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[27879]: audit 2024-02-13T03:29:49.612104+0000 mon.c (mon.2) 428 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:51.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[27879]: audit 2024-02-13T03:29:49.612801+0000 mon.a (mon.0) 1689 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:51.428 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:29:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:29:51] "GET /metrics HTTP/1.1" 200 185979 "" "Prometheus/2.33.4" 2024-02-13T03:29:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:51 smithi183 bash[20824]: audit 2024-02-13T03:29:49.602871+0000 mon.a (mon.0) 1684 : audit [INF] from='client.48248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi161-61719-38"}]': finished 2024-02-13T03:29:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:51 smithi183 bash[20824]: audit 2024-02-13T03:29:49.603080+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi161-61726-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:51 smithi183 bash[20824]: cluster 2024-02-13T03:29:49.603238+0000 mon.a (mon.0) 1686 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-02-13T03:29:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:51 smithi183 bash[20824]: audit 2024-02-13T03:29:49.606559+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi161-61726-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:51 smithi183 bash[20824]: audit 2024-02-13T03:29:49.607546+0000 mon.a (mon.0) 1688 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:51 smithi183 bash[20824]: audit 2024-02-13T03:29:49.609318+0000 mon.b (mon.1) 223 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:29:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:51 smithi183 bash[20824]: audit 2024-02-13T03:29:49.612104+0000 mon.c (mon.2) 428 : audit [INF] from='client.? v1:172.21.15.161:0/399367504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:51 smithi183 bash[20824]: audit 2024-02-13T03:29:49.612801+0000 mon.a (mon.0) 1689 : audit [INF] from='client.48248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]: dispatch 2024-02-13T03:29:52.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.017291+0000 mon.a (mon.0) 1690 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:52.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.017799+0000 mon.a (mon.0) 1691 : audit [INF] from='client.48248 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]': finished 2024-02-13T03:29:52.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: cluster 2024-02-13T03:29:51.018088+0000 mon.a (mon.0) 1692 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-02-13T03:29:52.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.022643+0000 mon.b (mon.1) 224 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:52.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.030188+0000 mon.a (mon.0) 1693 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:52.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.038024+0000 mon.c (mon.2) 429 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.038407+0000 mon.a (mon.0) 1694 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.039307+0000 mon.c (mon.2) 430 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.039562+0000 mon.a (mon.0) 1695 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.039989+0000 mon.c (mon.2) 431 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi161-61719-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:52.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.040257+0000 mon.a (mon.0) 1696 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi161-61719-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:52.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: cluster 2024-02-13T03:29:51.292595+0000 mgr.y (mgr.24473) 360 : cluster [DBG] pgmap v428: 228 pgs: 24 creating+peering, 8 creating+activating, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:52.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.611731+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi161-61726-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi161-61726-48"}]': finished 2024-02-13T03:29:52.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.017291+0000 mon.a (mon.0) 1690 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:52.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.017799+0000 mon.a (mon.0) 1691 : audit [INF] from='client.48248 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]': finished 2024-02-13T03:29:52.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: cluster 2024-02-13T03:29:51.018088+0000 mon.a (mon.0) 1692 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-02-13T03:29:52.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.022643+0000 mon.b (mon.1) 224 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:52.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.030188+0000 mon.a (mon.0) 1693 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:52.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.038024+0000 mon.c (mon.2) 429 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.038407+0000 mon.a (mon.0) 1694 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.039307+0000 mon.c (mon.2) 430 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.039562+0000 mon.a (mon.0) 1695 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.039989+0000 mon.c (mon.2) 431 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi161-61719-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:52.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.040257+0000 mon.a (mon.0) 1696 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi161-61719-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:52.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: cluster 2024-02-13T03:29:51.292595+0000 mgr.y (mgr.24473) 360 : cluster [DBG] pgmap v428: 228 pgs: 24 creating+peering, 8 creating+activating, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:52.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.611731+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi161-61726-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi161-61726-48"}]': finished 2024-02-13T03:29:52.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.611880+0000 mon.a (mon.0) 1698 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:52.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.611998+0000 mon.a (mon.0) 1699 : audit [INF] from='client.48260 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi161-61719-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:52.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: cluster 2024-02-13T03:29:51.612095+0000 mon.a (mon.0) 1700 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-02-13T03:29:52.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.613165+0000 mon.c (mon.2) 432 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi161-61719-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.614494+0000 mon.a (mon.0) 1701 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:29:52.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.614778+0000 mon.a (mon.0) 1702 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi161-61719-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[27879]: audit 2024-02-13T03:29:51.616078+0000 mon.b (mon.1) 225 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:29:52.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.611880+0000 mon.a (mon.0) 1698 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:52.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.611998+0000 mon.a (mon.0) 1699 : audit [INF] from='client.48260 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi161-61719-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:52.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: cluster 2024-02-13T03:29:51.612095+0000 mon.a (mon.0) 1700 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-02-13T03:29:52.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.613165+0000 mon.c (mon.2) 432 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi161-61719-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.614494+0000 mon.a (mon.0) 1701 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:29:52.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.614778+0000 mon.a (mon.0) 1702 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi161-61719-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.431 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:52 smithi161 bash[19769]: audit 2024-02-13T03:29:51.616078+0000 mon.b (mon.1) 225 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:29:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.017291+0000 mon.a (mon.0) 1690 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:29:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.017799+0000 mon.a (mon.0) 1691 : audit [INF] from='client.48248 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi161-61719-38"}]': finished 2024-02-13T03:29:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: cluster 2024-02-13T03:29:51.018088+0000 mon.a (mon.0) 1692 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-02-13T03:29:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.022643+0000 mon.b (mon.1) 224 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.030188+0000 mon.a (mon.0) 1693 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:29:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.038024+0000 mon.c (mon.2) 429 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.038407+0000 mon.a (mon.0) 1694 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.039307+0000 mon.c (mon.2) 430 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.039562+0000 mon.a (mon.0) 1695 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.039989+0000 mon.c (mon.2) 431 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi161-61719-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.040257+0000 mon.a (mon.0) 1696 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi161-61719-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: cluster 2024-02-13T03:29:51.292595+0000 mgr.y (mgr.24473) 360 : cluster [DBG] pgmap v428: 228 pgs: 24 creating+peering, 8 creating+activating, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.611731+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi161-61726-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi161-61726-48"}]': finished 2024-02-13T03:29:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.611880+0000 mon.a (mon.0) 1698 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:29:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.611998+0000 mon.a (mon.0) 1699 : audit [INF] from='client.48260 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi161-61719-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: cluster 2024-02-13T03:29:51.612095+0000 mon.a (mon.0) 1700 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-02-13T03:29:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.613165+0000 mon.c (mon.2) 432 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi161-61719-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.614494+0000 mon.a (mon.0) 1701 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:29:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.614778+0000 mon.a (mon.0) 1702 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi161-61719-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:52.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:52 smithi183 bash[20824]: audit 2024-02-13T03:29:51.616078+0000 mon.b (mon.1) 225 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:29:53.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:53 smithi161 bash[27879]: audit 2024-02-13T03:29:52.615722+0000 mon.a (mon.0) 1703 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]': finished 2024-02-13T03:29:53.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:53 smithi161 bash[27879]: audit 2024-02-13T03:29:52.622609+0000 mon.b (mon.1) 226 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-33", "mode": "writeback"}]: dispatch 2024-02-13T03:29:53.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:53 smithi161 bash[27879]: cluster 2024-02-13T03:29:53.050094+0000 mon.a (mon.0) 1704 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-02-13T03:29:53.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:53 smithi161 bash[27879]: audit 2024-02-13T03:29:53.051553+0000 mon.a (mon.0) 1705 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-33", "mode": "writeback"}]: dispatch 2024-02-13T03:29:53.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:53 smithi161 bash[27879]: cluster 2024-02-13T03:29:53.293441+0000 mgr.y (mgr.24473) 361 : cluster [DBG] pgmap v431: 236 pgs: 8 unknown, 24 creating+peering, 8 creating+activating, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:53.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:53 smithi161 bash[19769]: audit 2024-02-13T03:29:52.615722+0000 mon.a (mon.0) 1703 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]': finished 2024-02-13T03:29:53.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:53 smithi161 bash[19769]: audit 2024-02-13T03:29:52.622609+0000 mon.b (mon.1) 226 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-33", "mode": "writeback"}]: dispatch 2024-02-13T03:29:53.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:53 smithi161 bash[19769]: cluster 2024-02-13T03:29:53.050094+0000 mon.a (mon.0) 1704 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-02-13T03:29:53.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:53 smithi161 bash[19769]: audit 2024-02-13T03:29:53.051553+0000 mon.a (mon.0) 1705 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-33", "mode": "writeback"}]: dispatch 2024-02-13T03:29:53.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:53 smithi161 bash[19769]: cluster 2024-02-13T03:29:53.293441+0000 mgr.y (mgr.24473) 361 : cluster [DBG] pgmap v431: 236 pgs: 8 unknown, 24 creating+peering, 8 creating+activating, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:53.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:53 smithi183 bash[20824]: audit 2024-02-13T03:29:52.615722+0000 mon.a (mon.0) 1703 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]': finished 2024-02-13T03:29:53.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:53 smithi183 bash[20824]: audit 2024-02-13T03:29:52.622609+0000 mon.b (mon.1) 226 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-33", "mode": "writeback"}]: dispatch 2024-02-13T03:29:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:53 smithi183 bash[20824]: cluster 2024-02-13T03:29:53.050094+0000 mon.a (mon.0) 1704 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-02-13T03:29:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:53 smithi183 bash[20824]: audit 2024-02-13T03:29:53.051553+0000 mon.a (mon.0) 1705 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-33", "mode": "writeback"}]: dispatch 2024-02-13T03:29:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:53 smithi183 bash[20824]: cluster 2024-02-13T03:29:53.293441+0000 mgr.y (mgr.24473) 361 : cluster [DBG] pgmap v431: 236 pgs: 8 unknown, 24 creating+peering, 8 creating+activating, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:29:54.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:54 smithi161 bash[27879]: cluster 2024-02-13T03:29:53.606519+0000 mon.a (mon.0) 1706 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:54 smithi161 bash[27879]: audit 2024-02-13T03:29:53.612833+0000 mon.a (mon.0) 1707 : audit [INF] from='client.48260 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi161-61719-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi161-61719-39"}]': finished 2024-02-13T03:29:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:54 smithi161 bash[27879]: audit 2024-02-13T03:29:53.612954+0000 mon.a (mon.0) 1708 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-33", "mode": "writeback"}]': finished 2024-02-13T03:29:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:54 smithi161 bash[27879]: cluster 2024-02-13T03:29:53.613078+0000 mon.a (mon.0) 1709 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-02-13T03:29:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:54 smithi161 bash[27879]: audit 2024-02-13T03:29:53.615355+0000 mon.a (mon.0) 1710 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:54.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:54 smithi161 bash[19769]: cluster 2024-02-13T03:29:53.606519+0000 mon.a (mon.0) 1706 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:54.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:54 smithi161 bash[19769]: audit 2024-02-13T03:29:53.612833+0000 mon.a (mon.0) 1707 : audit [INF] from='client.48260 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi161-61719-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi161-61719-39"}]': finished 2024-02-13T03:29:54.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:54 smithi161 bash[19769]: audit 2024-02-13T03:29:53.612954+0000 mon.a (mon.0) 1708 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-33", "mode": "writeback"}]': finished 2024-02-13T03:29:54.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:54 smithi161 bash[19769]: cluster 2024-02-13T03:29:53.613078+0000 mon.a (mon.0) 1709 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-02-13T03:29:54.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:54 smithi161 bash[19769]: audit 2024-02-13T03:29:53.615355+0000 mon.a (mon.0) 1710 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:54.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:54 smithi183 bash[20824]: cluster 2024-02-13T03:29:53.606519+0000 mon.a (mon.0) 1706 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:29:54.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:54 smithi183 bash[20824]: audit 2024-02-13T03:29:53.612833+0000 mon.a (mon.0) 1707 : audit [INF] from='client.48260 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi161-61719-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi161-61719-39"}]': finished 2024-02-13T03:29:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:54 smithi183 bash[20824]: audit 2024-02-13T03:29:53.612954+0000 mon.a (mon.0) 1708 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-33", "mode": "writeback"}]': finished 2024-02-13T03:29:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:54 smithi183 bash[20824]: cluster 2024-02-13T03:29:53.613078+0000 mon.a (mon.0) 1709 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-02-13T03:29:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:54 smithi183 bash[20824]: audit 2024-02-13T03:29:53.615355+0000 mon.a (mon.0) 1710 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:55.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:55 smithi161 bash[27879]: audit 2024-02-13T03:29:54.621233+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi161-61726-48"}]': finished 2024-02-13T03:29:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:55 smithi161 bash[27879]: cluster 2024-02-13T03:29:54.621667+0000 mon.a (mon.0) 1712 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-02-13T03:29:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:55 smithi161 bash[27879]: audit 2024-02-13T03:29:54.623296+0000 mon.a (mon.0) 1713 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:55 smithi161 bash[27879]: cluster 2024-02-13T03:29:55.294278+0000 mgr.y (mgr.24473) 362 : cluster [DBG] pgmap v434: 236 pgs: 8 unknown, 24 creating+peering, 8 creating+activating, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:55 smithi161 bash[27879]: audit 2024-02-13T03:29:55.622557+0000 mon.a (mon.0) 1714 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi161-61726-48"}]': finished 2024-02-13T03:29:55.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:55 smithi161 bash[27879]: cluster 2024-02-13T03:29:55.622727+0000 mon.a (mon.0) 1715 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-02-13T03:29:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:55 smithi161 bash[19769]: audit 2024-02-13T03:29:54.621233+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi161-61726-48"}]': finished 2024-02-13T03:29:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:55 smithi161 bash[19769]: cluster 2024-02-13T03:29:54.621667+0000 mon.a (mon.0) 1712 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-02-13T03:29:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:55 smithi161 bash[19769]: audit 2024-02-13T03:29:54.623296+0000 mon.a (mon.0) 1713 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:55 smithi161 bash[19769]: cluster 2024-02-13T03:29:55.294278+0000 mgr.y (mgr.24473) 362 : cluster [DBG] pgmap v434: 236 pgs: 8 unknown, 24 creating+peering, 8 creating+activating, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:55 smithi161 bash[19769]: audit 2024-02-13T03:29:55.622557+0000 mon.a (mon.0) 1714 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi161-61726-48"}]': finished 2024-02-13T03:29:55.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:55 smithi161 bash[19769]: cluster 2024-02-13T03:29:55.622727+0000 mon.a (mon.0) 1715 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-02-13T03:29:55.964 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:55 smithi183 bash[20824]: audit 2024-02-13T03:29:54.621233+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi161-61726-48"}]': finished 2024-02-13T03:29:55.964 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:55 smithi183 bash[20824]: cluster 2024-02-13T03:29:54.621667+0000 mon.a (mon.0) 1712 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-02-13T03:29:55.964 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:55 smithi183 bash[20824]: audit 2024-02-13T03:29:54.623296+0000 mon.a (mon.0) 1713 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi161-61726-48"}]: dispatch 2024-02-13T03:29:55.964 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:55 smithi183 bash[20824]: cluster 2024-02-13T03:29:55.294278+0000 mgr.y (mgr.24473) 362 : cluster [DBG] pgmap v434: 236 pgs: 8 unknown, 24 creating+peering, 8 creating+activating, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:29:55.964 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:55 smithi183 bash[20824]: audit 2024-02-13T03:29:55.622557+0000 mon.a (mon.0) 1714 : audit [INF] from='client.? v1:172.21.15.161:0/2987952654' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi161-61726-48"}]': finished 2024-02-13T03:29:55.964 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:55 smithi183 bash[20824]: cluster 2024-02-13T03:29:55.622727+0000 mon.a (mon.0) 1715 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-02-13T03:29:56.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:29:55 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:29:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:55.626951+0000 mon.c (mon.2) 433 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:55.627931+0000 mon.a (mon.0) 1716 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:55.647741+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:29:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:55.649166+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:29:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:55.649500+0000 mon.a (mon.0) 1719 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi161-61726-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:56.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:55.695554+0000 mon.a (mon.0) 1720 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:56.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:55.698238+0000 mon.b (mon.1) 227 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:56.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:55.961690+0000 mgr.y (mgr.24473) 363 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:56.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:56.622834+0000 mon.a (mon.0) 1721 : audit [INF] from='client.48260 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]': finished 2024-02-13T03:29:56.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:56.623036+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi161-61726-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:56.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:56.623173+0000 mon.a (mon.0) 1723 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:56.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: cluster 2024-02-13T03:29:56.623295+0000 mon.a (mon.0) 1724 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-02-13T03:29:56.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:56.624311+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi161-61726-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:29:56.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:56.627769+0000 mon.c (mon.2) 434 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:56.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[27879]: audit 2024-02-13T03:29:56.628264+0000 mon.a (mon.0) 1726 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:55.626951+0000 mon.c (mon.2) 433 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:55.627931+0000 mon.a (mon.0) 1716 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:55.647741+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:55.649166+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:55.649500+0000 mon.a (mon.0) 1719 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi161-61726-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:55.695554+0000 mon.a (mon.0) 1720 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:55.698238+0000 mon.b (mon.1) 227 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:55.961690+0000 mgr.y (mgr.24473) 363 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:56.622834+0000 mon.a (mon.0) 1721 : audit [INF] from='client.48260 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]': finished 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:56.623036+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi161-61726-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:56.623173+0000 mon.a (mon.0) 1723 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: cluster 2024-02-13T03:29:56.623295+0000 mon.a (mon.0) 1724 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:56.624311+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi161-61726-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:29:56.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:56.627769+0000 mon.c (mon.2) 434 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:56.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:56 smithi161 bash[19769]: audit 2024-02-13T03:29:56.628264+0000 mon.a (mon.0) 1726 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:55.626951+0000 mon.c (mon.2) 433 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:55.627931+0000 mon.a (mon.0) 1716 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:55.647741+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:29:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:55.649166+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:29:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:55.649500+0000 mon.a (mon.0) 1719 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi161-61726-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:55.695554+0000 mon.a (mon.0) 1720 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:55.698238+0000 mon.b (mon.1) 227 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:56.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:55.961690+0000 mgr.y (mgr.24473) 363 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:29:56.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:56.622834+0000 mon.a (mon.0) 1721 : audit [INF] from='client.48260 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi161-61719-39"}]': finished 2024-02-13T03:29:56.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:56.623036+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi161-61726-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:56.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:56.623173+0000 mon.a (mon.0) 1723 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:29:56.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: cluster 2024-02-13T03:29:56.623295+0000 mon.a (mon.0) 1724 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-02-13T03:29:56.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:56.624311+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi161-61726-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:29:56.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:56.627769+0000 mon.c (mon.2) 434 : audit [INF] from='client.? v1:172.21.15.161:0/3290220919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:56.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:56 smithi183 bash[20824]: audit 2024-02-13T03:29:56.628264+0000 mon.a (mon.0) 1726 : audit [INF] from='client.48260 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]: dispatch 2024-02-13T03:29:58.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:58 smithi161 bash[27879]: audit 2024-02-13T03:29:56.634643+0000 mon.a (mon.0) 1727 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:29:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:58 smithi161 bash[27879]: audit 2024-02-13T03:29:56.637035+0000 mon.b (mon.1) 228 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:29:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:58 smithi161 bash[27879]: cluster 2024-02-13T03:29:57.295232+0000 mgr.y (mgr.24473) 364 : cluster [DBG] pgmap v437: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:29:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:58 smithi161 bash[27879]: audit 2024-02-13T03:29:58.068364+0000 mon.a (mon.0) 1728 : audit [INF] from='client.48260 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]': finished 2024-02-13T03:29:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:58 smithi161 bash[27879]: audit 2024-02-13T03:29:58.068597+0000 mon.a (mon.0) 1729 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]': finished 2024-02-13T03:29:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:58 smithi161 bash[27879]: cluster 2024-02-13T03:29:58.068763+0000 mon.a (mon.0) 1730 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-02-13T03:29:58.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:58 smithi161 bash[19769]: audit 2024-02-13T03:29:56.634643+0000 mon.a (mon.0) 1727 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:29:58.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:58 smithi161 bash[19769]: audit 2024-02-13T03:29:56.637035+0000 mon.b (mon.1) 228 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:29:58.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:58 smithi161 bash[19769]: cluster 2024-02-13T03:29:57.295232+0000 mgr.y (mgr.24473) 364 : cluster [DBG] pgmap v437: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:29:58.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:58 smithi161 bash[19769]: audit 2024-02-13T03:29:58.068364+0000 mon.a (mon.0) 1728 : audit [INF] from='client.48260 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]': finished 2024-02-13T03:29:58.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:58 smithi161 bash[19769]: audit 2024-02-13T03:29:58.068597+0000 mon.a (mon.0) 1729 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]': finished 2024-02-13T03:29:58.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:58 smithi161 bash[19769]: cluster 2024-02-13T03:29:58.068763+0000 mon.a (mon.0) 1730 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-02-13T03:29:58.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:58 smithi183 bash[20824]: audit 2024-02-13T03:29:56.634643+0000 mon.a (mon.0) 1727 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:29:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:58 smithi183 bash[20824]: audit 2024-02-13T03:29:56.637035+0000 mon.b (mon.1) 228 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:29:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:58 smithi183 bash[20824]: cluster 2024-02-13T03:29:57.295232+0000 mgr.y (mgr.24473) 364 : cluster [DBG] pgmap v437: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:29:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:58 smithi183 bash[20824]: audit 2024-02-13T03:29:58.068364+0000 mon.a (mon.0) 1728 : audit [INF] from='client.48260 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi161-61719-39"}]': finished 2024-02-13T03:29:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:58 smithi183 bash[20824]: audit 2024-02-13T03:29:58.068597+0000 mon.a (mon.0) 1729 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-33"}]': finished 2024-02-13T03:29:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:58 smithi183 bash[20824]: cluster 2024-02-13T03:29:58.068763+0000 mon.a (mon.0) 1730 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-02-13T03:29:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[27879]: audit 2024-02-13T03:29:58.086861+0000 mon.c (mon.2) 435 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[27879]: audit 2024-02-13T03:29:58.087185+0000 mon.a (mon.0) 1731 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[27879]: audit 2024-02-13T03:29:58.088395+0000 mon.c (mon.2) 436 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[27879]: audit 2024-02-13T03:29:58.088672+0000 mon.a (mon.0) 1732 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[27879]: audit 2024-02-13T03:29:58.089135+0000 mon.c (mon.2) 437 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi161-61719-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[27879]: audit 2024-02-13T03:29:58.089416+0000 mon.a (mon.0) 1733 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi161-61719-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[27879]: audit 2024-02-13T03:29:58.631226+0000 mon.a (mon.0) 1734 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi161-61726-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi161-61726-49"}]': finished 2024-02-13T03:29:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[27879]: audit 2024-02-13T03:29:58.631816+0000 mon.a (mon.0) 1735 : audit [INF] from='client.48269 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi161-61719-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[27879]: cluster 2024-02-13T03:29:58.631955+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-02-13T03:29:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[27879]: audit 2024-02-13T03:29:58.632873+0000 mon.c (mon.2) 438 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi161-61719-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[27879]: audit 2024-02-13T03:29:58.637471+0000 mon.a (mon.0) 1737 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi161-61719-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[27879]: audit 2024-02-13T03:29:58.663740+0000 mon.a (mon.0) 1738 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:59.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[27879]: audit 2024-02-13T03:29:58.666424+0000 mon.b (mon.1) 229 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[19769]: audit 2024-02-13T03:29:58.086861+0000 mon.c (mon.2) 435 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[19769]: audit 2024-02-13T03:29:58.087185+0000 mon.a (mon.0) 1731 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[19769]: audit 2024-02-13T03:29:58.088395+0000 mon.c (mon.2) 436 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[19769]: audit 2024-02-13T03:29:58.088672+0000 mon.a (mon.0) 1732 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[19769]: audit 2024-02-13T03:29:58.089135+0000 mon.c (mon.2) 437 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi161-61719-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[19769]: audit 2024-02-13T03:29:58.089416+0000 mon.a (mon.0) 1733 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi161-61719-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:59.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[19769]: audit 2024-02-13T03:29:58.631226+0000 mon.a (mon.0) 1734 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi161-61726-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi161-61726-49"}]': finished 2024-02-13T03:29:59.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[19769]: audit 2024-02-13T03:29:58.631816+0000 mon.a (mon.0) 1735 : audit [INF] from='client.48269 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi161-61719-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:59.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[19769]: cluster 2024-02-13T03:29:58.631955+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-02-13T03:29:59.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[19769]: audit 2024-02-13T03:29:58.632873+0000 mon.c (mon.2) 438 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi161-61719-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[19769]: audit 2024-02-13T03:29:58.637471+0000 mon.a (mon.0) 1737 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi161-61719-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[19769]: audit 2024-02-13T03:29:58.663740+0000 mon.a (mon.0) 1738 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:59.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:29:59 smithi161 bash[19769]: audit 2024-02-13T03:29:58.666424+0000 mon.b (mon.1) 229 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:59.486 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:29:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:29:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[20824]: audit 2024-02-13T03:29:58.086861+0000 mon.c (mon.2) 435 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[20824]: audit 2024-02-13T03:29:58.087185+0000 mon.a (mon.0) 1731 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[20824]: audit 2024-02-13T03:29:58.088395+0000 mon.c (mon.2) 436 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[20824]: audit 2024-02-13T03:29:58.088672+0000 mon.a (mon.0) 1732 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[20824]: audit 2024-02-13T03:29:58.089135+0000 mon.c (mon.2) 437 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi161-61719-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[20824]: audit 2024-02-13T03:29:58.089416+0000 mon.a (mon.0) 1733 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi161-61719-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:29:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[20824]: audit 2024-02-13T03:29:58.631226+0000 mon.a (mon.0) 1734 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi161-61726-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi161-61726-49"}]': finished 2024-02-13T03:29:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[20824]: audit 2024-02-13T03:29:58.631816+0000 mon.a (mon.0) 1735 : audit [INF] from='client.48269 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi161-61719-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:29:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[20824]: cluster 2024-02-13T03:29:58.631955+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-02-13T03:29:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[20824]: audit 2024-02-13T03:29:58.632873+0000 mon.c (mon.2) 438 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi161-61719-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[20824]: audit 2024-02-13T03:29:58.637471+0000 mon.a (mon.0) 1737 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi161-61719-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:29:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[20824]: audit 2024-02-13T03:29:58.663740+0000 mon.a (mon.0) 1738 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:29:59.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:29:59 smithi183 bash[20824]: audit 2024-02-13T03:29:58.666424+0000 mon.b (mon.1) 229 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:00.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[27879]: cluster 2024-02-13T03:29:59.296249+0000 mgr.y (mgr.24473) 365 : cluster [DBG] pgmap v440: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:30:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[27879]: audit 2024-02-13T03:30:00.062334+0000 mon.a (mon.0) 1739 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:30:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[27879]: cluster 2024-02-13T03:30:00.062485+0000 mon.a (mon.0) 1740 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-02-13T03:30:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[27879]: cluster 2024-02-13T03:30:00.062902+0000 mon.a (mon.0) 1741 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 1 pool(s) do not have an application enabled 2024-02-13T03:30:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[27879]: cluster 2024-02-13T03:30:00.062945+0000 mon.a (mon.0) 1742 : cluster [WRN] [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-13T03:30:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[27879]: cluster 2024-02-13T03:30:00.062975+0000 mon.a (mon.0) 1743 : cluster [WRN] pool 'test-rados-api-smithi161-61906-33' with cache_mode writeback needs hit_set_type to be set but it is not 2024-02-13T03:30:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[27879]: cluster 2024-02-13T03:30:00.063002+0000 mon.a (mon.0) 1744 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-13T03:30:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[27879]: cluster 2024-02-13T03:30:00.063025+0000 mon.a (mon.0) 1745 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-13T03:30:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[27879]: cluster 2024-02-13T03:30:00.063050+0000 mon.a (mon.0) 1746 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-13T03:30:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[27879]: audit 2024-02-13T03:30:00.066613+0000 mon.a (mon.0) 1747 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:30:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[27879]: audit 2024-02-13T03:30:00.066692+0000 mon.b (mon.1) 230 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:30:00.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[19769]: cluster 2024-02-13T03:29:59.296249+0000 mgr.y (mgr.24473) 365 : cluster [DBG] pgmap v440: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:30:00.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[19769]: audit 2024-02-13T03:30:00.062334+0000 mon.a (mon.0) 1739 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:30:00.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[19769]: cluster 2024-02-13T03:30:00.062485+0000 mon.a (mon.0) 1740 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-02-13T03:30:00.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[19769]: cluster 2024-02-13T03:30:00.062902+0000 mon.a (mon.0) 1741 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 1 pool(s) do not have an application enabled 2024-02-13T03:30:00.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[19769]: cluster 2024-02-13T03:30:00.062945+0000 mon.a (mon.0) 1742 : cluster [WRN] [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-13T03:30:00.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[19769]: cluster 2024-02-13T03:30:00.062975+0000 mon.a (mon.0) 1743 : cluster [WRN] pool 'test-rados-api-smithi161-61906-33' with cache_mode writeback needs hit_set_type to be set but it is not 2024-02-13T03:30:00.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[19769]: cluster 2024-02-13T03:30:00.063002+0000 mon.a (mon.0) 1744 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-13T03:30:00.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[19769]: cluster 2024-02-13T03:30:00.063025+0000 mon.a (mon.0) 1745 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-13T03:30:00.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[19769]: cluster 2024-02-13T03:30:00.063050+0000 mon.a (mon.0) 1746 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-13T03:30:00.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[19769]: audit 2024-02-13T03:30:00.066613+0000 mon.a (mon.0) 1747 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:30:00.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:00 smithi161 bash[19769]: audit 2024-02-13T03:30:00.066692+0000 mon.b (mon.1) 230 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:30:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:00 smithi183 bash[20824]: cluster 2024-02-13T03:29:59.296249+0000 mgr.y (mgr.24473) 365 : cluster [DBG] pgmap v440: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:30:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:00 smithi183 bash[20824]: audit 2024-02-13T03:30:00.062334+0000 mon.a (mon.0) 1739 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:30:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:00 smithi183 bash[20824]: cluster 2024-02-13T03:30:00.062485+0000 mon.a (mon.0) 1740 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-02-13T03:30:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:00 smithi183 bash[20824]: cluster 2024-02-13T03:30:00.062902+0000 mon.a (mon.0) 1741 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 1 pool(s) do not have an application enabled 2024-02-13T03:30:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:00 smithi183 bash[20824]: cluster 2024-02-13T03:30:00.062945+0000 mon.a (mon.0) 1742 : cluster [WRN] [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-13T03:30:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:00 smithi183 bash[20824]: cluster 2024-02-13T03:30:00.062975+0000 mon.a (mon.0) 1743 : cluster [WRN] pool 'test-rados-api-smithi161-61906-33' with cache_mode writeback needs hit_set_type to be set but it is not 2024-02-13T03:30:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:00 smithi183 bash[20824]: cluster 2024-02-13T03:30:00.063002+0000 mon.a (mon.0) 1744 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-13T03:30:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:00 smithi183 bash[20824]: cluster 2024-02-13T03:30:00.063025+0000 mon.a (mon.0) 1745 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-13T03:30:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:00 smithi183 bash[20824]: cluster 2024-02-13T03:30:00.063050+0000 mon.a (mon.0) 1746 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-13T03:30:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:00 smithi183 bash[20824]: audit 2024-02-13T03:30:00.066613+0000 mon.a (mon.0) 1747 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:30:00.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:00 smithi183 bash[20824]: audit 2024-02-13T03:30:00.066692+0000 mon.b (mon.1) 230 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33"}]: dispatch 2024-02-13T03:30:01.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:30:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:30:01] "GET /metrics HTTP/1.1" 200 185979 "" "Prometheus/2.33.4" 2024-02-13T03:30:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:01 smithi161 bash[27879]: cluster 2024-02-13T03:30:00.627934+0000 mon.a (mon.0) 1748 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:30:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:01 smithi161 bash[27879]: audit 2024-02-13T03:30:00.636263+0000 mon.a (mon.0) 1749 : audit [INF] from='client.48269 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi161-61719-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi161-61719-40"}]': finished 2024-02-13T03:30:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:01 smithi161 bash[27879]: audit 2024-02-13T03:30:00.636998+0000 mon.a (mon.0) 1750 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33"}]': finished 2024-02-13T03:30:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:01 smithi161 bash[27879]: cluster 2024-02-13T03:30:00.637223+0000 mon.a (mon.0) 1751 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-02-13T03:30:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:01 smithi161 bash[27879]: audit 2024-02-13T03:30:00.643332+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:30:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:01 smithi161 bash[19769]: cluster 2024-02-13T03:30:00.627934+0000 mon.a (mon.0) 1748 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:30:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:01 smithi161 bash[19769]: audit 2024-02-13T03:30:00.636263+0000 mon.a (mon.0) 1749 : audit [INF] from='client.48269 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi161-61719-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi161-61719-40"}]': finished 2024-02-13T03:30:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:01 smithi161 bash[19769]: audit 2024-02-13T03:30:00.636998+0000 mon.a (mon.0) 1750 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33"}]': finished 2024-02-13T03:30:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:01 smithi161 bash[19769]: cluster 2024-02-13T03:30:00.637223+0000 mon.a (mon.0) 1751 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-02-13T03:30:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:01 smithi161 bash[19769]: audit 2024-02-13T03:30:00.643332+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:30:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:01 smithi183 bash[20824]: cluster 2024-02-13T03:30:00.627934+0000 mon.a (mon.0) 1748 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:30:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:01 smithi183 bash[20824]: audit 2024-02-13T03:30:00.636263+0000 mon.a (mon.0) 1749 : audit [INF] from='client.48269 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi161-61719-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi161-61719-40"}]': finished 2024-02-13T03:30:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:01 smithi183 bash[20824]: audit 2024-02-13T03:30:00.636998+0000 mon.a (mon.0) 1750 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-33"}]': finished 2024-02-13T03:30:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:01 smithi183 bash[20824]: cluster 2024-02-13T03:30:00.637223+0000 mon.a (mon.0) 1751 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-02-13T03:30:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:01 smithi183 bash[20824]: audit 2024-02-13T03:30:00.643332+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:30:02.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:02 smithi161 bash[19769]: cluster 2024-02-13T03:30:01.297853+0000 mgr.y (mgr.24473) 366 : cluster [DBG] pgmap v443: 236 pgs: 3 creating+peering, 5 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-13T03:30:02.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:02 smithi161 bash[19769]: audit 2024-02-13T03:30:01.639140+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi161-61726-49"}]': finished 2024-02-13T03:30:02.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:02 smithi161 bash[19769]: cluster 2024-02-13T03:30:01.639320+0000 mon.a (mon.0) 1754 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-02-13T03:30:02.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:02 smithi161 bash[19769]: audit 2024-02-13T03:30:01.645430+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:30:02.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:02 smithi161 bash[27879]: cluster 2024-02-13T03:30:01.297853+0000 mgr.y (mgr.24473) 366 : cluster [DBG] pgmap v443: 236 pgs: 3 creating+peering, 5 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-13T03:30:02.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:02 smithi161 bash[27879]: audit 2024-02-13T03:30:01.639140+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi161-61726-49"}]': finished 2024-02-13T03:30:02.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:02 smithi161 bash[27879]: cluster 2024-02-13T03:30:01.639320+0000 mon.a (mon.0) 1754 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-02-13T03:30:02.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:02 smithi161 bash[27879]: audit 2024-02-13T03:30:01.645430+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:30:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:02 smithi183 bash[20824]: cluster 2024-02-13T03:30:01.297853+0000 mgr.y (mgr.24473) 366 : cluster [DBG] pgmap v443: 236 pgs: 3 creating+peering, 5 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-13T03:30:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:02 smithi183 bash[20824]: audit 2024-02-13T03:30:01.639140+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi161-61726-49"}]': finished 2024-02-13T03:30:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:02 smithi183 bash[20824]: cluster 2024-02-13T03:30:01.639320+0000 mon.a (mon.0) 1754 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-02-13T03:30:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:02 smithi183 bash[20824]: audit 2024-02-13T03:30:01.645430+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi161-61726-49"}]: dispatch 2024-02-13T03:30:03.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: audit 2024-02-13T03:30:02.647917+0000 mon.a (mon.0) 1756 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi161-61726-49"}]': finished 2024-02-13T03:30:03.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: cluster 2024-02-13T03:30:02.648198+0000 mon.a (mon.0) 1757 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-02-13T03:30:03.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: audit 2024-02-13T03:30:02.650317+0000 mon.c (mon.2) 439 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:30:03.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: audit 2024-02-13T03:30:02.654464+0000 mon.b (mon.1) 231 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:03.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: audit 2024-02-13T03:30:02.655167+0000 mon.a (mon.0) 1758 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:30:03.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: audit 2024-02-13T03:30:02.655661+0000 mon.a (mon.0) 1759 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:03.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: cluster 2024-02-13T03:30:03.298696+0000 mgr.y (mgr.24473) 367 : cluster [DBG] pgmap v446: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:30:03.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: audit 2024-02-13T03:30:03.541537+0000 mon.a (mon.0) 1760 : audit [INF] from='client.48269 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]': finished 2024-02-13T03:30:03.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: audit 2024-02-13T03:30:03.542292+0000 mon.a (mon.0) 1761 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:03.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: cluster 2024-02-13T03:30:03.543274+0000 mon.a (mon.0) 1762 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-02-13T03:30:03.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: audit 2024-02-13T03:30:03.544392+0000 mon.c (mon.2) 440 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:30:03.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: audit 2024-02-13T03:30:03.548761+0000 mon.b (mon.1) 232 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:03.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: audit 2024-02-13T03:30:03.561188+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? v1:172.21.15.161:0/4065717390' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:03.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: audit 2024-02-13T03:30:03.561855+0000 mon.a (mon.0) 1764 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:30:03.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[19769]: audit 2024-02-13T03:30:03.562961+0000 mon.a (mon.0) 1765 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:03.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: audit 2024-02-13T03:30:02.647917+0000 mon.a (mon.0) 1756 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi161-61726-49"}]': finished 2024-02-13T03:30:03.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: cluster 2024-02-13T03:30:02.648198+0000 mon.a (mon.0) 1757 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-02-13T03:30:03.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: audit 2024-02-13T03:30:02.650317+0000 mon.c (mon.2) 439 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:30:03.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: audit 2024-02-13T03:30:02.654464+0000 mon.b (mon.1) 231 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:03.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: audit 2024-02-13T03:30:02.655167+0000 mon.a (mon.0) 1758 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:30:03.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: audit 2024-02-13T03:30:02.655661+0000 mon.a (mon.0) 1759 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:03.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: cluster 2024-02-13T03:30:03.298696+0000 mgr.y (mgr.24473) 367 : cluster [DBG] pgmap v446: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:30:03.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: audit 2024-02-13T03:30:03.541537+0000 mon.a (mon.0) 1760 : audit [INF] from='client.48269 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]': finished 2024-02-13T03:30:03.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: audit 2024-02-13T03:30:03.542292+0000 mon.a (mon.0) 1761 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:03.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: cluster 2024-02-13T03:30:03.543274+0000 mon.a (mon.0) 1762 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-02-13T03:30:03.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: audit 2024-02-13T03:30:03.544392+0000 mon.c (mon.2) 440 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:30:03.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: audit 2024-02-13T03:30:03.548761+0000 mon.b (mon.1) 232 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:03.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: audit 2024-02-13T03:30:03.561188+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? v1:172.21.15.161:0/4065717390' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:03.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: audit 2024-02-13T03:30:03.561855+0000 mon.a (mon.0) 1764 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:30:03.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:03 smithi161 bash[27879]: audit 2024-02-13T03:30:03.562961+0000 mon.a (mon.0) 1765 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: audit 2024-02-13T03:30:02.647917+0000 mon.a (mon.0) 1756 : audit [INF] from='client.? v1:172.21.15.161:0/2683969115' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi161-61726-49"}]': finished 2024-02-13T03:30:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: cluster 2024-02-13T03:30:02.648198+0000 mon.a (mon.0) 1757 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-02-13T03:30:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: audit 2024-02-13T03:30:02.650317+0000 mon.c (mon.2) 439 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:30:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: audit 2024-02-13T03:30:02.654464+0000 mon.b (mon.1) 231 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: audit 2024-02-13T03:30:02.655167+0000 mon.a (mon.0) 1758 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:30:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: audit 2024-02-13T03:30:02.655661+0000 mon.a (mon.0) 1759 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: cluster 2024-02-13T03:30:03.298696+0000 mgr.y (mgr.24473) 367 : cluster [DBG] pgmap v446: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:30:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: audit 2024-02-13T03:30:03.541537+0000 mon.a (mon.0) 1760 : audit [INF] from='client.48269 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi161-61719-40"}]': finished 2024-02-13T03:30:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: audit 2024-02-13T03:30:03.542292+0000 mon.a (mon.0) 1761 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: cluster 2024-02-13T03:30:03.543274+0000 mon.a (mon.0) 1762 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-02-13T03:30:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: audit 2024-02-13T03:30:03.544392+0000 mon.c (mon.2) 440 : audit [INF] from='client.? v1:172.21.15.161:0/1836851455' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:30:03.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: audit 2024-02-13T03:30:03.548761+0000 mon.b (mon.1) 232 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:03.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: audit 2024-02-13T03:30:03.561188+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? v1:172.21.15.161:0/4065717390' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:03.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: audit 2024-02-13T03:30:03.561855+0000 mon.a (mon.0) 1764 : audit [INF] from='client.48269 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]: dispatch 2024-02-13T03:30:03.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:03 smithi183 bash[20824]: audit 2024-02-13T03:30:03.562961+0000 mon.a (mon.0) 1765 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:05.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[27879]: audit 2024-02-13T03:30:04.537984+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? v1:172.21.15.161:0/4065717390' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[27879]: audit 2024-02-13T03:30:04.538576+0000 mon.a (mon.0) 1767 : audit [INF] from='client.48269 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]': finished 2024-02-13T03:30:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[27879]: audit 2024-02-13T03:30:04.538847+0000 mon.a (mon.0) 1768 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[27879]: cluster 2024-02-13T03:30:04.539088+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-02-13T03:30:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[27879]: audit 2024-02-13T03:30:04.542601+0000 mon.a (mon.0) 1770 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-35"}]: dispatch 2024-02-13T03:30:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[27879]: audit 2024-02-13T03:30:04.544819+0000 mon.b (mon.1) 233 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-35"}]: dispatch 2024-02-13T03:30:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[27879]: audit 2024-02-13T03:30:04.568857+0000 mon.c (mon.2) 441 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[27879]: audit 2024-02-13T03:30:04.569185+0000 mon.a (mon.0) 1771 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[27879]: audit 2024-02-13T03:30:04.570401+0000 mon.c (mon.2) 442 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[27879]: audit 2024-02-13T03:30:04.570676+0000 mon.a (mon.0) 1772 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[27879]: audit 2024-02-13T03:30:04.571132+0000 mon.c (mon.2) 443 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi161-61719-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[27879]: audit 2024-02-13T03:30:04.571482+0000 mon.a (mon.0) 1773 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi161-61719-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[27879]: cluster 2024-02-13T03:30:05.299529+0000 mgr.y (mgr.24473) 368 : cluster [DBG] pgmap v449: 260 pgs: 64 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:05.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[19769]: audit 2024-02-13T03:30:04.537984+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? v1:172.21.15.161:0/4065717390' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:05.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[19769]: audit 2024-02-13T03:30:04.538576+0000 mon.a (mon.0) 1767 : audit [INF] from='client.48269 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]': finished 2024-02-13T03:30:05.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[19769]: audit 2024-02-13T03:30:04.538847+0000 mon.a (mon.0) 1768 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:05.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[19769]: cluster 2024-02-13T03:30:04.539088+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-02-13T03:30:05.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[19769]: audit 2024-02-13T03:30:04.542601+0000 mon.a (mon.0) 1770 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-35"}]: dispatch 2024-02-13T03:30:05.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[19769]: audit 2024-02-13T03:30:04.544819+0000 mon.b (mon.1) 233 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-35"}]: dispatch 2024-02-13T03:30:05.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[19769]: audit 2024-02-13T03:30:04.568857+0000 mon.c (mon.2) 441 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:05.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[19769]: audit 2024-02-13T03:30:04.569185+0000 mon.a (mon.0) 1771 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:05.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[19769]: audit 2024-02-13T03:30:04.570401+0000 mon.c (mon.2) 442 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:05.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[19769]: audit 2024-02-13T03:30:04.570676+0000 mon.a (mon.0) 1772 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:05.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[19769]: audit 2024-02-13T03:30:04.571132+0000 mon.c (mon.2) 443 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi161-61719-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:05.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[19769]: audit 2024-02-13T03:30:04.571482+0000 mon.a (mon.0) 1773 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi161-61719-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:05.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:05 smithi161 bash[19769]: cluster 2024-02-13T03:30:05.299529+0000 mgr.y (mgr.24473) 368 : cluster [DBG] pgmap v449: 260 pgs: 64 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:05.966 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[20824]: audit 2024-02-13T03:30:04.537984+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? v1:172.21.15.161:0/4065717390' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi161-61726-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:05.966 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[20824]: audit 2024-02-13T03:30:04.538576+0000 mon.a (mon.0) 1767 : audit [INF] from='client.48269 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi161-61719-40"}]': finished 2024-02-13T03:30:05.966 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[20824]: audit 2024-02-13T03:30:04.538847+0000 mon.a (mon.0) 1768 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:05.967 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[20824]: cluster 2024-02-13T03:30:04.539088+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-02-13T03:30:05.967 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[20824]: audit 2024-02-13T03:30:04.542601+0000 mon.a (mon.0) 1770 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-35"}]: dispatch 2024-02-13T03:30:05.967 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[20824]: audit 2024-02-13T03:30:04.544819+0000 mon.b (mon.1) 233 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-35"}]: dispatch 2024-02-13T03:30:05.967 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[20824]: audit 2024-02-13T03:30:04.568857+0000 mon.c (mon.2) 441 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:05.967 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[20824]: audit 2024-02-13T03:30:04.569185+0000 mon.a (mon.0) 1771 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:05.967 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[20824]: audit 2024-02-13T03:30:04.570401+0000 mon.c (mon.2) 442 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:05.967 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[20824]: audit 2024-02-13T03:30:04.570676+0000 mon.a (mon.0) 1772 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:05.967 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[20824]: audit 2024-02-13T03:30:04.571132+0000 mon.c (mon.2) 443 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi161-61719-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:05.967 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[20824]: audit 2024-02-13T03:30:04.571482+0000 mon.a (mon.0) 1773 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi161-61719-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:05.967 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[20824]: cluster 2024-02-13T03:30:05.299529+0000 mgr.y (mgr.24473) 368 : cluster [DBG] pgmap v449: 260 pgs: 64 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:06.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:30:05 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:30:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.541064+0000 mon.a (mon.0) 1774 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-35"}]': finished 2024-02-13T03:30:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.541470+0000 mon.a (mon.0) 1775 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi161-61719-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: cluster 2024-02-13T03:30:05.541583+0000 mon.a (mon.0) 1776 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-02-13T03:30:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.543191+0000 mon.a (mon.0) 1777 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-35", "mode": "writeback"}]: dispatch 2024-02-13T03:30:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.543210+0000 mon.c (mon.2) 444 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi161-61719-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.545583+0000 mon.b (mon.1) 234 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-35", "mode": "writeback"}]: dispatch 2024-02-13T03:30:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.552257+0000 mon.a (mon.0) 1778 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi161-61719-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.564313+0000 mon.c (mon.2) 445 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.564693+0000 mon.a (mon.0) 1779 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.565593+0000 mon.c (mon.2) 446 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.565903+0000 mon.a (mon.0) 1780 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.566430+0000 mon.c (mon.2) 447 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.566757+0000 mon.a (mon.0) 1781 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.577104+0000 mon.c (mon.2) 448 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:30:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.894652+0000 mon.c (mon.2) 449 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:30:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.895806+0000 mon.c (mon.2) 450 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:30:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.903823+0000 mon.a (mon.0) 1782 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:30:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:05.963649+0000 mgr.y (mgr.24473) 369 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: cluster 2024-02-13T03:30:06.534861+0000 mon.a (mon.0) 1783 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:30:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:06 smithi183 bash[20824]: audit 2024-02-13T03:30:06.542966+0000 mon.a (mon.0) 1784 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-35", "mode": "writeback"}]': finished 2024-02-13T03:30:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.541064+0000 mon.a (mon.0) 1774 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-35"}]': finished 2024-02-13T03:30:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.541470+0000 mon.a (mon.0) 1775 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi161-61719-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: cluster 2024-02-13T03:30:05.541583+0000 mon.a (mon.0) 1776 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-02-13T03:30:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.543191+0000 mon.a (mon.0) 1777 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-35", "mode": "writeback"}]: dispatch 2024-02-13T03:30:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.543210+0000 mon.c (mon.2) 444 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi161-61719-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.545583+0000 mon.b (mon.1) 234 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-35", "mode": "writeback"}]: dispatch 2024-02-13T03:30:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.552257+0000 mon.a (mon.0) 1778 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi161-61719-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.564313+0000 mon.c (mon.2) 445 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.564693+0000 mon.a (mon.0) 1779 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:07.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.565593+0000 mon.c (mon.2) 446 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:07.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.565903+0000 mon.a (mon.0) 1780 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:07.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.566430+0000 mon.c (mon.2) 447 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:07.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.566757+0000 mon.a (mon.0) 1781 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:07.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.577104+0000 mon.c (mon.2) 448 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:30:07.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.541064+0000 mon.a (mon.0) 1774 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-35"}]': finished 2024-02-13T03:30:07.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.541470+0000 mon.a (mon.0) 1775 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi161-61719-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:07.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: cluster 2024-02-13T03:30:05.541583+0000 mon.a (mon.0) 1776 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-02-13T03:30:07.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.543191+0000 mon.a (mon.0) 1777 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-35", "mode": "writeback"}]: dispatch 2024-02-13T03:30:07.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.543210+0000 mon.c (mon.2) 444 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi161-61719-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:07.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.545583+0000 mon.b (mon.1) 234 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-35", "mode": "writeback"}]: dispatch 2024-02-13T03:30:07.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.552257+0000 mon.a (mon.0) 1778 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi161-61719-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:07.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.564313+0000 mon.c (mon.2) 445 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:07.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.564693+0000 mon.a (mon.0) 1779 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:07.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.565593+0000 mon.c (mon.2) 446 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:07.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.565903+0000 mon.a (mon.0) 1780 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:07.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.566430+0000 mon.c (mon.2) 447 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:07.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.566757+0000 mon.a (mon.0) 1781 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:07.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.577104+0000 mon.c (mon.2) 448 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:30:07.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.894652+0000 mon.c (mon.2) 449 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:30:07.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.895806+0000 mon.c (mon.2) 450 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:30:07.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.903823+0000 mon.a (mon.0) 1782 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:30:07.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:05.963649+0000 mgr.y (mgr.24473) 369 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:07.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: cluster 2024-02-13T03:30:06.534861+0000 mon.a (mon.0) 1783 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:30:07.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[27879]: audit 2024-02-13T03:30:06.542966+0000 mon.a (mon.0) 1784 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-35", "mode": "writeback"}]': finished 2024-02-13T03:30:07.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.894652+0000 mon.c (mon.2) 449 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:30:07.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.895806+0000 mon.c (mon.2) 450 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:30:07.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.903823+0000 mon.a (mon.0) 1782 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:30:07.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:05.963649+0000 mgr.y (mgr.24473) 369 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:07.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: cluster 2024-02-13T03:30:06.534861+0000 mon.a (mon.0) 1783 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:30:07.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:06 smithi161 bash[19769]: audit 2024-02-13T03:30:06.542966+0000 mon.a (mon.0) 1784 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-35", "mode": "writeback"}]': finished 2024-02-13T03:30:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[27879]: audit 2024-02-13T03:30:06.637662+0000 mon.b (mon.1) 235 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[27879]: audit 2024-02-13T03:30:06.973544+0000 mon.a (mon.0) 1785 : audit [INF] from='client.48287 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[27879]: cluster 2024-02-13T03:30:06.973810+0000 mon.a (mon.0) 1786 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-02-13T03:30:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[27879]: audit 2024-02-13T03:30:06.974723+0000 mon.c (mon.2) 451 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi161-61726-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[27879]: audit 2024-02-13T03:30:06.976243+0000 mon.a (mon.0) 1787 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[27879]: audit 2024-02-13T03:30:06.976981+0000 mon.a (mon.0) 1788 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi161-61726-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[27879]: cluster 2024-02-13T03:30:07.300361+0000 mgr.y (mgr.24473) 370 : cluster [DBG] pgmap v452: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[27879]: audit 2024-02-13T03:30:07.549137+0000 mon.a (mon.0) 1789 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi161-61719-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi161-61719-41"}]': finished 2024-02-13T03:30:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[27879]: audit 2024-02-13T03:30:07.549305+0000 mon.a (mon.0) 1790 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:30:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[27879]: audit 2024-02-13T03:30:07.560573+0000 mon.b (mon.1) 236 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35"}]: dispatch 2024-02-13T03:30:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[27879]: cluster 2024-02-13T03:30:07.956287+0000 mon.a (mon.0) 1791 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-02-13T03:30:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[27879]: audit 2024-02-13T03:30:07.959482+0000 mon.a (mon.0) 1792 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35"}]: dispatch 2024-02-13T03:30:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[19769]: audit 2024-02-13T03:30:06.637662+0000 mon.b (mon.1) 235 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[19769]: audit 2024-02-13T03:30:06.973544+0000 mon.a (mon.0) 1785 : audit [INF] from='client.48287 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[19769]: cluster 2024-02-13T03:30:06.973810+0000 mon.a (mon.0) 1786 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-02-13T03:30:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[19769]: audit 2024-02-13T03:30:06.974723+0000 mon.c (mon.2) 451 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi161-61726-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[19769]: audit 2024-02-13T03:30:06.976243+0000 mon.a (mon.0) 1787 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[19769]: audit 2024-02-13T03:30:06.976981+0000 mon.a (mon.0) 1788 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi161-61726-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[19769]: cluster 2024-02-13T03:30:07.300361+0000 mgr.y (mgr.24473) 370 : cluster [DBG] pgmap v452: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[19769]: audit 2024-02-13T03:30:07.549137+0000 mon.a (mon.0) 1789 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi161-61719-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi161-61719-41"}]': finished 2024-02-13T03:30:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[19769]: audit 2024-02-13T03:30:07.549305+0000 mon.a (mon.0) 1790 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:30:08.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[19769]: audit 2024-02-13T03:30:07.560573+0000 mon.b (mon.1) 236 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35"}]: dispatch 2024-02-13T03:30:08.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[19769]: cluster 2024-02-13T03:30:07.956287+0000 mon.a (mon.0) 1791 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-02-13T03:30:08.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:07 smithi161 bash[19769]: audit 2024-02-13T03:30:07.959482+0000 mon.a (mon.0) 1792 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35"}]: dispatch 2024-02-13T03:30:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:07 smithi183 bash[20824]: audit 2024-02-13T03:30:06.637662+0000 mon.b (mon.1) 235 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:07 smithi183 bash[20824]: audit 2024-02-13T03:30:06.973544+0000 mon.a (mon.0) 1785 : audit [INF] from='client.48287 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:07 smithi183 bash[20824]: cluster 2024-02-13T03:30:06.973810+0000 mon.a (mon.0) 1786 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-02-13T03:30:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:07 smithi183 bash[20824]: audit 2024-02-13T03:30:06.974723+0000 mon.c (mon.2) 451 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi161-61726-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:07 smithi183 bash[20824]: audit 2024-02-13T03:30:06.976243+0000 mon.a (mon.0) 1787 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:07 smithi183 bash[20824]: audit 2024-02-13T03:30:06.976981+0000 mon.a (mon.0) 1788 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi161-61726-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:07 smithi183 bash[20824]: cluster 2024-02-13T03:30:07.300361+0000 mgr.y (mgr.24473) 370 : cluster [DBG] pgmap v452: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:07 smithi183 bash[20824]: audit 2024-02-13T03:30:07.549137+0000 mon.a (mon.0) 1789 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi161-61719-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi161-61719-41"}]': finished 2024-02-13T03:30:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:07 smithi183 bash[20824]: audit 2024-02-13T03:30:07.549305+0000 mon.a (mon.0) 1790 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:30:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:07 smithi183 bash[20824]: audit 2024-02-13T03:30:07.560573+0000 mon.b (mon.1) 236 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35"}]: dispatch 2024-02-13T03:30:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:07 smithi183 bash[20824]: cluster 2024-02-13T03:30:07.956287+0000 mon.a (mon.0) 1791 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-02-13T03:30:08.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:07 smithi183 bash[20824]: audit 2024-02-13T03:30:07.959482+0000 mon.a (mon.0) 1792 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35"}]: dispatch 2024-02-13T03:30:09.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:30:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:30:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:30:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:09 smithi183 bash[20824]: cluster 2024-02-13T03:30:08.541356+0000 mon.a (mon.0) 1793 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:30:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:09 smithi183 bash[20824]: audit 2024-02-13T03:30:08.549316+0000 mon.a (mon.0) 1794 : audit [INF] from='client.48287 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi161-61726-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]': finished 2024-02-13T03:30:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:09 smithi183 bash[20824]: audit 2024-02-13T03:30:08.550213+0000 mon.a (mon.0) 1795 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35"}]': finished 2024-02-13T03:30:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:09 smithi183 bash[20824]: cluster 2024-02-13T03:30:08.550479+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-02-13T03:30:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:09 smithi183 bash[20824]: cluster 2024-02-13T03:30:09.301222+0000 mgr.y (mgr.24473) 371 : cluster [DBG] pgmap v455: 244 pgs: 16 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:09.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:09 smithi161 bash[27879]: cluster 2024-02-13T03:30:08.541356+0000 mon.a (mon.0) 1793 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:30:09.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:09 smithi161 bash[27879]: audit 2024-02-13T03:30:08.549316+0000 mon.a (mon.0) 1794 : audit [INF] from='client.48287 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi161-61726-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]': finished 2024-02-13T03:30:09.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:09 smithi161 bash[27879]: audit 2024-02-13T03:30:08.550213+0000 mon.a (mon.0) 1795 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35"}]': finished 2024-02-13T03:30:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:09 smithi161 bash[27879]: cluster 2024-02-13T03:30:08.550479+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-02-13T03:30:09.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:09 smithi161 bash[27879]: cluster 2024-02-13T03:30:09.301222+0000 mgr.y (mgr.24473) 371 : cluster [DBG] pgmap v455: 244 pgs: 16 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:09.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:09 smithi161 bash[19769]: cluster 2024-02-13T03:30:08.541356+0000 mon.a (mon.0) 1793 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:30:09.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:09 smithi161 bash[19769]: audit 2024-02-13T03:30:08.549316+0000 mon.a (mon.0) 1794 : audit [INF] from='client.48287 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi161-61726-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]': finished 2024-02-13T03:30:09.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:09 smithi161 bash[19769]: audit 2024-02-13T03:30:08.550213+0000 mon.a (mon.0) 1795 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-35"}]': finished 2024-02-13T03:30:09.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:09 smithi161 bash[19769]: cluster 2024-02-13T03:30:08.550479+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-02-13T03:30:09.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:09 smithi161 bash[19769]: cluster 2024-02-13T03:30:09.301222+0000 mgr.y (mgr.24473) 371 : cluster [DBG] pgmap v455: 244 pgs: 16 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:10.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:10 smithi161 bash[27879]: cluster 2024-02-13T03:30:09.556659+0000 mon.a (mon.0) 1797 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-02-13T03:30:10.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:10 smithi161 bash[27879]: audit 2024-02-13T03:30:09.561671+0000 mon.c (mon.2) 452 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:10 smithi161 bash[27879]: audit 2024-02-13T03:30:09.562609+0000 mon.a (mon.0) 1798 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:10.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:10 smithi161 bash[19769]: cluster 2024-02-13T03:30:09.556659+0000 mon.a (mon.0) 1797 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-02-13T03:30:10.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:10 smithi161 bash[19769]: audit 2024-02-13T03:30:09.561671+0000 mon.c (mon.2) 452 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:10.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:10 smithi161 bash[19769]: audit 2024-02-13T03:30:09.562609+0000 mon.a (mon.0) 1798 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:10.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:10 smithi183 bash[20824]: cluster 2024-02-13T03:30:09.556659+0000 mon.a (mon.0) 1797 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-02-13T03:30:10.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:10 smithi183 bash[20824]: audit 2024-02-13T03:30:09.561671+0000 mon.c (mon.2) 452 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:10.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:10 smithi183 bash[20824]: audit 2024-02-13T03:30:09.562609+0000 mon.a (mon.0) 1798 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:11.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:30:11] "GET /metrics HTTP/1.1" 200 188635 "" "Prometheus/2.33.4" 2024-02-13T03:30:11.572 INFO:tasks.workunit.client.0.smithi161.stdout:ete 2024-02-13T03:30:11.572 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6913 ms) 2024-02-13T03:30:11.572 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-02-13T03:30:11.572 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7016 ms) 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7438 ms) 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (6581 ms) 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7066 ms) 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7049 ms) 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7034 ms) 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6863 ms) 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-02-13T03:30:11.573 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7034 ms) 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7371 ms) 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7046 ms) 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7017 ms) 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7050 ms) 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (6472 ms) 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7026 ms) 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112300 ms total) 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: 2024-02-13T03:30:11.574 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [----------] Global test environment tear-down 2024-02-13T03:30:11.575 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [==========] 41 tests from 2 test suites ran. (205361 ms total) 2024-02-13T03:30:11.575 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio: [ PASSED ] 41 tests. 2024-02-13T03:30:11.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:10.565302+0000 mon.a (mon.0) 1799 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]': finished 2024-02-13T03:30:11.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: cluster 2024-02-13T03:30:10.566206+0000 mon.a (mon.0) 1800 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-02-13T03:30:11.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:10.567359+0000 mon.c (mon.2) 453 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:11.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:10.567717+0000 mon.c (mon.2) 454 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:11.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:10.569588+0000 mon.a (mon.0) 1801 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:11.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:10.570002+0000 mon.a (mon.0) 1802 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:11.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:10.570425+0000 mon.a (mon.0) 1803 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:11.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:10.570488+0000 mon.b (mon.1) 237 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:11.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: cluster 2024-02-13T03:30:11.303399+0000 mgr.y (mgr.24473) 372 : cluster [DBG] pgmap v458: 228 pgs: 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:30:11.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:11.564738+0000 mon.a (mon.0) 1804 : audit [INF] from='client.48287 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]': finished 2024-02-13T03:30:11.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:11.566299+0000 mon.a (mon.0) 1805 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]': finished 2024-02-13T03:30:11.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:11.566406+0000 mon.a (mon.0) 1806 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:11.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: cluster 2024-02-13T03:30:11.566615+0000 mon.a (mon.0) 1807 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-02-13T03:30:11.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:11.567862+0000 mon.c (mon.2) 455 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:11.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:11.568932+0000 mon.a (mon.0) 1808 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:10.565302+0000 mon.a (mon.0) 1799 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]': finished 2024-02-13T03:30:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: cluster 2024-02-13T03:30:10.566206+0000 mon.a (mon.0) 1800 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-02-13T03:30:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:10.567359+0000 mon.c (mon.2) 453 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:10.567717+0000 mon.c (mon.2) 454 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:10.569588+0000 mon.a (mon.0) 1801 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:10.570002+0000 mon.a (mon.0) 1802 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:10.570425+0000 mon.a (mon.0) 1803 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:10.570488+0000 mon.b (mon.1) 237 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: cluster 2024-02-13T03:30:11.303399+0000 mgr.y (mgr.24473) 372 : cluster [DBG] pgmap v458: 228 pgs: 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:30:11.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:11.564738+0000 mon.a (mon.0) 1804 : audit [INF] from='client.48287 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]': finished 2024-02-13T03:30:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:11.566299+0000 mon.a (mon.0) 1805 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]': finished 2024-02-13T03:30:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:11.566406+0000 mon.a (mon.0) 1806 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: cluster 2024-02-13T03:30:11.566615+0000 mon.a (mon.0) 1807 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-02-13T03:30:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:11.567862+0000 mon.c (mon.2) 455 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:11.568932+0000 mon.a (mon.0) 1808 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:11.571100+0000 mon.a (mon.0) 1809 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:11.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[19769]: audit 2024-02-13T03:30:11.571686+0000 mon.b (mon.1) 238 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:11.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:11.571100+0000 mon.a (mon.0) 1809 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:11.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:11 smithi161 bash[27879]: audit 2024-02-13T03:30:11.571686+0000 mon.b (mon.1) 238 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:10.565302+0000 mon.a (mon.0) 1799 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi161-61719-41"}]': finished 2024-02-13T03:30:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: cluster 2024-02-13T03:30:10.566206+0000 mon.a (mon.0) 1800 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-02-13T03:30:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:10.567359+0000 mon.c (mon.2) 453 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:10.567717+0000 mon.c (mon.2) 454 : audit [INF] from='client.? v1:172.21.15.161:0/1913709786' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:10.569588+0000 mon.a (mon.0) 1801 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:10.570002+0000 mon.a (mon.0) 1802 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]: dispatch 2024-02-13T03:30:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:10.570425+0000 mon.a (mon.0) 1803 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:10.570488+0000 mon.b (mon.1) 237 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: cluster 2024-02-13T03:30:11.303399+0000 mgr.y (mgr.24473) 372 : cluster [DBG] pgmap v458: 228 pgs: 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:30:11.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:11.564738+0000 mon.a (mon.0) 1804 : audit [INF] from='client.48287 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi161-61726-51"}]': finished 2024-02-13T03:30:11.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:11.566299+0000 mon.a (mon.0) 1805 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi161-61719-41"}]': finished 2024-02-13T03:30:11.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:11.566406+0000 mon.a (mon.0) 1806 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:11.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: cluster 2024-02-13T03:30:11.566615+0000 mon.a (mon.0) 1807 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-02-13T03:30:11.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:11.567862+0000 mon.c (mon.2) 455 : audit [INF] from='client.? v1:172.21.15.161:0/1927344279' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:11.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:11.568932+0000 mon.a (mon.0) 1808 : audit [INF] from='client.48287 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]: dispatch 2024-02-13T03:30:11.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:11.571100+0000 mon.a (mon.0) 1809 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:11.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:11 smithi183 bash[20824]: audit 2024-02-13T03:30:11.571686+0000 mon.b (mon.1) 238 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:13.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[19769]: audit 2024-02-13T03:30:12.568037+0000 mon.a (mon.0) 1810 : audit [INF] from='client.48287 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]': finished 2024-02-13T03:30:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[19769]: audit 2024-02-13T03:30:12.568221+0000 mon.a (mon.0) 1811 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[19769]: cluster 2024-02-13T03:30:12.568321+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-02-13T03:30:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[19769]: audit 2024-02-13T03:30:12.572266+0000 mon.a (mon.0) 1813 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-37"}]: dispatch 2024-02-13T03:30:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[19769]: audit 2024-02-13T03:30:12.572437+0000 mon.b (mon.1) 239 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-37"}]: dispatch 2024-02-13T03:30:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[19769]: audit 2024-02-13T03:30:12.589332+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[19769]: audit 2024-02-13T03:30:12.591020+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[19769]: audit 2024-02-13T03:30:12.591411+0000 mon.a (mon.0) 1816 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:13.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[19769]: cluster 2024-02-13T03:30:13.304234+0000 mgr.y (mgr.24473) 373 : cluster [DBG] pgmap v461: 228 pgs: 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:30:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[27879]: audit 2024-02-13T03:30:12.568037+0000 mon.a (mon.0) 1810 : audit [INF] from='client.48287 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]': finished 2024-02-13T03:30:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[27879]: audit 2024-02-13T03:30:12.568221+0000 mon.a (mon.0) 1811 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[27879]: cluster 2024-02-13T03:30:12.568321+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-02-13T03:30:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[27879]: audit 2024-02-13T03:30:12.572266+0000 mon.a (mon.0) 1813 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-37"}]: dispatch 2024-02-13T03:30:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[27879]: audit 2024-02-13T03:30:12.572437+0000 mon.b (mon.1) 239 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-37"}]: dispatch 2024-02-13T03:30:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[27879]: audit 2024-02-13T03:30:12.589332+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[27879]: audit 2024-02-13T03:30:12.591020+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[27879]: audit 2024-02-13T03:30:12.591411+0000 mon.a (mon.0) 1816 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:13.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:13 smithi161 bash[27879]: cluster 2024-02-13T03:30:13.304234+0000 mgr.y (mgr.24473) 373 : cluster [DBG] pgmap v461: 228 pgs: 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:30:13.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:13 smithi183 bash[20824]: audit 2024-02-13T03:30:12.568037+0000 mon.a (mon.0) 1810 : audit [INF] from='client.48287 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi161-61726-51"}]': finished 2024-02-13T03:30:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:13 smithi183 bash[20824]: audit 2024-02-13T03:30:12.568221+0000 mon.a (mon.0) 1811 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:13 smithi183 bash[20824]: cluster 2024-02-13T03:30:12.568321+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-02-13T03:30:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:13 smithi183 bash[20824]: audit 2024-02-13T03:30:12.572266+0000 mon.a (mon.0) 1813 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-37"}]: dispatch 2024-02-13T03:30:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:13 smithi183 bash[20824]: audit 2024-02-13T03:30:12.572437+0000 mon.b (mon.1) 239 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-37"}]: dispatch 2024-02-13T03:30:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:13 smithi183 bash[20824]: audit 2024-02-13T03:30:12.589332+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:13 smithi183 bash[20824]: audit 2024-02-13T03:30:12.591020+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:13 smithi183 bash[20824]: audit 2024-02-13T03:30:12.591411+0000 mon.a (mon.0) 1816 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:13.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:13 smithi183 bash[20824]: cluster 2024-02-13T03:30:13.304234+0000 mgr.y (mgr.24473) 373 : cluster [DBG] pgmap v461: 228 pgs: 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:30:15.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[19769]: audit 2024-02-13T03:30:13.570843+0000 mon.a (mon.0) 1817 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-37"}]': finished 2024-02-13T03:30:15.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[19769]: audit 2024-02-13T03:30:13.570987+0000 mon.a (mon.0) 1818 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:15.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[19769]: cluster 2024-02-13T03:30:13.571073+0000 mon.a (mon.0) 1819 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-02-13T03:30:15.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[19769]: audit 2024-02-13T03:30:13.572594+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi161-61726-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:15.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[19769]: audit 2024-02-13T03:30:13.576276+0000 mon.b (mon.1) 240 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-37", "mode": "writeback"}]: dispatch 2024-02-13T03:30:15.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[19769]: audit 2024-02-13T03:30:13.580177+0000 mon.a (mon.0) 1821 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-37", "mode": "writeback"}]: dispatch 2024-02-13T03:30:15.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[19769]: cluster 2024-02-13T03:30:14.566673+0000 mon.a (mon.0) 1822 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:30:15.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[27879]: audit 2024-02-13T03:30:13.570843+0000 mon.a (mon.0) 1817 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-37"}]': finished 2024-02-13T03:30:15.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[27879]: audit 2024-02-13T03:30:13.570987+0000 mon.a (mon.0) 1818 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:15.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[27879]: cluster 2024-02-13T03:30:13.571073+0000 mon.a (mon.0) 1819 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-02-13T03:30:15.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[27879]: audit 2024-02-13T03:30:13.572594+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi161-61726-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:15.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[27879]: audit 2024-02-13T03:30:13.576276+0000 mon.b (mon.1) 240 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-37", "mode": "writeback"}]: dispatch 2024-02-13T03:30:15.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[27879]: audit 2024-02-13T03:30:13.580177+0000 mon.a (mon.0) 1821 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-37", "mode": "writeback"}]: dispatch 2024-02-13T03:30:15.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:15 smithi161 bash[27879]: cluster 2024-02-13T03:30:14.566673+0000 mon.a (mon.0) 1822 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:30:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:15 smithi183 bash[20824]: audit 2024-02-13T03:30:13.570843+0000 mon.a (mon.0) 1817 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-37"}]': finished 2024-02-13T03:30:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:15 smithi183 bash[20824]: audit 2024-02-13T03:30:13.570987+0000 mon.a (mon.0) 1818 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:15.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:15 smithi183 bash[20824]: cluster 2024-02-13T03:30:13.571073+0000 mon.a (mon.0) 1819 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-02-13T03:30:15.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:15 smithi183 bash[20824]: audit 2024-02-13T03:30:13.572594+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi161-61726-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:15.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:15 smithi183 bash[20824]: audit 2024-02-13T03:30:13.576276+0000 mon.b (mon.1) 240 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-37", "mode": "writeback"}]: dispatch 2024-02-13T03:30:15.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:15 smithi183 bash[20824]: audit 2024-02-13T03:30:13.580177+0000 mon.a (mon.0) 1821 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-37", "mode": "writeback"}]: dispatch 2024-02-13T03:30:15.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:15 smithi183 bash[20824]: cluster 2024-02-13T03:30:14.566673+0000 mon.a (mon.0) 1822 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:30:16.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:30:15 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:30:16.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:16 smithi183 bash[20824]: audit 2024-02-13T03:30:15.003517+0000 mon.a (mon.0) 1823 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-37", "mode": "writeback"}]': finished 2024-02-13T03:30:16.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:16 smithi183 bash[20824]: cluster 2024-02-13T03:30:15.003597+0000 mon.a (mon.0) 1824 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-02-13T03:30:16.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:16 smithi183 bash[20824]: audit 2024-02-13T03:30:15.109416+0000 mon.a (mon.0) 1825 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:16.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:16 smithi183 bash[20824]: audit 2024-02-13T03:30:15.111710+0000 mon.b (mon.1) 241 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:16.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:16 smithi183 bash[20824]: cluster 2024-02-13T03:30:15.305061+0000 mgr.y (mgr.24473) 374 : cluster [DBG] pgmap v464: 228 pgs: 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:16.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:16 smithi183 bash[20824]: audit 2024-02-13T03:30:15.578300+0000 mon.a (mon.0) 1826 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi161-61726-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi161-61726-52"}]': finished 2024-02-13T03:30:16.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:16 smithi183 bash[20824]: audit 2024-02-13T03:30:15.578691+0000 mon.a (mon.0) 1827 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:30:16.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:16 smithi183 bash[20824]: cluster 2024-02-13T03:30:15.578786+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-02-13T03:30:16.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:16 smithi183 bash[20824]: audit 2024-02-13T03:30:15.580182+0000 mon.a (mon.0) 1829 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37"}]: dispatch 2024-02-13T03:30:16.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:16 smithi183 bash[20824]: audit 2024-02-13T03:30:15.582749+0000 mon.b (mon.1) 242 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37"}]: dispatch 2024-02-13T03:30:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[27879]: audit 2024-02-13T03:30:15.003517+0000 mon.a (mon.0) 1823 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-37", "mode": "writeback"}]': finished 2024-02-13T03:30:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[27879]: cluster 2024-02-13T03:30:15.003597+0000 mon.a (mon.0) 1824 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-02-13T03:30:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[27879]: audit 2024-02-13T03:30:15.109416+0000 mon.a (mon.0) 1825 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[27879]: audit 2024-02-13T03:30:15.111710+0000 mon.b (mon.1) 241 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[27879]: cluster 2024-02-13T03:30:15.305061+0000 mgr.y (mgr.24473) 374 : cluster [DBG] pgmap v464: 228 pgs: 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[27879]: audit 2024-02-13T03:30:15.578300+0000 mon.a (mon.0) 1826 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi161-61726-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi161-61726-52"}]': finished 2024-02-13T03:30:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[27879]: audit 2024-02-13T03:30:15.578691+0000 mon.a (mon.0) 1827 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:30:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[27879]: cluster 2024-02-13T03:30:15.578786+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-02-13T03:30:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[27879]: audit 2024-02-13T03:30:15.580182+0000 mon.a (mon.0) 1829 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37"}]: dispatch 2024-02-13T03:30:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[27879]: audit 2024-02-13T03:30:15.582749+0000 mon.b (mon.1) 242 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37"}]: dispatch 2024-02-13T03:30:16.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[19769]: audit 2024-02-13T03:30:15.003517+0000 mon.a (mon.0) 1823 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-37", "mode": "writeback"}]': finished 2024-02-13T03:30:16.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[19769]: cluster 2024-02-13T03:30:15.003597+0000 mon.a (mon.0) 1824 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-02-13T03:30:16.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[19769]: audit 2024-02-13T03:30:15.109416+0000 mon.a (mon.0) 1825 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:16.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[19769]: audit 2024-02-13T03:30:15.111710+0000 mon.b (mon.1) 241 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:16.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[19769]: cluster 2024-02-13T03:30:15.305061+0000 mgr.y (mgr.24473) 374 : cluster [DBG] pgmap v464: 228 pgs: 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:16.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[19769]: audit 2024-02-13T03:30:15.578300+0000 mon.a (mon.0) 1826 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi161-61726-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi161-61726-52"}]': finished 2024-02-13T03:30:16.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[19769]: audit 2024-02-13T03:30:15.578691+0000 mon.a (mon.0) 1827 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:30:16.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[19769]: cluster 2024-02-13T03:30:15.578786+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-02-13T03:30:16.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[19769]: audit 2024-02-13T03:30:15.580182+0000 mon.a (mon.0) 1829 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37"}]: dispatch 2024-02-13T03:30:16.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:16 smithi161 bash[19769]: audit 2024-02-13T03:30:15.582749+0000 mon.b (mon.1) 242 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37"}]: dispatch 2024-02-13T03:30:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:17 smithi161 bash[27879]: audit 2024-02-13T03:30:15.975052+0000 mgr.y (mgr.24473) 375 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:17 smithi161 bash[27879]: cluster 2024-02-13T03:30:16.572425+0000 mon.a (mon.0) 1830 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:30:17.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:17 smithi161 bash[27879]: audit 2024-02-13T03:30:16.582799+0000 mon.a (mon.0) 1831 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37"}]': finished 2024-02-13T03:30:17.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:17 smithi161 bash[27879]: cluster 2024-02-13T03:30:16.585494+0000 mon.a (mon.0) 1832 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-02-13T03:30:17.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:17 smithi161 bash[19769]: audit 2024-02-13T03:30:15.975052+0000 mgr.y (mgr.24473) 375 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:17.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:17 smithi161 bash[19769]: cluster 2024-02-13T03:30:16.572425+0000 mon.a (mon.0) 1830 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:30:17.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:17 smithi161 bash[19769]: audit 2024-02-13T03:30:16.582799+0000 mon.a (mon.0) 1831 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37"}]': finished 2024-02-13T03:30:17.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:17 smithi161 bash[19769]: cluster 2024-02-13T03:30:16.585494+0000 mon.a (mon.0) 1832 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-02-13T03:30:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:17 smithi183 bash[20824]: audit 2024-02-13T03:30:15.975052+0000 mgr.y (mgr.24473) 375 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:17 smithi183 bash[20824]: cluster 2024-02-13T03:30:16.572425+0000 mon.a (mon.0) 1830 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:30:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:17 smithi183 bash[20824]: audit 2024-02-13T03:30:16.582799+0000 mon.a (mon.0) 1831 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-37"}]': finished 2024-02-13T03:30:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:17 smithi183 bash[20824]: cluster 2024-02-13T03:30:16.585494+0000 mon.a (mon.0) 1832 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-02-13T03:30:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:18 smithi161 bash[27879]: cluster 2024-02-13T03:30:17.305950+0000 mgr.y (mgr.24473) 376 : cluster [DBG] pgmap v467: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 74 op/s 2024-02-13T03:30:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:18 smithi161 bash[27879]: cluster 2024-02-13T03:30:17.583621+0000 mon.a (mon.0) 1833 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-02-13T03:30:18.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:18 smithi161 bash[27879]: audit 2024-02-13T03:30:17.585395+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:18.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:18 smithi161 bash[19769]: cluster 2024-02-13T03:30:17.305950+0000 mgr.y (mgr.24473) 376 : cluster [DBG] pgmap v467: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 74 op/s 2024-02-13T03:30:18.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:18 smithi161 bash[19769]: cluster 2024-02-13T03:30:17.583621+0000 mon.a (mon.0) 1833 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-02-13T03:30:18.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:18 smithi161 bash[19769]: audit 2024-02-13T03:30:17.585395+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:18.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:18 smithi183 bash[20824]: cluster 2024-02-13T03:30:17.305950+0000 mgr.y (mgr.24473) 376 : cluster [DBG] pgmap v467: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 74 op/s 2024-02-13T03:30:18.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:18 smithi183 bash[20824]: cluster 2024-02-13T03:30:17.583621+0000 mon.a (mon.0) 1833 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-02-13T03:30:18.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:18 smithi183 bash[20824]: audit 2024-02-13T03:30:17.585395+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:19.598 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:30:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:30:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:30:19.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:19 smithi161 bash[19769]: audit 2024-02-13T03:30:18.585492+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52"}]': finished 2024-02-13T03:30:19.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:19 smithi161 bash[19769]: cluster 2024-02-13T03:30:18.586088+0000 mon.a (mon.0) 1836 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-02-13T03:30:19.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:19 smithi161 bash[19769]: audit 2024-02-13T03:30:18.587497+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:19.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:19 smithi161 bash[19769]: audit 2024-02-13T03:30:18.592585+0000 mon.a (mon.0) 1838 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:19.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:19 smithi161 bash[19769]: audit 2024-02-13T03:30:18.594944+0000 mon.b (mon.1) 243 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:19.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:19 smithi161 bash[19769]: cluster 2024-02-13T03:30:19.306825+0000 mgr.y (mgr.24473) 377 : cluster [DBG] pgmap v470: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-13T03:30:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:19 smithi161 bash[27879]: audit 2024-02-13T03:30:18.585492+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52"}]': finished 2024-02-13T03:30:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:19 smithi161 bash[27879]: cluster 2024-02-13T03:30:18.586088+0000 mon.a (mon.0) 1836 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-02-13T03:30:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:19 smithi161 bash[27879]: audit 2024-02-13T03:30:18.587497+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:19 smithi161 bash[27879]: audit 2024-02-13T03:30:18.592585+0000 mon.a (mon.0) 1838 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:19 smithi161 bash[27879]: audit 2024-02-13T03:30:18.594944+0000 mon.b (mon.1) 243 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:19 smithi161 bash[27879]: cluster 2024-02-13T03:30:19.306825+0000 mgr.y (mgr.24473) 377 : cluster [DBG] pgmap v470: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-13T03:30:19.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:19 smithi183 bash[20824]: audit 2024-02-13T03:30:18.585492+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi161-61726-52"}]': finished 2024-02-13T03:30:19.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:19 smithi183 bash[20824]: cluster 2024-02-13T03:30:18.586088+0000 mon.a (mon.0) 1836 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-02-13T03:30:19.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:19 smithi183 bash[20824]: audit 2024-02-13T03:30:18.587497+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi161-61726-52"}]: dispatch 2024-02-13T03:30:19.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:19 smithi183 bash[20824]: audit 2024-02-13T03:30:18.592585+0000 mon.a (mon.0) 1838 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:19.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:19 smithi183 bash[20824]: audit 2024-02-13T03:30:18.594944+0000 mon.b (mon.1) 243 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:19.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:19 smithi183 bash[20824]: cluster 2024-02-13T03:30:19.306825+0000 mgr.y (mgr.24473) 377 : cluster [DBG] pgmap v470: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-13T03:30:20.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[27879]: audit 2024-02-13T03:30:19.601090+0000 mon.a (mon.0) 1839 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi161-61726-52"}]': finished 2024-02-13T03:30:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[27879]: audit 2024-02-13T03:30:19.601385+0000 mon.a (mon.0) 1840 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[27879]: cluster 2024-02-13T03:30:19.601643+0000 mon.a (mon.0) 1841 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-02-13T03:30:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[27879]: audit 2024-02-13T03:30:19.606279+0000 mon.a (mon.0) 1842 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[27879]: audit 2024-02-13T03:30:19.606482+0000 mon.b (mon.1) 244 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[27879]: audit 2024-02-13T03:30:19.632590+0000 mon.c (mon.2) 456 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[27879]: audit 2024-02-13T03:30:19.632923+0000 mon.a (mon.0) 1843 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[27879]: audit 2024-02-13T03:30:19.633814+0000 mon.c (mon.2) 457 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[27879]: audit 2024-02-13T03:30:19.634110+0000 mon.a (mon.0) 1844 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[27879]: audit 2024-02-13T03:30:19.634557+0000 mon.c (mon.2) 458 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi161-61726-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:20.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[27879]: audit 2024-02-13T03:30:19.634815+0000 mon.a (mon.0) 1845 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi161-61726-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[19769]: audit 2024-02-13T03:30:19.601090+0000 mon.a (mon.0) 1839 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi161-61726-52"}]': finished 2024-02-13T03:30:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[19769]: audit 2024-02-13T03:30:19.601385+0000 mon.a (mon.0) 1840 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[19769]: cluster 2024-02-13T03:30:19.601643+0000 mon.a (mon.0) 1841 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-02-13T03:30:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[19769]: audit 2024-02-13T03:30:19.606279+0000 mon.a (mon.0) 1842 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[19769]: audit 2024-02-13T03:30:19.606482+0000 mon.b (mon.1) 244 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[19769]: audit 2024-02-13T03:30:19.632590+0000 mon.c (mon.2) 456 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[19769]: audit 2024-02-13T03:30:19.632923+0000 mon.a (mon.0) 1843 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[19769]: audit 2024-02-13T03:30:19.633814+0000 mon.c (mon.2) 457 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[19769]: audit 2024-02-13T03:30:19.634110+0000 mon.a (mon.0) 1844 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:20.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[19769]: audit 2024-02-13T03:30:19.634557+0000 mon.c (mon.2) 458 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi161-61726-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:20.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:20 smithi161 bash[19769]: audit 2024-02-13T03:30:19.634815+0000 mon.a (mon.0) 1845 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi161-61726-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:20.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:20 smithi183 bash[20824]: audit 2024-02-13T03:30:19.601090+0000 mon.a (mon.0) 1839 : audit [INF] from='client.? v1:172.21.15.161:0/2167120279' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi161-61726-52"}]': finished 2024-02-13T03:30:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:20 smithi183 bash[20824]: audit 2024-02-13T03:30:19.601385+0000 mon.a (mon.0) 1840 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:20 smithi183 bash[20824]: cluster 2024-02-13T03:30:19.601643+0000 mon.a (mon.0) 1841 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-02-13T03:30:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:20 smithi183 bash[20824]: audit 2024-02-13T03:30:19.606279+0000 mon.a (mon.0) 1842 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:20 smithi183 bash[20824]: audit 2024-02-13T03:30:19.606482+0000 mon.b (mon.1) 244 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:20 smithi183 bash[20824]: audit 2024-02-13T03:30:19.632590+0000 mon.c (mon.2) 456 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:20 smithi183 bash[20824]: audit 2024-02-13T03:30:19.632923+0000 mon.a (mon.0) 1843 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:20 smithi183 bash[20824]: audit 2024-02-13T03:30:19.633814+0000 mon.c (mon.2) 457 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:20 smithi183 bash[20824]: audit 2024-02-13T03:30:19.634110+0000 mon.a (mon.0) 1844 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:20 smithi183 bash[20824]: audit 2024-02-13T03:30:19.634557+0000 mon.c (mon.2) 458 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi161-61726-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:20.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:20 smithi183 bash[20824]: audit 2024-02-13T03:30:19.634815+0000 mon.a (mon.0) 1845 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi161-61726-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:21.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:30:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:30:21] "GET /metrics HTTP/1.1" 200 186360 "" "Prometheus/2.33.4" 2024-02-13T03:30:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[27879]: audit 2024-02-13T03:30:20.601416+0000 mon.a (mon.0) 1846 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[27879]: audit 2024-02-13T03:30:20.601691+0000 mon.a (mon.0) 1847 : audit [INF] from='client.48299 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi161-61726-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[27879]: cluster 2024-02-13T03:30:20.601960+0000 mon.a (mon.0) 1848 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-02-13T03:30:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[27879]: audit 2024-02-13T03:30:20.604362+0000 mon.a (mon.0) 1849 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:30:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[27879]: audit 2024-02-13T03:30:20.605132+0000 mon.c (mon.2) 459 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi161-61726-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[27879]: audit 2024-02-13T03:30:20.605634+0000 mon.b (mon.1) 245 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:30:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[27879]: audit 2024-02-13T03:30:20.610658+0000 mon.a (mon.0) 1850 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi161-61726-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[27879]: cluster 2024-02-13T03:30:21.308931+0000 mgr.y (mgr.24473) 378 : cluster [DBG] pgmap v473: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[27879]: audit 2024-02-13T03:30:21.604625+0000 mon.a (mon.0) 1851 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:30:22.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[19769]: audit 2024-02-13T03:30:20.601416+0000 mon.a (mon.0) 1846 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:22.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[19769]: audit 2024-02-13T03:30:20.601691+0000 mon.a (mon.0) 1847 : audit [INF] from='client.48299 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi161-61726-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:22.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[19769]: cluster 2024-02-13T03:30:20.601960+0000 mon.a (mon.0) 1848 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-02-13T03:30:22.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[19769]: audit 2024-02-13T03:30:20.604362+0000 mon.a (mon.0) 1849 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:30:22.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[19769]: audit 2024-02-13T03:30:20.605132+0000 mon.c (mon.2) 459 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi161-61726-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:22.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[19769]: audit 2024-02-13T03:30:20.605634+0000 mon.b (mon.1) 245 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:30:22.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[19769]: audit 2024-02-13T03:30:20.610658+0000 mon.a (mon.0) 1850 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi161-61726-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:22.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[19769]: cluster 2024-02-13T03:30:21.308931+0000 mgr.y (mgr.24473) 378 : cluster [DBG] pgmap v473: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:22.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:22 smithi161 bash[19769]: audit 2024-02-13T03:30:21.604625+0000 mon.a (mon.0) 1851 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:30:22.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:22 smithi183 bash[20824]: audit 2024-02-13T03:30:20.601416+0000 mon.a (mon.0) 1846 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:22.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:22 smithi183 bash[20824]: audit 2024-02-13T03:30:20.601691+0000 mon.a (mon.0) 1847 : audit [INF] from='client.48299 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi161-61726-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:22.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:22 smithi183 bash[20824]: cluster 2024-02-13T03:30:20.601960+0000 mon.a (mon.0) 1848 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-02-13T03:30:22.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:22 smithi183 bash[20824]: audit 2024-02-13T03:30:20.604362+0000 mon.a (mon.0) 1849 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:30:22.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:22 smithi183 bash[20824]: audit 2024-02-13T03:30:20.605132+0000 mon.c (mon.2) 459 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi161-61726-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:22.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:22 smithi183 bash[20824]: audit 2024-02-13T03:30:20.605634+0000 mon.b (mon.1) 245 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:30:22.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:22 smithi183 bash[20824]: audit 2024-02-13T03:30:20.610658+0000 mon.a (mon.0) 1850 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi161-61726-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:22.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:22 smithi183 bash[20824]: cluster 2024-02-13T03:30:21.308931+0000 mgr.y (mgr.24473) 378 : cluster [DBG] pgmap v473: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:22.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:22 smithi183 bash[20824]: audit 2024-02-13T03:30:21.604625+0000 mon.a (mon.0) 1851 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:30:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[19769]: audit 2024-02-13T03:30:21.614057+0000 mon.b (mon.1) 246 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:30:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[19769]: cluster 2024-02-13T03:30:22.030675+0000 mon.a (mon.0) 1852 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-02-13T03:30:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[19769]: audit 2024-02-13T03:30:22.032565+0000 mon.a (mon.0) 1853 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:30:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[19769]: audit 2024-02-13T03:30:22.606795+0000 mon.a (mon.0) 1854 : audit [INF] from='client.48299 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi161-61726-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi161-61726-53"}]': finished 2024-02-13T03:30:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[19769]: audit 2024-02-13T03:30:22.606953+0000 mon.a (mon.0) 1855 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:30:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[19769]: cluster 2024-02-13T03:30:22.607044+0000 mon.a (mon.0) 1856 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-02-13T03:30:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[19769]: audit 2024-02-13T03:30:22.611140+0000 mon.a (mon.0) 1857 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:30:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[19769]: audit 2024-02-13T03:30:22.612849+0000 mon.b (mon.1) 247 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:30:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[27879]: audit 2024-02-13T03:30:21.614057+0000 mon.b (mon.1) 246 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:30:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[27879]: cluster 2024-02-13T03:30:22.030675+0000 mon.a (mon.0) 1852 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-02-13T03:30:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[27879]: audit 2024-02-13T03:30:22.032565+0000 mon.a (mon.0) 1853 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:30:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[27879]: audit 2024-02-13T03:30:22.606795+0000 mon.a (mon.0) 1854 : audit [INF] from='client.48299 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi161-61726-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi161-61726-53"}]': finished 2024-02-13T03:30:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[27879]: audit 2024-02-13T03:30:22.606953+0000 mon.a (mon.0) 1855 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:30:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[27879]: cluster 2024-02-13T03:30:22.607044+0000 mon.a (mon.0) 1856 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-02-13T03:30:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[27879]: audit 2024-02-13T03:30:22.611140+0000 mon.a (mon.0) 1857 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:30:23.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:23 smithi161 bash[27879]: audit 2024-02-13T03:30:22.612849+0000 mon.b (mon.1) 247 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:30:23.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:23 smithi183 bash[20824]: audit 2024-02-13T03:30:21.614057+0000 mon.b (mon.1) 246 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:30:23.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:23 smithi183 bash[20824]: cluster 2024-02-13T03:30:22.030675+0000 mon.a (mon.0) 1852 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-02-13T03:30:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:23 smithi183 bash[20824]: audit 2024-02-13T03:30:22.032565+0000 mon.a (mon.0) 1853 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:30:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:23 smithi183 bash[20824]: audit 2024-02-13T03:30:22.606795+0000 mon.a (mon.0) 1854 : audit [INF] from='client.48299 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi161-61726-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi161-61726-53"}]': finished 2024-02-13T03:30:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:23 smithi183 bash[20824]: audit 2024-02-13T03:30:22.606953+0000 mon.a (mon.0) 1855 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:30:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:23 smithi183 bash[20824]: cluster 2024-02-13T03:30:22.607044+0000 mon.a (mon.0) 1856 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-02-13T03:30:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:23 smithi183 bash[20824]: audit 2024-02-13T03:30:22.611140+0000 mon.a (mon.0) 1857 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:30:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:23 smithi183 bash[20824]: audit 2024-02-13T03:30:22.612849+0000 mon.b (mon.1) 247 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:30:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:24 smithi161 bash[27879]: cluster 2024-02-13T03:30:23.309794+0000 mgr.y (mgr.24473) 379 : cluster [DBG] pgmap v476: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:24 smithi161 bash[27879]: audit 2024-02-13T03:30:23.610375+0000 mon.a (mon.0) 1858 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-13T03:30:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:24 smithi161 bash[27879]: cluster 2024-02-13T03:30:23.610555+0000 mon.a (mon.0) 1859 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-02-13T03:30:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:24 smithi161 bash[19769]: cluster 2024-02-13T03:30:23.309794+0000 mgr.y (mgr.24473) 379 : cluster [DBG] pgmap v476: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:24 smithi161 bash[19769]: audit 2024-02-13T03:30:23.610375+0000 mon.a (mon.0) 1858 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-13T03:30:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:24 smithi161 bash[19769]: cluster 2024-02-13T03:30:23.610555+0000 mon.a (mon.0) 1859 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-02-13T03:30:24.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:24 smithi183 bash[20824]: cluster 2024-02-13T03:30:23.309794+0000 mgr.y (mgr.24473) 379 : cluster [DBG] pgmap v476: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:24.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:24 smithi183 bash[20824]: audit 2024-02-13T03:30:23.610375+0000 mon.a (mon.0) 1858 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-13T03:30:24.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:24 smithi183 bash[20824]: cluster 2024-02-13T03:30:23.610555+0000 mon.a (mon.0) 1859 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-02-13T03:30:25.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[27879]: cluster 2024-02-13T03:30:24.612186+0000 mon.a (mon.0) 1860 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-02-13T03:30:25.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[27879]: audit 2024-02-13T03:30:24.614580+0000 mon.c (mon.2) 460 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[27879]: audit 2024-02-13T03:30:24.618602+0000 mon.a (mon.0) 1861 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[27879]: audit 2024-02-13T03:30:24.635649+0000 mon.a (mon.0) 1862 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[27879]: audit 2024-02-13T03:30:24.636603+0000 mon.a (mon.0) 1863 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39"}]: dispatch 2024-02-13T03:30:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[27879]: audit 2024-02-13T03:30:24.638172+0000 mon.b (mon.1) 248 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[27879]: audit 2024-02-13T03:30:24.639303+0000 mon.b (mon.1) 249 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39"}]: dispatch 2024-02-13T03:30:25.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[27879]: cluster 2024-02-13T03:30:25.310719+0000 mgr.y (mgr.24473) 380 : cluster [DBG] pgmap v479: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[19769]: cluster 2024-02-13T03:30:24.612186+0000 mon.a (mon.0) 1860 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-02-13T03:30:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[19769]: audit 2024-02-13T03:30:24.614580+0000 mon.c (mon.2) 460 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[19769]: audit 2024-02-13T03:30:24.618602+0000 mon.a (mon.0) 1861 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[19769]: audit 2024-02-13T03:30:24.635649+0000 mon.a (mon.0) 1862 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[19769]: audit 2024-02-13T03:30:24.636603+0000 mon.a (mon.0) 1863 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39"}]: dispatch 2024-02-13T03:30:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[19769]: audit 2024-02-13T03:30:24.638172+0000 mon.b (mon.1) 248 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:25.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[19769]: audit 2024-02-13T03:30:24.639303+0000 mon.b (mon.1) 249 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39"}]: dispatch 2024-02-13T03:30:25.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:25 smithi161 bash[19769]: cluster 2024-02-13T03:30:25.310719+0000 mgr.y (mgr.24473) 380 : cluster [DBG] pgmap v479: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:25 smithi183 bash[20824]: cluster 2024-02-13T03:30:24.612186+0000 mon.a (mon.0) 1860 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-02-13T03:30:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:25 smithi183 bash[20824]: audit 2024-02-13T03:30:24.614580+0000 mon.c (mon.2) 460 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:25 smithi183 bash[20824]: audit 2024-02-13T03:30:24.618602+0000 mon.a (mon.0) 1861 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:25.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:25 smithi183 bash[20824]: audit 2024-02-13T03:30:24.635649+0000 mon.a (mon.0) 1862 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:25.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:25 smithi183 bash[20824]: audit 2024-02-13T03:30:24.636603+0000 mon.a (mon.0) 1863 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39"}]: dispatch 2024-02-13T03:30:25.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:25 smithi183 bash[20824]: audit 2024-02-13T03:30:24.638172+0000 mon.b (mon.1) 248 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:25.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:25 smithi183 bash[20824]: audit 2024-02-13T03:30:24.639303+0000 mon.b (mon.1) 249 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39"}]: dispatch 2024-02-13T03:30:25.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:25 smithi183 bash[20824]: cluster 2024-02-13T03:30:25.310719+0000 mgr.y (mgr.24473) 380 : cluster [DBG] pgmap v479: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:26.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:30:25 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:30:26.633 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [==========] Running 72 tests from 4 test suites. 2024-02-13T03:30:26.633 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-02-13T03:30:26.633 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-02-13T03:30:26.633 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: seed 61906 2024-02-13T03:30:26.633 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-02-13T03:30:26.634 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (272 ms) 2024-02-13T03:30:26.634 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-02-13T03:30:26.634 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (14561 ms) 2024-02-13T03:30:26.634 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-02-13T03:30:26.634 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (19 ms) 2024-02-13T03:30:26.634 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (14853 ms total) 2024-02-13T03:30:26.634 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: 2024-02-13T03:30:26.634 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [----------] 43 tests from LibRadosTwoPoolsPP 2024-02-13T03:30:26.634 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-02-13T03:30:26.634 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (8481 ms) 2024-02-13T03:30:26.634 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-02-13T03:30:26.634 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (9181 ms) 2024-02-13T03:30:26.634 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-02-13T03:30:26.635 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10466 ms) 2024-02-13T03:30:26.635 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-02-13T03:30:26.635 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: my_snaps [4] 2024-02-13T03:30:26.635 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: my_snaps [5,4] 2024-02-13T03:30:26.635 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: my_snaps [6,5,4] 2024-02-13T03:30:26.635 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: my_snaps [7,6,5,4] 2024-02-13T03:30:26.635 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: promoting some heads 2024-02-13T03:30:26.635 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: promoting from clones for snap 7 2024-02-13T03:30:26.635 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: promoting from clones for snap 6 2024-02-13T03:30:26.635 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: promoting from clones for snap 5 2024-02-13T03:30:26.635 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: promoting from clones for snap 4 2024-02-13T03:30:26.635 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: waiting for scrubs... 2024-02-13T03:30:26.635 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: done waiting 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (45834 ms) 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (10015 ms) 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7999 ms) 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8074 ms) 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8454 ms) 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (9647 ms) 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8867 ms) 2024-02-13T03:30:26.636 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10045 ms) 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7955 ms) 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8036 ms) 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13056 ms) 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7919 ms) 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8021 ms) 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-13T03:30:26.637 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: ok, hit_set contains 188:602f83fe:::foo:head 2024-02-13T03:30:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:26 smithi161 bash[27879]: audit 2024-02-13T03:30:25.615174+0000 mon.a (mon.0) 1864 : audit [INF] from='client.48299 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]': finished 2024-02-13T03:30:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:26 smithi161 bash[27879]: audit 2024-02-13T03:30:25.615327+0000 mon.a (mon.0) 1865 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39"}]': finished 2024-02-13T03:30:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:26 smithi161 bash[27879]: cluster 2024-02-13T03:30:25.615473+0000 mon.a (mon.0) 1866 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-02-13T03:30:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:26 smithi161 bash[27879]: audit 2024-02-13T03:30:25.619151+0000 mon.c (mon.2) 461 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:26 smithi161 bash[27879]: audit 2024-02-13T03:30:25.625614+0000 mon.a (mon.0) 1867 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:26 smithi161 bash[27879]: audit 2024-02-13T03:30:25.986417+0000 mgr.y (mgr.24473) 381 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:26 smithi161 bash[19769]: audit 2024-02-13T03:30:25.615174+0000 mon.a (mon.0) 1864 : audit [INF] from='client.48299 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]': finished 2024-02-13T03:30:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:26 smithi161 bash[19769]: audit 2024-02-13T03:30:25.615327+0000 mon.a (mon.0) 1865 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39"}]': finished 2024-02-13T03:30:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:26 smithi161 bash[19769]: cluster 2024-02-13T03:30:25.615473+0000 mon.a (mon.0) 1866 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-02-13T03:30:26.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:26 smithi161 bash[19769]: audit 2024-02-13T03:30:25.619151+0000 mon.c (mon.2) 461 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:26.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:26 smithi161 bash[19769]: audit 2024-02-13T03:30:25.625614+0000 mon.a (mon.0) 1867 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:26.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:26 smithi161 bash[19769]: audit 2024-02-13T03:30:25.986417+0000 mgr.y (mgr.24473) 381 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:26.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:26 smithi183 bash[20824]: audit 2024-02-13T03:30:25.615174+0000 mon.a (mon.0) 1864 : audit [INF] from='client.48299 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi161-61726-53"}]': finished 2024-02-13T03:30:26.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:26 smithi183 bash[20824]: audit 2024-02-13T03:30:25.615327+0000 mon.a (mon.0) 1865 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-39"}]': finished 2024-02-13T03:30:26.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:26 smithi183 bash[20824]: cluster 2024-02-13T03:30:25.615473+0000 mon.a (mon.0) 1866 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-02-13T03:30:26.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:26 smithi183 bash[20824]: audit 2024-02-13T03:30:25.619151+0000 mon.c (mon.2) 461 : audit [INF] from='client.? v1:172.21.15.161:0/569508245' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:26.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:26 smithi183 bash[20824]: audit 2024-02-13T03:30:25.625614+0000 mon.a (mon.0) 1867 : audit [INF] from='client.48299 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]: dispatch 2024-02-13T03:30:26.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:26 smithi183 bash[20824]: audit 2024-02-13T03:30:25.986417+0000 mgr.y (mgr.24473) 381 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:27.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[27879]: audit 2024-02-13T03:30:26.631138+0000 mon.a (mon.0) 1868 : audit [INF] from='client.48299 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]': finished 2024-02-13T03:30:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[27879]: cluster 2024-02-13T03:30:26.631260+0000 mon.a (mon.0) 1869 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-02-13T03:30:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[27879]: audit 2024-02-13T03:30:26.662643+0000 mon.c (mon.2) 462 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[27879]: audit 2024-02-13T03:30:27.069032+0000 mon.a (mon.0) 1870 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[27879]: audit 2024-02-13T03:30:27.070111+0000 mon.c (mon.2) 463 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[27879]: audit 2024-02-13T03:30:27.070761+0000 mon.a (mon.0) 1871 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[27879]: audit 2024-02-13T03:30:27.071733+0000 mon.c (mon.2) 464 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi161-61726-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[27879]: audit 2024-02-13T03:30:27.072375+0000 mon.a (mon.0) 1872 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi161-61726-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[27879]: cluster 2024-02-13T03:30:27.311586+0000 mgr.y (mgr.24473) 382 : cluster [DBG] pgmap v482: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:27.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[19769]: audit 2024-02-13T03:30:26.631138+0000 mon.a (mon.0) 1868 : audit [INF] from='client.48299 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]': finished 2024-02-13T03:30:27.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[19769]: cluster 2024-02-13T03:30:26.631260+0000 mon.a (mon.0) 1869 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-02-13T03:30:27.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[19769]: audit 2024-02-13T03:30:26.662643+0000 mon.c (mon.2) 462 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:27.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[19769]: audit 2024-02-13T03:30:27.069032+0000 mon.a (mon.0) 1870 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:27.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[19769]: audit 2024-02-13T03:30:27.070111+0000 mon.c (mon.2) 463 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:27.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[19769]: audit 2024-02-13T03:30:27.070761+0000 mon.a (mon.0) 1871 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:27.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[19769]: audit 2024-02-13T03:30:27.071733+0000 mon.c (mon.2) 464 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi161-61726-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:27.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[19769]: audit 2024-02-13T03:30:27.072375+0000 mon.a (mon.0) 1872 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi161-61726-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:27.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:27 smithi161 bash[19769]: cluster 2024-02-13T03:30:27.311586+0000 mgr.y (mgr.24473) 382 : cluster [DBG] pgmap v482: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:27.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:27 smithi183 bash[20824]: audit 2024-02-13T03:30:26.631138+0000 mon.a (mon.0) 1868 : audit [INF] from='client.48299 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi161-61726-53"}]': finished 2024-02-13T03:30:27.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:27 smithi183 bash[20824]: cluster 2024-02-13T03:30:26.631260+0000 mon.a (mon.0) 1869 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-02-13T03:30:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:27 smithi183 bash[20824]: audit 2024-02-13T03:30:26.662643+0000 mon.c (mon.2) 462 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:27 smithi183 bash[20824]: audit 2024-02-13T03:30:27.069032+0000 mon.a (mon.0) 1870 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:27 smithi183 bash[20824]: audit 2024-02-13T03:30:27.070111+0000 mon.c (mon.2) 463 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:27 smithi183 bash[20824]: audit 2024-02-13T03:30:27.070761+0000 mon.a (mon.0) 1871 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:27 smithi183 bash[20824]: audit 2024-02-13T03:30:27.071733+0000 mon.c (mon.2) 464 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi161-61726-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:27 smithi183 bash[20824]: audit 2024-02-13T03:30:27.072375+0000 mon.a (mon.0) 1872 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi161-61726-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:27.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:27 smithi183 bash[20824]: cluster 2024-02-13T03:30:27.311586+0000 mgr.y (mgr.24473) 382 : cluster [DBG] pgmap v482: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:29.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:29 smithi161 bash[27879]: audit 2024-02-13T03:30:27.652452+0000 mon.a (mon.0) 1873 : audit [INF] from='client.48305 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi161-61726-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:29.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:29 smithi161 bash[27879]: cluster 2024-02-13T03:30:27.652790+0000 mon.a (mon.0) 1874 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-02-13T03:30:29.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:29 smithi161 bash[27879]: audit 2024-02-13T03:30:27.653684+0000 mon.c (mon.2) 465 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi161-61726-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:29.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:29 smithi161 bash[27879]: audit 2024-02-13T03:30:27.654200+0000 mon.a (mon.0) 1875 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi161-61726-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:29.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:29 smithi161 bash[27879]: audit 2024-02-13T03:30:27.659581+0000 mon.a (mon.0) 1876 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:29.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:29 smithi161 bash[27879]: audit 2024-02-13T03:30:27.661710+0000 mon.b (mon.1) 250 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:29.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:29 smithi161 bash[19769]: audit 2024-02-13T03:30:27.652452+0000 mon.a (mon.0) 1873 : audit [INF] from='client.48305 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi161-61726-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:29.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:29 smithi161 bash[19769]: cluster 2024-02-13T03:30:27.652790+0000 mon.a (mon.0) 1874 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-02-13T03:30:29.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:29 smithi161 bash[19769]: audit 2024-02-13T03:30:27.653684+0000 mon.c (mon.2) 465 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi161-61726-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:29.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:29 smithi161 bash[19769]: audit 2024-02-13T03:30:27.654200+0000 mon.a (mon.0) 1875 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi161-61726-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:29.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:29 smithi161 bash[19769]: audit 2024-02-13T03:30:27.659581+0000 mon.a (mon.0) 1876 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:29.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:29 smithi161 bash[19769]: audit 2024-02-13T03:30:27.661710+0000 mon.b (mon.1) 250 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:29.486 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:30:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:30:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:30:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:29 smithi183 bash[20824]: audit 2024-02-13T03:30:27.652452+0000 mon.a (mon.0) 1873 : audit [INF] from='client.48305 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi161-61726-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:29 smithi183 bash[20824]: cluster 2024-02-13T03:30:27.652790+0000 mon.a (mon.0) 1874 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-02-13T03:30:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:29 smithi183 bash[20824]: audit 2024-02-13T03:30:27.653684+0000 mon.c (mon.2) 465 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi161-61726-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:29 smithi183 bash[20824]: audit 2024-02-13T03:30:27.654200+0000 mon.a (mon.0) 1875 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi161-61726-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:29 smithi183 bash[20824]: audit 2024-02-13T03:30:27.659581+0000 mon.a (mon.0) 1876 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:29 smithi183 bash[20824]: audit 2024-02-13T03:30:27.661710+0000 mon.b (mon.1) 250 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:30.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[19769]: audit 2024-02-13T03:30:29.085035+0000 mon.a (mon.0) 1877 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:30.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[19769]: cluster 2024-02-13T03:30:29.085388+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-02-13T03:30:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[19769]: audit 2024-02-13T03:30:29.090236+0000 mon.b (mon.1) 251 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi161-61906-6","var": "pg_num","format": "json"}]: dispatch 2024-02-13T03:30:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[19769]: audit 2024-02-13T03:30:29.095556+0000 mon.a (mon.0) 1879 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[19769]: audit 2024-02-13T03:30:29.097959+0000 mon.b (mon.1) 252 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[19769]: cluster 2024-02-13T03:30:29.312381+0000 mgr.y (mgr.24473) 383 : cluster [DBG] pgmap v485: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[19769]: audit 2024-02-13T03:30:29.659751+0000 mon.a (mon.0) 1880 : audit [INF] from='client.48305 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi161-61726-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi161-61726-54"}]': finished 2024-02-13T03:30:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[19769]: audit 2024-02-13T03:30:29.659960+0000 mon.a (mon.0) 1881 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[19769]: cluster 2024-02-13T03:30:29.660093+0000 mon.a (mon.0) 1882 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-02-13T03:30:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[19769]: audit 2024-02-13T03:30:29.665665+0000 mon.b (mon.1) 253 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-13T03:30:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[19769]: audit 2024-02-13T03:30:29.670103+0000 mon.a (mon.0) 1883 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-13T03:30:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[27879]: audit 2024-02-13T03:30:29.085035+0000 mon.a (mon.0) 1877 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[27879]: cluster 2024-02-13T03:30:29.085388+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-02-13T03:30:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[27879]: audit 2024-02-13T03:30:29.090236+0000 mon.b (mon.1) 251 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi161-61906-6","var": "pg_num","format": "json"}]: dispatch 2024-02-13T03:30:30.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[27879]: audit 2024-02-13T03:30:29.095556+0000 mon.a (mon.0) 1879 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:30.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[27879]: audit 2024-02-13T03:30:29.097959+0000 mon.b (mon.1) 252 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:30.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[27879]: cluster 2024-02-13T03:30:29.312381+0000 mgr.y (mgr.24473) 383 : cluster [DBG] pgmap v485: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:30.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[27879]: audit 2024-02-13T03:30:29.659751+0000 mon.a (mon.0) 1880 : audit [INF] from='client.48305 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi161-61726-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi161-61726-54"}]': finished 2024-02-13T03:30:30.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[27879]: audit 2024-02-13T03:30:29.659960+0000 mon.a (mon.0) 1881 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:30.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[27879]: cluster 2024-02-13T03:30:29.660093+0000 mon.a (mon.0) 1882 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-02-13T03:30:30.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[27879]: audit 2024-02-13T03:30:29.665665+0000 mon.b (mon.1) 253 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-13T03:30:30.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:30 smithi161 bash[27879]: audit 2024-02-13T03:30:29.670103+0000 mon.a (mon.0) 1883 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-13T03:30:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:30 smithi183 bash[20824]: audit 2024-02-13T03:30:29.085035+0000 mon.a (mon.0) 1877 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:30 smithi183 bash[20824]: cluster 2024-02-13T03:30:29.085388+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-02-13T03:30:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:30 smithi183 bash[20824]: audit 2024-02-13T03:30:29.090236+0000 mon.b (mon.1) 251 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi161-61906-6","var": "pg_num","format": "json"}]: dispatch 2024-02-13T03:30:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:30 smithi183 bash[20824]: audit 2024-02-13T03:30:29.095556+0000 mon.a (mon.0) 1879 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:30 smithi183 bash[20824]: audit 2024-02-13T03:30:29.097959+0000 mon.b (mon.1) 252 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:30 smithi183 bash[20824]: cluster 2024-02-13T03:30:29.312381+0000 mgr.y (mgr.24473) 383 : cluster [DBG] pgmap v485: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:30 smithi183 bash[20824]: audit 2024-02-13T03:30:29.659751+0000 mon.a (mon.0) 1880 : audit [INF] from='client.48305 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi161-61726-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi161-61726-54"}]': finished 2024-02-13T03:30:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:30 smithi183 bash[20824]: audit 2024-02-13T03:30:29.659960+0000 mon.a (mon.0) 1881 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:30 smithi183 bash[20824]: cluster 2024-02-13T03:30:29.660093+0000 mon.a (mon.0) 1882 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-02-13T03:30:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:30 smithi183 bash[20824]: audit 2024-02-13T03:30:29.665665+0000 mon.b (mon.1) 253 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-13T03:30:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:30 smithi183 bash[20824]: audit 2024-02-13T03:30:29.670103+0000 mon.a (mon.0) 1883 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-13T03:30:31.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:30:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:30:31] "GET /metrics HTTP/1.1" 200 186360 "" "Prometheus/2.33.4" 2024-02-13T03:30:31.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:31 smithi161 bash[27879]: audit 2024-02-13T03:30:30.663066+0000 mon.a (mon.0) 1884 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_count","val": "8"}]': finished 2024-02-13T03:30:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:31 smithi161 bash[27879]: cluster 2024-02-13T03:30:30.663428+0000 mon.a (mon.0) 1885 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-02-13T03:30:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:31 smithi161 bash[27879]: audit 2024-02-13T03:30:30.667693+0000 mon.a (mon.0) 1886 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:30:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:31 smithi161 bash[27879]: audit 2024-02-13T03:30:30.668660+0000 mon.b (mon.1) 254 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:30:31.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:31 smithi161 bash[27879]: cluster 2024-02-13T03:30:31.313654+0000 mgr.y (mgr.24473) 384 : cluster [DBG] pgmap v488: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:31.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:31 smithi161 bash[19769]: audit 2024-02-13T03:30:30.663066+0000 mon.a (mon.0) 1884 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_count","val": "8"}]': finished 2024-02-13T03:30:31.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:31 smithi161 bash[19769]: cluster 2024-02-13T03:30:30.663428+0000 mon.a (mon.0) 1885 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-02-13T03:30:31.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:31 smithi161 bash[19769]: audit 2024-02-13T03:30:30.667693+0000 mon.a (mon.0) 1886 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:30:31.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:31 smithi161 bash[19769]: audit 2024-02-13T03:30:30.668660+0000 mon.b (mon.1) 254 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:30:31.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:31 smithi161 bash[19769]: cluster 2024-02-13T03:30:31.313654+0000 mgr.y (mgr.24473) 384 : cluster [DBG] pgmap v488: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:31.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:31 smithi183 bash[20824]: audit 2024-02-13T03:30:30.663066+0000 mon.a (mon.0) 1884 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_count","val": "8"}]': finished 2024-02-13T03:30:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:31 smithi183 bash[20824]: cluster 2024-02-13T03:30:30.663428+0000 mon.a (mon.0) 1885 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-02-13T03:30:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:31 smithi183 bash[20824]: audit 2024-02-13T03:30:30.667693+0000 mon.a (mon.0) 1886 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:30:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:31 smithi183 bash[20824]: audit 2024-02-13T03:30:30.668660+0000 mon.b (mon.1) 254 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:30:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:31 smithi183 bash[20824]: cluster 2024-02-13T03:30:31.313654+0000 mgr.y (mgr.24473) 384 : cluster [DBG] pgmap v488: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[27879]: audit 2024-02-13T03:30:31.663916+0000 mon.a (mon.0) 1887 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:30:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[27879]: cluster 2024-02-13T03:30:31.664167+0000 mon.a (mon.0) 1888 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-02-13T03:30:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[27879]: audit 2024-02-13T03:30:31.667829+0000 mon.a (mon.0) 1889 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-13T03:30:32.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[27879]: audit 2024-02-13T03:30:31.669018+0000 mon.c (mon.2) 466 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:32.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[27879]: audit 2024-02-13T03:30:31.670269+0000 mon.b (mon.1) 255 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-13T03:30:32.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[27879]: audit 2024-02-13T03:30:31.675126+0000 mon.a (mon.0) 1890 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:32.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[27879]: audit 2024-02-13T03:30:32.664828+0000 mon.a (mon.0) 1891 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-13T03:30:32.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[27879]: audit 2024-02-13T03:30:32.664914+0000 mon.a (mon.0) 1892 : audit [INF] from='client.48305 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]': finished 2024-02-13T03:30:32.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[27879]: cluster 2024-02-13T03:30:32.664960+0000 mon.a (mon.0) 1893 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-02-13T03:30:32.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[27879]: audit 2024-02-13T03:30:32.669026+0000 mon.c (mon.2) 467 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:32.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[27879]: audit 2024-02-13T03:30:32.669440+0000 mon.a (mon.0) 1894 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[19769]: audit 2024-02-13T03:30:31.663916+0000 mon.a (mon.0) 1887 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:30:32.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[19769]: cluster 2024-02-13T03:30:31.664167+0000 mon.a (mon.0) 1888 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-02-13T03:30:32.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[19769]: audit 2024-02-13T03:30:31.667829+0000 mon.a (mon.0) 1889 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-13T03:30:32.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[19769]: audit 2024-02-13T03:30:31.669018+0000 mon.c (mon.2) 466 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:32.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[19769]: audit 2024-02-13T03:30:31.670269+0000 mon.b (mon.1) 255 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-13T03:30:32.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[19769]: audit 2024-02-13T03:30:31.675126+0000 mon.a (mon.0) 1890 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:32.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[19769]: audit 2024-02-13T03:30:32.664828+0000 mon.a (mon.0) 1891 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-13T03:30:32.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[19769]: audit 2024-02-13T03:30:32.664914+0000 mon.a (mon.0) 1892 : audit [INF] from='client.48305 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]': finished 2024-02-13T03:30:32.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[19769]: cluster 2024-02-13T03:30:32.664960+0000 mon.a (mon.0) 1893 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-02-13T03:30:32.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[19769]: audit 2024-02-13T03:30:32.669026+0000 mon.c (mon.2) 467 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:32.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:32 smithi161 bash[19769]: audit 2024-02-13T03:30:32.669440+0000 mon.a (mon.0) 1894 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:32.941 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9047 ms) 2024-02-13T03:30:32.941 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-02-13T03:30:32.941 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg_num = 32 2024-02-13T03:30:32.941 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 0 ls 1707795033,0 2024-02-13T03:30:32.941 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 1 ls 1707795033,0 2024-02-13T03:30:32.941 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 2 ls 1707795033,0 2024-02-13T03:30:32.941 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 3 ls 1707795033,0 2024-02-13T03:30:32.941 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 4 ls 1707795033,0 2024-02-13T03:30:32.941 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 5 ls 1707795033,0 2024-02-13T03:30:32.941 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 6 ls 1707795033,0 2024-02-13T03:30:32.942 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 7 ls 1707795033,0 2024-02-13T03:30:32.942 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 8 ls 1707795033,0 2024-02-13T03:30:32.942 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 9 ls 1707795033,0 2024-02-13T03:30:32.942 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 10 ls 1707795033,0 2024-02-13T03:30:32.942 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 11 ls 1707795033,0 2024-02-13T03:30:32.942 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 12 ls 1707795033,0 2024-02-13T03:30:32.942 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 13 ls 1707795033,0 2024-02-13T03:30:32.942 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 14 ls 1707795033,0 2024-02-13T03:30:32.942 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 15 ls 1707795033,0 2024-02-13T03:30:32.942 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 16 ls 1707795033,0 2024-02-13T03:30:32.942 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 17 ls 1707795033,0 2024-02-13T03:30:32.942 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 18 ls 1707795033,0 2024-02-13T03:30:32.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 19 ls 1707795033,0 2024-02-13T03:30:32.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 20 ls 1707795033,0 2024-02-13T03:30:32.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 21 ls 1707795033,0 2024-02-13T03:30:32.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 22 ls 1707795033,0 2024-02-13T03:30:32.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 23 ls 1707795033,0 2024-02-13T03:30:32.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 24 ls 1707795033,0 2024-02-13T03:30:32.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 25 ls 1707795033,0 2024-02-13T03:30:32.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 26 ls 1707795033,0 2024-02-13T03:30:32.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 27 ls 1707795033,0 2024-02-13T03:30:32.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 28 ls 1707795033,0 2024-02-13T03:30:32.943 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 29 ls 1707795033,0 2024-02-13T03:30:32.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 30 ls 1707795033,0 2024-02-13T03:30:32.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg 31 ls 1707795033,0 2024-02-13T03:30:32.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: pg_num = 32 2024-02-13T03:30:32.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:6cac518f:::0:head 2024-02-13T03:30:32.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:02547ec2:::1:head 2024-02-13T03:30:32.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:f905c69b:::2:head 2024-02-13T03:30:32.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:cfc208b3:::3:head 2024-02-13T03:30:32.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:d83876eb:::4:head 2024-02-13T03:30:32.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:b29083e3:::5:head 2024-02-13T03:30:32.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:c4fdafeb:::6:head 2024-02-13T03:30:32.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:5c6b0b28:::7:head 2024-02-13T03:30:32.944 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:bd63b0f1:::8:head 2024-02-13T03:30:32.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:e960b815:::9:head 2024-02-13T03:30:32.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:52ea6a34:::10:head 2024-02-13T03:30:32.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:89d3ae78:::11:head 2024-02-13T03:30:32.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:de5d7c5f:::12:head 2024-02-13T03:30:32.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:566253c9:::13:head 2024-02-13T03:30:32.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:62a1935d:::14:head 2024-02-13T03:30:32.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:863748b0:::15:head 2024-02-13T03:30:32.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:3958e169:::16:head 2024-02-13T03:30:32.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:4d4dabf9:::17:head 2024-02-13T03:30:32.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:8391935d:::18:head 2024-02-13T03:30:32.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:28883081:::19:head 2024-02-13T03:30:32.945 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:69259c59:::20:head 2024-02-13T03:30:32.946 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:4bdb80b7:::21:head 2024-02-13T03:30:32.946 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:a11c5d71:::22:head 2024-02-13T03:30:32.946 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:271af37b:::23:head 2024-02-13T03:30:32.946 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:95b121be:::24:head 2024-02-13T03:30:32.946 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:58d1031b:::25:head 2024-02-13T03:30:32.946 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:0a050783:::26:head 2024-02-13T03:30:32.946 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:c709704c:::27:head 2024-02-13T03:30:32.946 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:cbe56eaf:::28:head 2024-02-13T03:30:32.946 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:86b4b162:::29:head 2024-02-13T03:30:32.946 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:70d89383:::30:head 2024-02-13T03:30:32.946 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:dd450c7c:::31:head 2024-02-13T03:30:32.946 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:6d5729b1:::32:head 2024-02-13T03:30:32.946 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:c388f3fb:::33:head 2024-02-13T03:30:32.947 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:56cfea31:::34:head 2024-02-13T03:30:32.947 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:9dbc1bf7:::35:head 2024-02-13T03:30:32.947 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:40b74ccd:::36:head 2024-02-13T03:30:32.947 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:4d5aaf42:::37:head 2024-02-13T03:30:32.947 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:920f362c:::38:head 2024-02-13T03:30:32.947 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:6cc53222:::39:head 2024-02-13T03:30:32.947 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:9cad833f:::40:head 2024-02-13T03:30:32.947 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:1ea84d41:::41:head 2024-02-13T03:30:32.947 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:c4480ef6:::42:head 2024-02-13T03:30:32.947 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:a694361e:::43:head 2024-02-13T03:30:32.947 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:d1bd33e9:::44:head 2024-02-13T03:30:32.947 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:ddc2cd5d:::45:head 2024-02-13T03:30:32.948 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:2b782207:::46:head 2024-02-13T03:30:32.948 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:7b187fca:::47:head 2024-02-13T03:30:32.948 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:90ecdf6f:::48:head 2024-02-13T03:30:32.948 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:a5ed95fe:::49:head 2024-02-13T03:30:32.948 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:ea0eaa55:::50:head 2024-02-13T03:30:32.948 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:f33ef17b:::51:head 2024-02-13T03:30:32.948 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:a0d1b2f6:::52:head 2024-02-13T03:30:32.948 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:60c5229e:::53:head 2024-02-13T03:30:32.948 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:edcbc575:::54:head 2024-02-13T03:30:32.948 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:102cf253:::55:head 2024-02-13T03:30:32.948 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:efb7fb0b:::56:head 2024-02-13T03:30:32.949 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:50d0a326:::57:head 2024-02-13T03:30:32.949 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:d4dc5daf:::58:head 2024-02-13T03:30:32.949 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:3a130462:::59:head 2024-02-13T03:30:32.949 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:ec87ed71:::60:head 2024-02-13T03:30:32.949 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:d5bc9454:::61:head 2024-02-13T03:30:32.949 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:3ddfe313:::62:head 2024-02-13T03:30:32.949 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:7c2816b9:::63:head 2024-02-13T03:30:32.949 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:47e00e4d:::64:head 2024-02-13T03:30:32.949 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:c6410c18:::65:head 2024-02-13T03:30:32.949 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:b48ed237:::66:head 2024-02-13T03:30:32.949 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:cd63ad31:::67:head 2024-02-13T03:30:32.949 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:b179e92b:::68:head 2024-02-13T03:30:32.949 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:0d9f741a:::69:head 2024-02-13T03:30:32.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:6d3352ae:::70:head 2024-02-13T03:30:32.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:c6d5c19e:::71:head 2024-02-13T03:30:32.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:bc4729c3:::72:head 2024-02-13T03:30:32.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:77e930b9:::73:head 2024-02-13T03:30:32.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:0abeecfd:::74:head 2024-02-13T03:30:32.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:b7c37e15:::75:head 2024-02-13T03:30:32.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:b6378398:::76:head 2024-02-13T03:30:32.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:02bd68de:::77:head 2024-02-13T03:30:32.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:cc795d2d:::78:head 2024-02-13T03:30:32.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:630d4fea:::79:head 2024-02-13T03:30:32.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:e0d29ef5:::80:head 2024-02-13T03:30:32.950 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:fd6f13d2:::81:head 2024-02-13T03:30:32.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:606461d5:::82:head 2024-02-13T03:30:32.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:eadbdc43:::83:head 2024-02-13T03:30:32.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:8761d0bb:::84:head 2024-02-13T03:30:32.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:9ef0186f:::85:head 2024-02-13T03:30:32.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:e0d41294:::86:head 2024-02-13T03:30:32.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:961de695:::87:head 2024-02-13T03:30:32.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:1423148f:::88:head 2024-02-13T03:30:32.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:633a8fa2:::89:head 2024-02-13T03:30:32.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:a8653809:::90:head 2024-02-13T03:30:32.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:3dac8b33:::91:head 2024-02-13T03:30:32.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:35aad435:::92:head 2024-02-13T03:30:32.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:f6dcc343:::93:head 2024-02-13T03:30:32.951 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:dbbdad87:::94:head 2024-02-13T03:30:32.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:1cb48ce0:::95:head 2024-02-13T03:30:32.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:03cd461c:::96:head 2024-02-13T03:30:32.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:17a4ea99:::97:head 2024-02-13T03:30:32.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:9993c9a7:::98:head 2024-02-13T03:30:32.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:6394211c:::99:head 2024-02-13T03:30:32.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:94c7ae57:::100:head 2024-02-13T03:30:32.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:6fdee5bb:::101:head 2024-02-13T03:30:32.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:9a477fd1:::102:head 2024-02-13T03:30:32.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:eb850916:::103:head 2024-02-13T03:30:32.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:affc56b9:::104:head 2024-02-13T03:30:32.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:b42dc814:::105:head 2024-02-13T03:30:32.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:f319f8f0:::106:head 2024-02-13T03:30:32.952 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:9a40b9de:::107:head 2024-02-13T03:30:32.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:8b524f28:::108:head 2024-02-13T03:30:32.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:e3de589f:::109:head 2024-02-13T03:30:32.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:90f90a5b:::110:head 2024-02-13T03:30:32.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:a7b4f1d7:::111:head 2024-02-13T03:30:32.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:af51766e:::112:head 2024-02-13T03:30:32.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:b6f90bd1:::113:head 2024-02-13T03:30:32.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:e0261208:::114:head 2024-02-13T03:30:32.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:c9569ef7:::115:head 2024-02-13T03:30:32.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:61bebe50:::116:head 2024-02-13T03:30:32.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:fe93412b:::117:head 2024-02-13T03:30:32.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:d3d38bee:::118:head 2024-02-13T03:30:32.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:3100ba0c:::119:head 2024-02-13T03:30:32.953 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:d0560ada:::120:head 2024-02-13T03:30:32.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:f0ea8b35:::121:head 2024-02-13T03:30:32.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:766f231a:::122:head 2024-02-13T03:30:32.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:a07a2582:::123:head 2024-02-13T03:30:32.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:bd7c6b3a:::124:head 2024-02-13T03:30:32.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:fb2ddaff:::125:head 2024-02-13T03:30:32.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:4408e1fe:::126:head 2024-02-13T03:30:32.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:ee1df7a7:::127:head 2024-02-13T03:30:32.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:c3002909:::128:head 2024-02-13T03:30:32.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:4f48ffa9:::129:head 2024-02-13T03:30:32.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:edf38733:::130:head 2024-02-13T03:30:32.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:c08425c0:::131:head 2024-02-13T03:30:32.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:5f902d98:::132:head 2024-02-13T03:30:32.954 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:41ea2c93:::133:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:813cee13:::134:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:0131818d:::135:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:26ba5a85:::136:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:381b8a5a:::137:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:28797e47:::138:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:bfca7f22:::139:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:36807075:::140:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:80b03975:::141:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:5c15709b:::142:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:f39ea15e:::143:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:ea992956:::144:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:48887b1c:::145:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:9f24a9dd:::146:head 2024-02-13T03:30:32.955 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:987f100b:::147:head 2024-02-13T03:30:32.956 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:d2dd3581:::148:head 2024-02-13T03:30:32.956 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:7fed1808:::149:head 2024-02-13T03:30:32.956 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:c80b70e9:::150:head 2024-02-13T03:30:32.956 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:85ed90f9:::151:head 2024-02-13T03:30:32.956 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:36428b24:::152:head 2024-02-13T03:30:32.956 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:d044c34a:::153:head 2024-02-13T03:30:32.956 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:7c18bf58:::154:head 2024-02-13T03:30:32.956 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:d1c21232:::155:head 2024-02-13T03:30:32.956 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:a7a3c575:::156:head 2024-02-13T03:30:32.956 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:87da0633:::157:head 2024-02-13T03:30:32.956 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:d5ac3822:::158:head 2024-02-13T03:30:32.956 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:3f20522d:::159:head 2024-02-13T03:30:32.957 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:6ca26563:::160:head 2024-02-13T03:30:32.957 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:532ce135:::161:head 2024-02-13T03:30:32.957 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:c78863e6:::162:head 2024-02-13T03:30:32.957 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:7462ddf6:::163:head 2024-02-13T03:30:32.957 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:5d165639:::164:head 2024-02-13T03:30:32.957 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:f43765fc:::165:head 2024-02-13T03:30:32.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:32 smithi183 bash[20824]: audit 2024-02-13T03:30:31.663916+0000 mon.a (mon.0) 1887 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:30:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:32 smithi183 bash[20824]: cluster 2024-02-13T03:30:31.664167+0000 mon.a (mon.0) 1888 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-02-13T03:30:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:32 smithi183 bash[20824]: audit 2024-02-13T03:30:31.667829+0000 mon.a (mon.0) 1889 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-13T03:30:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:32 smithi183 bash[20824]: audit 2024-02-13T03:30:31.669018+0000 mon.c (mon.2) 466 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:32 smithi183 bash[20824]: audit 2024-02-13T03:30:31.670269+0000 mon.b (mon.1) 255 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-13T03:30:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:32 smithi183 bash[20824]: audit 2024-02-13T03:30:31.675126+0000 mon.a (mon.0) 1890 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:32 smithi183 bash[20824]: audit 2024-02-13T03:30:32.664828+0000 mon.a (mon.0) 1891 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-13T03:30:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:32 smithi183 bash[20824]: audit 2024-02-13T03:30:32.664914+0000 mon.a (mon.0) 1892 : audit [INF] from='client.48305 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi161-61726-54"}]': finished 2024-02-13T03:30:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:32 smithi183 bash[20824]: cluster 2024-02-13T03:30:32.664960+0000 mon.a (mon.0) 1893 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-02-13T03:30:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:32 smithi183 bash[20824]: audit 2024-02-13T03:30:32.669026+0000 mon.c (mon.2) 467 : audit [INF] from='client.? v1:172.21.15.161:0/1595036758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:32 smithi183 bash[20824]: audit 2024-02-13T03:30:32.669440+0000 mon.a (mon.0) 1894 : audit [INF] from='client.48305 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]: dispatch 2024-02-13T03:30:33.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:33 smithi183 bash[20824]: audit 2024-02-13T03:30:32.942178+0000 mon.b (mon.1) 256 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi161-61906-41","var": "pg_num","format": "json"}]: dispatch 2024-02-13T03:30:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:33 smithi183 bash[20824]: audit 2024-02-13T03:30:33.192612+0000 mon.a (mon.0) 1895 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:33 smithi183 bash[20824]: audit 2024-02-13T03:30:33.194162+0000 mon.a (mon.0) 1896 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41"}]: dispatch 2024-02-13T03:30:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:33 smithi183 bash[20824]: audit 2024-02-13T03:30:33.194962+0000 mon.b (mon.1) 257 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:33 smithi183 bash[20824]: audit 2024-02-13T03:30:33.196573+0000 mon.b (mon.1) 258 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41"}]: dispatch 2024-02-13T03:30:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:33 smithi183 bash[20824]: cluster 2024-02-13T03:30:33.314629+0000 mgr.y (mgr.24473) 385 : cluster [DBG] pgmap v491: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:34.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:33 smithi161 bash[19769]: audit 2024-02-13T03:30:32.942178+0000 mon.b (mon.1) 256 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi161-61906-41","var": "pg_num","format": "json"}]: dispatch 2024-02-13T03:30:34.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:33 smithi161 bash[19769]: audit 2024-02-13T03:30:33.192612+0000 mon.a (mon.0) 1895 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:34.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:33 smithi161 bash[19769]: audit 2024-02-13T03:30:33.194162+0000 mon.a (mon.0) 1896 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41"}]: dispatch 2024-02-13T03:30:34.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:33 smithi161 bash[19769]: audit 2024-02-13T03:30:33.194962+0000 mon.b (mon.1) 257 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:34.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:33 smithi161 bash[19769]: audit 2024-02-13T03:30:33.196573+0000 mon.b (mon.1) 258 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41"}]: dispatch 2024-02-13T03:30:34.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:33 smithi161 bash[19769]: cluster 2024-02-13T03:30:33.314629+0000 mgr.y (mgr.24473) 385 : cluster [DBG] pgmap v491: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:34.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:33 smithi161 bash[27879]: audit 2024-02-13T03:30:32.942178+0000 mon.b (mon.1) 256 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi161-61906-41","var": "pg_num","format": "json"}]: dispatch 2024-02-13T03:30:34.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:33 smithi161 bash[27879]: audit 2024-02-13T03:30:33.192612+0000 mon.a (mon.0) 1895 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:34.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:33 smithi161 bash[27879]: audit 2024-02-13T03:30:33.194162+0000 mon.a (mon.0) 1896 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41"}]: dispatch 2024-02-13T03:30:34.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:33 smithi161 bash[27879]: audit 2024-02-13T03:30:33.194962+0000 mon.b (mon.1) 257 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:34.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:33 smithi161 bash[27879]: audit 2024-02-13T03:30:33.196573+0000 mon.b (mon.1) 258 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41"}]: dispatch 2024-02-13T03:30:34.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:33 smithi161 bash[27879]: cluster 2024-02-13T03:30:33.314629+0000 mgr.y (mgr.24473) 385 : cluster [DBG] pgmap v491: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:34 smithi183 bash[20824]: audit 2024-02-13T03:30:33.676139+0000 mon.a (mon.0) 1897 : audit [INF] from='client.48305 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]': finished 2024-02-13T03:30:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:34 smithi183 bash[20824]: audit 2024-02-13T03:30:33.676386+0000 mon.a (mon.0) 1898 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41"}]': finished 2024-02-13T03:30:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:34 smithi183 bash[20824]: cluster 2024-02-13T03:30:33.676502+0000 mon.a (mon.0) 1899 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-02-13T03:30:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:34 smithi183 bash[20824]: audit 2024-02-13T03:30:33.699127+0000 mon.c (mon.2) 468 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:34 smithi183 bash[20824]: audit 2024-02-13T03:30:33.699765+0000 mon.a (mon.0) 1900 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:34 smithi183 bash[20824]: audit 2024-02-13T03:30:33.700550+0000 mon.c (mon.2) 469 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:34 smithi183 bash[20824]: audit 2024-02-13T03:30:33.700880+0000 mon.a (mon.0) 1901 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:34 smithi183 bash[20824]: audit 2024-02-13T03:30:33.701395+0000 mon.c (mon.2) 470 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi161-61726-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:34 smithi183 bash[20824]: audit 2024-02-13T03:30:33.701713+0000 mon.a (mon.0) 1902 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi161-61726-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:35.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[19769]: audit 2024-02-13T03:30:33.676139+0000 mon.a (mon.0) 1897 : audit [INF] from='client.48305 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]': finished 2024-02-13T03:30:35.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[19769]: audit 2024-02-13T03:30:33.676386+0000 mon.a (mon.0) 1898 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41"}]': finished 2024-02-13T03:30:35.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[19769]: cluster 2024-02-13T03:30:33.676502+0000 mon.a (mon.0) 1899 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-02-13T03:30:35.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[19769]: audit 2024-02-13T03:30:33.699127+0000 mon.c (mon.2) 468 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:35.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[19769]: audit 2024-02-13T03:30:33.699765+0000 mon.a (mon.0) 1900 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:35.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[19769]: audit 2024-02-13T03:30:33.700550+0000 mon.c (mon.2) 469 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:35.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[19769]: audit 2024-02-13T03:30:33.700880+0000 mon.a (mon.0) 1901 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:35.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[19769]: audit 2024-02-13T03:30:33.701395+0000 mon.c (mon.2) 470 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi161-61726-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:35.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[19769]: audit 2024-02-13T03:30:33.701713+0000 mon.a (mon.0) 1902 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi161-61726-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:35.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[27879]: audit 2024-02-13T03:30:33.676139+0000 mon.a (mon.0) 1897 : audit [INF] from='client.48305 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi161-61726-54"}]': finished 2024-02-13T03:30:35.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[27879]: audit 2024-02-13T03:30:33.676386+0000 mon.a (mon.0) 1898 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-41"}]': finished 2024-02-13T03:30:35.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[27879]: cluster 2024-02-13T03:30:33.676502+0000 mon.a (mon.0) 1899 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-02-13T03:30:35.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[27879]: audit 2024-02-13T03:30:33.699127+0000 mon.c (mon.2) 468 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:35.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[27879]: audit 2024-02-13T03:30:33.699765+0000 mon.a (mon.0) 1900 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:35.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[27879]: audit 2024-02-13T03:30:33.700550+0000 mon.c (mon.2) 469 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:35.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[27879]: audit 2024-02-13T03:30:33.700880+0000 mon.a (mon.0) 1901 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:35.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[27879]: audit 2024-02-13T03:30:33.701395+0000 mon.c (mon.2) 470 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi161-61726-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:35.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:34 smithi161 bash[27879]: audit 2024-02-13T03:30:33.701713+0000 mon.a (mon.0) 1902 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi161-61726-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:35.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:35 smithi183 bash[20824]: audit 2024-02-13T03:30:34.697393+0000 mon.a (mon.0) 1903 : audit [INF] from='client.48311 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi161-61726-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:35 smithi183 bash[20824]: cluster 2024-02-13T03:30:34.697626+0000 mon.a (mon.0) 1904 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-02-13T03:30:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:35 smithi183 bash[20824]: audit 2024-02-13T03:30:34.700787+0000 mon.c (mon.2) 471 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi161-61726-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:35 smithi183 bash[20824]: audit 2024-02-13T03:30:35.023847+0000 mon.c (mon.2) 472 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.9", "id": [2, 1]}]: dispatch 2024-02-13T03:30:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:35 smithi183 bash[20824]: audit 2024-02-13T03:30:35.024439+0000 mon.c (mon.2) 473 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.10", "id": [7, 0]}]: dispatch 2024-02-13T03:30:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:35 smithi183 bash[20824]: audit 2024-02-13T03:30:35.024850+0000 mon.c (mon.2) 474 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]: dispatch 2024-02-13T03:30:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:35 smithi183 bash[20824]: audit 2024-02-13T03:30:35.147899+0000 mon.a (mon.0) 1905 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi161-61726-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:35 smithi183 bash[20824]: audit 2024-02-13T03:30:35.149159+0000 mon.a (mon.0) 1906 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.9", "id": [2, 1]}]: dispatch 2024-02-13T03:30:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:35 smithi183 bash[20824]: audit 2024-02-13T03:30:35.149768+0000 mon.a (mon.0) 1907 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.10", "id": [7, 0]}]: dispatch 2024-02-13T03:30:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:35 smithi183 bash[20824]: audit 2024-02-13T03:30:35.150301+0000 mon.a (mon.0) 1908 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]: dispatch 2024-02-13T03:30:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:35 smithi183 bash[20824]: cluster 2024-02-13T03:30:35.315374+0000 mgr.y (mgr.24473) 386 : cluster [DBG] pgmap v494: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:36.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[27879]: audit 2024-02-13T03:30:34.697393+0000 mon.a (mon.0) 1903 : audit [INF] from='client.48311 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi161-61726-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:36.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[27879]: cluster 2024-02-13T03:30:34.697626+0000 mon.a (mon.0) 1904 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-02-13T03:30:36.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[27879]: audit 2024-02-13T03:30:34.700787+0000 mon.c (mon.2) 471 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi161-61726-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:36.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[27879]: audit 2024-02-13T03:30:35.023847+0000 mon.c (mon.2) 472 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.9", "id": [2, 1]}]: dispatch 2024-02-13T03:30:36.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[27879]: audit 2024-02-13T03:30:35.024439+0000 mon.c (mon.2) 473 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.10", "id": [7, 0]}]: dispatch 2024-02-13T03:30:36.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[27879]: audit 2024-02-13T03:30:35.024850+0000 mon.c (mon.2) 474 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]: dispatch 2024-02-13T03:30:36.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[27879]: audit 2024-02-13T03:30:35.147899+0000 mon.a (mon.0) 1905 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi161-61726-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:36.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[27879]: audit 2024-02-13T03:30:35.149159+0000 mon.a (mon.0) 1906 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.9", "id": [2, 1]}]: dispatch 2024-02-13T03:30:36.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[27879]: audit 2024-02-13T03:30:35.149768+0000 mon.a (mon.0) 1907 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.10", "id": [7, 0]}]: dispatch 2024-02-13T03:30:36.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[27879]: audit 2024-02-13T03:30:35.150301+0000 mon.a (mon.0) 1908 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]: dispatch 2024-02-13T03:30:36.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[27879]: cluster 2024-02-13T03:30:35.315374+0000 mgr.y (mgr.24473) 386 : cluster [DBG] pgmap v494: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:36.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[19769]: audit 2024-02-13T03:30:34.697393+0000 mon.a (mon.0) 1903 : audit [INF] from='client.48311 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi161-61726-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:36.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[19769]: cluster 2024-02-13T03:30:34.697626+0000 mon.a (mon.0) 1904 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-02-13T03:30:36.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[19769]: audit 2024-02-13T03:30:34.700787+0000 mon.c (mon.2) 471 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi161-61726-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:36.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[19769]: audit 2024-02-13T03:30:35.023847+0000 mon.c (mon.2) 472 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.9", "id": [2, 1]}]: dispatch 2024-02-13T03:30:36.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[19769]: audit 2024-02-13T03:30:35.024439+0000 mon.c (mon.2) 473 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.10", "id": [7, 0]}]: dispatch 2024-02-13T03:30:36.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[19769]: audit 2024-02-13T03:30:35.024850+0000 mon.c (mon.2) 474 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]: dispatch 2024-02-13T03:30:36.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[19769]: audit 2024-02-13T03:30:35.147899+0000 mon.a (mon.0) 1905 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi161-61726-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:36.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[19769]: audit 2024-02-13T03:30:35.149159+0000 mon.a (mon.0) 1906 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.9", "id": [2, 1]}]: dispatch 2024-02-13T03:30:36.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[19769]: audit 2024-02-13T03:30:35.149768+0000 mon.a (mon.0) 1907 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.10", "id": [7, 0]}]: dispatch 2024-02-13T03:30:36.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[19769]: audit 2024-02-13T03:30:35.150301+0000 mon.a (mon.0) 1908 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]: dispatch 2024-02-13T03:30:36.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:35 smithi161 bash[19769]: cluster 2024-02-13T03:30:35.315374+0000 mgr.y (mgr.24473) 386 : cluster [DBG] pgmap v494: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:36.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:30:36 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:30:36.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:36 smithi183 bash[20824]: audit 2024-02-13T03:30:35.728119+0000 mon.b (mon.1) 259 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:36 smithi183 bash[20824]: audit 2024-02-13T03:30:35.997803+0000 mgr.y (mgr.24473) 387 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:36 smithi183 bash[20824]: audit 2024-02-13T03:30:36.145280+0000 mon.a (mon.0) 1909 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.9", "id": [2, 1]}]': finished 2024-02-13T03:30:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:36 smithi183 bash[20824]: audit 2024-02-13T03:30:36.145371+0000 mon.a (mon.0) 1910 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.10", "id": [7, 0]}]': finished 2024-02-13T03:30:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:36 smithi183 bash[20824]: audit 2024-02-13T03:30:36.145464+0000 mon.a (mon.0) 1911 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]': finished 2024-02-13T03:30:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:36 smithi183 bash[20824]: cluster 2024-02-13T03:30:36.145530+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-02-13T03:30:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:36 smithi183 bash[20824]: audit 2024-02-13T03:30:36.147629+0000 mon.a (mon.0) 1913 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:37.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[27879]: audit 2024-02-13T03:30:35.728119+0000 mon.b (mon.1) 259 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:37.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[27879]: audit 2024-02-13T03:30:35.997803+0000 mgr.y (mgr.24473) 387 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:37.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[27879]: audit 2024-02-13T03:30:36.145280+0000 mon.a (mon.0) 1909 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.9", "id": [2, 1]}]': finished 2024-02-13T03:30:37.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[27879]: audit 2024-02-13T03:30:36.145371+0000 mon.a (mon.0) 1910 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.10", "id": [7, 0]}]': finished 2024-02-13T03:30:37.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[27879]: audit 2024-02-13T03:30:36.145464+0000 mon.a (mon.0) 1911 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]': finished 2024-02-13T03:30:37.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[27879]: cluster 2024-02-13T03:30:36.145530+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-02-13T03:30:37.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[27879]: audit 2024-02-13T03:30:36.147629+0000 mon.a (mon.0) 1913 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:37.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[19769]: audit 2024-02-13T03:30:35.728119+0000 mon.b (mon.1) 259 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:37.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[19769]: audit 2024-02-13T03:30:35.997803+0000 mgr.y (mgr.24473) 387 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:37.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[19769]: audit 2024-02-13T03:30:36.145280+0000 mon.a (mon.0) 1909 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.9", "id": [2, 1]}]': finished 2024-02-13T03:30:37.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[19769]: audit 2024-02-13T03:30:36.145371+0000 mon.a (mon.0) 1910 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.10", "id": [7, 0]}]': finished 2024-02-13T03:30:37.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[19769]: audit 2024-02-13T03:30:36.145464+0000 mon.a (mon.0) 1911 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.1a", "id": [7, 5]}]': finished 2024-02-13T03:30:37.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[19769]: cluster 2024-02-13T03:30:36.145530+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-02-13T03:30:37.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:36 smithi161 bash[19769]: audit 2024-02-13T03:30:36.147629+0000 mon.a (mon.0) 1913 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:38.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:37 smithi161 bash[19769]: audit 2024-02-13T03:30:36.733628+0000 mon.a (mon.0) 1914 : audit [INF] from='client.48311 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi161-61726-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi161-61726-55"}]': finished 2024-02-13T03:30:38.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:37 smithi161 bash[19769]: audit 2024-02-13T03:30:36.733900+0000 mon.a (mon.0) 1915 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:38.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:37 smithi161 bash[19769]: cluster 2024-02-13T03:30:36.734097+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-02-13T03:30:38.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:37 smithi161 bash[19769]: audit 2024-02-13T03:30:36.741218+0000 mon.a (mon.0) 1917 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:38.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:37 smithi161 bash[19769]: audit 2024-02-13T03:30:36.743502+0000 mon.b (mon.1) 260 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:38.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:37 smithi161 bash[19769]: cluster 2024-02-13T03:30:37.316216+0000 mgr.y (mgr.24473) 388 : cluster [DBG] pgmap v497: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:38.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:37 smithi161 bash[27879]: audit 2024-02-13T03:30:36.733628+0000 mon.a (mon.0) 1914 : audit [INF] from='client.48311 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi161-61726-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi161-61726-55"}]': finished 2024-02-13T03:30:38.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:37 smithi161 bash[27879]: audit 2024-02-13T03:30:36.733900+0000 mon.a (mon.0) 1915 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:38.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:37 smithi161 bash[27879]: cluster 2024-02-13T03:30:36.734097+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-02-13T03:30:38.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:37 smithi161 bash[27879]: audit 2024-02-13T03:30:36.741218+0000 mon.a (mon.0) 1917 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:38.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:37 smithi161 bash[27879]: audit 2024-02-13T03:30:36.743502+0000 mon.b (mon.1) 260 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:38.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:37 smithi161 bash[27879]: cluster 2024-02-13T03:30:37.316216+0000 mgr.y (mgr.24473) 388 : cluster [DBG] pgmap v497: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:37 smithi183 bash[20824]: audit 2024-02-13T03:30:36.733628+0000 mon.a (mon.0) 1914 : audit [INF] from='client.48311 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi161-61726-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi161-61726-55"}]': finished 2024-02-13T03:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:37 smithi183 bash[20824]: audit 2024-02-13T03:30:36.733900+0000 mon.a (mon.0) 1915 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:37 smithi183 bash[20824]: cluster 2024-02-13T03:30:36.734097+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-02-13T03:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:37 smithi183 bash[20824]: audit 2024-02-13T03:30:36.741218+0000 mon.a (mon.0) 1917 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:37 smithi183 bash[20824]: audit 2024-02-13T03:30:36.743502+0000 mon.b (mon.1) 260 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:37 smithi183 bash[20824]: cluster 2024-02-13T03:30:37.316216+0000 mgr.y (mgr.24473) 388 : cluster [DBG] pgmap v497: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:39.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:38 smithi161 bash[27879]: audit 2024-02-13T03:30:37.747829+0000 mon.a (mon.0) 1918 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:39.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:38 smithi161 bash[27879]: cluster 2024-02-13T03:30:37.748064+0000 mon.a (mon.0) 1919 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-02-13T03:30:39.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:38 smithi161 bash[27879]: audit 2024-02-13T03:30:37.749577+0000 mon.a (mon.0) 1920 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-13T03:30:39.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:38 smithi161 bash[27879]: audit 2024-02-13T03:30:37.751800+0000 mon.b (mon.1) 261 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-13T03:30:39.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:38 smithi161 bash[19769]: audit 2024-02-13T03:30:37.747829+0000 mon.a (mon.0) 1918 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:39.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:38 smithi161 bash[19769]: cluster 2024-02-13T03:30:37.748064+0000 mon.a (mon.0) 1919 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-02-13T03:30:39.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:38 smithi161 bash[19769]: audit 2024-02-13T03:30:37.749577+0000 mon.a (mon.0) 1920 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-13T03:30:39.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:38 smithi161 bash[19769]: audit 2024-02-13T03:30:37.751800+0000 mon.b (mon.1) 261 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-13T03:30:39.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:38 smithi183 bash[20824]: audit 2024-02-13T03:30:37.747829+0000 mon.a (mon.0) 1918 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:39.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:38 smithi183 bash[20824]: cluster 2024-02-13T03:30:37.748064+0000 mon.a (mon.0) 1919 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-02-13T03:30:39.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:38 smithi183 bash[20824]: audit 2024-02-13T03:30:37.749577+0000 mon.a (mon.0) 1920 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-13T03:30:39.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:38 smithi183 bash[20824]: audit 2024-02-13T03:30:37.751800+0000 mon.b (mon.1) 261 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-13T03:30:39.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:30:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:30:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:30:40.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[27879]: audit 2024-02-13T03:30:38.758000+0000 mon.a (mon.0) 1921 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_count","val": "3"}]': finished 2024-02-13T03:30:40.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[27879]: cluster 2024-02-13T03:30:38.758170+0000 mon.a (mon.0) 1922 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-02-13T03:30:40.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[27879]: audit 2024-02-13T03:30:38.759954+0000 mon.c (mon.2) 475 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[27879]: audit 2024-02-13T03:30:38.762279+0000 mon.b (mon.1) 262 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-13T03:30:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[27879]: audit 2024-02-13T03:30:38.763575+0000 mon.a (mon.0) 1923 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-13T03:30:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[27879]: audit 2024-02-13T03:30:38.763841+0000 mon.a (mon.0) 1924 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:40.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[27879]: cluster 2024-02-13T03:30:39.317069+0000 mgr.y (mgr.24473) 389 : cluster [DBG] pgmap v500: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:40.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[19769]: audit 2024-02-13T03:30:38.758000+0000 mon.a (mon.0) 1921 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_count","val": "3"}]': finished 2024-02-13T03:30:40.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[19769]: cluster 2024-02-13T03:30:38.758170+0000 mon.a (mon.0) 1922 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-02-13T03:30:40.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[19769]: audit 2024-02-13T03:30:38.759954+0000 mon.c (mon.2) 475 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:40.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[19769]: audit 2024-02-13T03:30:38.762279+0000 mon.b (mon.1) 262 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-13T03:30:40.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[19769]: audit 2024-02-13T03:30:38.763575+0000 mon.a (mon.0) 1923 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-13T03:30:40.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[19769]: audit 2024-02-13T03:30:38.763841+0000 mon.a (mon.0) 1924 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:40.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:39 smithi161 bash[19769]: cluster 2024-02-13T03:30:39.317069+0000 mgr.y (mgr.24473) 389 : cluster [DBG] pgmap v500: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:40.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:39 smithi183 bash[20824]: audit 2024-02-13T03:30:38.758000+0000 mon.a (mon.0) 1921 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_count","val": "3"}]': finished 2024-02-13T03:30:40.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:39 smithi183 bash[20824]: cluster 2024-02-13T03:30:38.758170+0000 mon.a (mon.0) 1922 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-02-13T03:30:40.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:39 smithi183 bash[20824]: audit 2024-02-13T03:30:38.759954+0000 mon.c (mon.2) 475 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:39 smithi183 bash[20824]: audit 2024-02-13T03:30:38.762279+0000 mon.b (mon.1) 262 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-13T03:30:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:39 smithi183 bash[20824]: audit 2024-02-13T03:30:38.763575+0000 mon.a (mon.0) 1923 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-13T03:30:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:39 smithi183 bash[20824]: audit 2024-02-13T03:30:38.763841+0000 mon.a (mon.0) 1924 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:39 smithi183 bash[20824]: cluster 2024-02-13T03:30:39.317069+0000 mgr.y (mgr.24473) 389 : cluster [DBG] pgmap v500: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:41.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[27879]: audit 2024-02-13T03:30:39.762569+0000 mon.a (mon.0) 1925 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_period","val": "3"}]': finished 2024-02-13T03:30:41.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[27879]: audit 2024-02-13T03:30:39.762809+0000 mon.a (mon.0) 1926 : audit [INF] from='client.48311 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]': finished 2024-02-13T03:30:41.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[27879]: cluster 2024-02-13T03:30:39.763019+0000 mon.a (mon.0) 1927 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-02-13T03:30:41.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[27879]: audit 2024-02-13T03:30:39.764861+0000 mon.a (mon.0) 1928 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:30:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[27879]: audit 2024-02-13T03:30:39.766576+0000 mon.b (mon.1) 263 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:30:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[27879]: audit 2024-02-13T03:30:39.767827+0000 mon.c (mon.2) 476 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[27879]: audit 2024-02-13T03:30:39.775096+0000 mon.a (mon.0) 1929 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[27879]: audit 2024-02-13T03:30:40.762789+0000 mon.a (mon.0) 1930 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:30:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[27879]: audit 2024-02-13T03:30:40.763039+0000 mon.a (mon.0) 1931 : audit [INF] from='client.48311 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]': finished 2024-02-13T03:30:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[27879]: cluster 2024-02-13T03:30:40.763209+0000 mon.a (mon.0) 1932 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-02-13T03:30:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[27879]: audit 2024-02-13T03:30:40.765859+0000 mon.a (mon.0) 1933 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-13T03:30:41.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[27879]: audit 2024-02-13T03:30:40.768146+0000 mon.b (mon.1) 264 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-13T03:30:41.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[19769]: audit 2024-02-13T03:30:39.762569+0000 mon.a (mon.0) 1925 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_period","val": "3"}]': finished 2024-02-13T03:30:41.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[19769]: audit 2024-02-13T03:30:39.762809+0000 mon.a (mon.0) 1926 : audit [INF] from='client.48311 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]': finished 2024-02-13T03:30:41.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[19769]: cluster 2024-02-13T03:30:39.763019+0000 mon.a (mon.0) 1927 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-02-13T03:30:41.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[19769]: audit 2024-02-13T03:30:39.764861+0000 mon.a (mon.0) 1928 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:30:41.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[19769]: audit 2024-02-13T03:30:39.766576+0000 mon.b (mon.1) 263 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:30:41.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[19769]: audit 2024-02-13T03:30:39.767827+0000 mon.c (mon.2) 476 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:41.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[19769]: audit 2024-02-13T03:30:39.775096+0000 mon.a (mon.0) 1929 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:41.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[19769]: audit 2024-02-13T03:30:40.762789+0000 mon.a (mon.0) 1930 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:30:41.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[19769]: audit 2024-02-13T03:30:40.763039+0000 mon.a (mon.0) 1931 : audit [INF] from='client.48311 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]': finished 2024-02-13T03:30:41.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[19769]: cluster 2024-02-13T03:30:40.763209+0000 mon.a (mon.0) 1932 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-02-13T03:30:41.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[19769]: audit 2024-02-13T03:30:40.765859+0000 mon.a (mon.0) 1933 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-13T03:30:41.101 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:40 smithi161 bash[19769]: audit 2024-02-13T03:30:40.768146+0000 mon.b (mon.1) 264 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-13T03:30:41.101 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:30:41] "GET /metrics HTTP/1.1" 200 183932 "" "Prometheus/2.33.4" 2024-02-13T03:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:40 smithi183 bash[20824]: audit 2024-02-13T03:30:39.762569+0000 mon.a (mon.0) 1925 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_period","val": "3"}]': finished 2024-02-13T03:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:40 smithi183 bash[20824]: audit 2024-02-13T03:30:39.762809+0000 mon.a (mon.0) 1926 : audit [INF] from='client.48311 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi161-61726-55"}]': finished 2024-02-13T03:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:40 smithi183 bash[20824]: cluster 2024-02-13T03:30:39.763019+0000 mon.a (mon.0) 1927 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-02-13T03:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:40 smithi183 bash[20824]: audit 2024-02-13T03:30:39.764861+0000 mon.a (mon.0) 1928 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:40 smithi183 bash[20824]: audit 2024-02-13T03:30:39.766576+0000 mon.b (mon.1) 263 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:40 smithi183 bash[20824]: audit 2024-02-13T03:30:39.767827+0000 mon.c (mon.2) 476 : audit [INF] from='client.? v1:172.21.15.161:0/2762789482' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:40 smithi183 bash[20824]: audit 2024-02-13T03:30:39.775096+0000 mon.a (mon.0) 1929 : audit [INF] from='client.48311 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]: dispatch 2024-02-13T03:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:40 smithi183 bash[20824]: audit 2024-02-13T03:30:40.762789+0000 mon.a (mon.0) 1930 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:40 smithi183 bash[20824]: audit 2024-02-13T03:30:40.763039+0000 mon.a (mon.0) 1931 : audit [INF] from='client.48311 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi161-61726-55"}]': finished 2024-02-13T03:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:40 smithi183 bash[20824]: cluster 2024-02-13T03:30:40.763209+0000 mon.a (mon.0) 1932 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-02-13T03:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:40 smithi183 bash[20824]: audit 2024-02-13T03:30:40.765859+0000 mon.a (mon.0) 1933 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-13T03:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:40 smithi183 bash[20824]: audit 2024-02-13T03:30:40.768146+0000 mon.b (mon.1) 264 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-13T03:30:42.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[27879]: audit 2024-02-13T03:30:40.784907+0000 mon.c (mon.2) 477 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[27879]: audit 2024-02-13T03:30:40.785356+0000 mon.a (mon.0) 1934 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[27879]: audit 2024-02-13T03:30:40.786578+0000 mon.c (mon.2) 478 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[27879]: audit 2024-02-13T03:30:40.786914+0000 mon.a (mon.0) 1935 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[27879]: audit 2024-02-13T03:30:40.787546+0000 mon.c (mon.2) 479 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi161-61726-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:42.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[27879]: audit 2024-02-13T03:30:40.787998+0000 mon.a (mon.0) 1936 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi161-61726-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:42.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[27879]: cluster 2024-02-13T03:30:41.317959+0000 mgr.y (mgr.24473) 390 : cluster [DBG] pgmap v503: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 20 B/s, 8 objects/s recovering 2024-02-13T03:30:42.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[27879]: audit 2024-02-13T03:30:41.769270+0000 mon.a (mon.0) 1937 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-13T03:30:42.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[27879]: audit 2024-02-13T03:30:41.769395+0000 mon.a (mon.0) 1938 : audit [INF] from='client.48317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi161-61726-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:42.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[27879]: cluster 2024-02-13T03:30:41.769483+0000 mon.a (mon.0) 1939 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-02-13T03:30:42.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[27879]: audit 2024-02-13T03:30:41.771521+0000 mon.c (mon.2) 480 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi161-61726-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[19769]: audit 2024-02-13T03:30:40.784907+0000 mon.c (mon.2) 477 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[19769]: audit 2024-02-13T03:30:40.785356+0000 mon.a (mon.0) 1934 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[19769]: audit 2024-02-13T03:30:40.786578+0000 mon.c (mon.2) 478 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[19769]: audit 2024-02-13T03:30:40.786914+0000 mon.a (mon.0) 1935 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[19769]: audit 2024-02-13T03:30:40.787546+0000 mon.c (mon.2) 479 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi161-61726-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[19769]: audit 2024-02-13T03:30:40.787998+0000 mon.a (mon.0) 1936 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi161-61726-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[19769]: cluster 2024-02-13T03:30:41.317959+0000 mgr.y (mgr.24473) 390 : cluster [DBG] pgmap v503: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 20 B/s, 8 objects/s recovering 2024-02-13T03:30:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[19769]: audit 2024-02-13T03:30:41.769270+0000 mon.a (mon.0) 1937 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-13T03:30:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[19769]: audit 2024-02-13T03:30:41.769395+0000 mon.a (mon.0) 1938 : audit [INF] from='client.48317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi161-61726-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[19769]: cluster 2024-02-13T03:30:41.769483+0000 mon.a (mon.0) 1939 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-02-13T03:30:42.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:41 smithi161 bash[19769]: audit 2024-02-13T03:30:41.771521+0000 mon.c (mon.2) 480 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi161-61726-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:41 smithi183 bash[20824]: audit 2024-02-13T03:30:40.784907+0000 mon.c (mon.2) 477 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:41 smithi183 bash[20824]: audit 2024-02-13T03:30:40.785356+0000 mon.a (mon.0) 1934 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:41 smithi183 bash[20824]: audit 2024-02-13T03:30:40.786578+0000 mon.c (mon.2) 478 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:41 smithi183 bash[20824]: audit 2024-02-13T03:30:40.786914+0000 mon.a (mon.0) 1935 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:41 smithi183 bash[20824]: audit 2024-02-13T03:30:40.787546+0000 mon.c (mon.2) 479 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi161-61726-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:41 smithi183 bash[20824]: audit 2024-02-13T03:30:40.787998+0000 mon.a (mon.0) 1936 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi161-61726-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:41 smithi183 bash[20824]: cluster 2024-02-13T03:30:41.317959+0000 mgr.y (mgr.24473) 390 : cluster [DBG] pgmap v503: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 20 B/s, 8 objects/s recovering 2024-02-13T03:30:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:41 smithi183 bash[20824]: audit 2024-02-13T03:30:41.769270+0000 mon.a (mon.0) 1937 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-13T03:30:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:41 smithi183 bash[20824]: audit 2024-02-13T03:30:41.769395+0000 mon.a (mon.0) 1938 : audit [INF] from='client.48317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi161-61726-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:41 smithi183 bash[20824]: cluster 2024-02-13T03:30:41.769483+0000 mon.a (mon.0) 1939 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-02-13T03:30:42.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:41 smithi183 bash[20824]: audit 2024-02-13T03:30:41.771521+0000 mon.c (mon.2) 480 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi161-61726-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:43 smithi183 bash[20824]: audit 2024-02-13T03:30:41.773065+0000 mon.a (mon.0) 1940 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi161-61726-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:43.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:43 smithi161 bash[27879]: audit 2024-02-13T03:30:41.773065+0000 mon.a (mon.0) 1940 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi161-61726-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:43.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:43 smithi161 bash[19769]: audit 2024-02-13T03:30:41.773065+0000 mon.a (mon.0) 1940 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi161-61726-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:44.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:44 smithi183 bash[20824]: cluster 2024-02-13T03:30:43.193742+0000 mon.a (mon.0) 1941 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-02-13T03:30:44.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:44 smithi183 bash[20824]: cluster 2024-02-13T03:30:43.318790+0000 mgr.y (mgr.24473) 391 : cluster [DBG] pgmap v506: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 20 B/s, 8 objects/s recovering 2024-02-13T03:30:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:44 smithi183 bash[20824]: audit 2024-02-13T03:30:43.770872+0000 mon.a (mon.0) 1942 : audit [INF] from='client.48317 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi161-61726-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi161-61726-56"}]': finished 2024-02-13T03:30:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:44 smithi183 bash[20824]: cluster 2024-02-13T03:30:43.771115+0000 mon.a (mon.0) 1943 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-02-13T03:30:44.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:44 smithi161 bash[27879]: cluster 2024-02-13T03:30:43.193742+0000 mon.a (mon.0) 1941 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-02-13T03:30:44.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:44 smithi161 bash[27879]: cluster 2024-02-13T03:30:43.318790+0000 mgr.y (mgr.24473) 391 : cluster [DBG] pgmap v506: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 20 B/s, 8 objects/s recovering 2024-02-13T03:30:44.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:44 smithi161 bash[27879]: audit 2024-02-13T03:30:43.770872+0000 mon.a (mon.0) 1942 : audit [INF] from='client.48317 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi161-61726-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi161-61726-56"}]': finished 2024-02-13T03:30:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:44 smithi161 bash[27879]: cluster 2024-02-13T03:30:43.771115+0000 mon.a (mon.0) 1943 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-02-13T03:30:44.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:44 smithi161 bash[19769]: cluster 2024-02-13T03:30:43.193742+0000 mon.a (mon.0) 1941 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-02-13T03:30:44.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:44 smithi161 bash[19769]: cluster 2024-02-13T03:30:43.318790+0000 mgr.y (mgr.24473) 391 : cluster [DBG] pgmap v506: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 20 B/s, 8 objects/s recovering 2024-02-13T03:30:44.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:44 smithi161 bash[19769]: audit 2024-02-13T03:30:43.770872+0000 mon.a (mon.0) 1942 : audit [INF] from='client.48317 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi161-61726-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi161-61726-56"}]': finished 2024-02-13T03:30:44.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:44 smithi161 bash[19769]: cluster 2024-02-13T03:30:43.771115+0000 mon.a (mon.0) 1943 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-02-13T03:30:46.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:45 smithi161 bash[27879]: cluster 2024-02-13T03:30:44.773287+0000 mon.a (mon.0) 1944 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-02-13T03:30:46.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:45 smithi161 bash[27879]: cluster 2024-02-13T03:30:45.319631+0000 mgr.y (mgr.24473) 392 : cluster [DBG] pgmap v509: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:46.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:45 smithi161 bash[19769]: cluster 2024-02-13T03:30:44.773287+0000 mon.a (mon.0) 1944 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-02-13T03:30:46.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:45 smithi161 bash[19769]: cluster 2024-02-13T03:30:45.319631+0000 mgr.y (mgr.24473) 392 : cluster [DBG] pgmap v509: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:46.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:30:46 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:30:46.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:45 smithi183 bash[20824]: cluster 2024-02-13T03:30:44.773287+0000 mon.a (mon.0) 1944 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-02-13T03:30:46.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:45 smithi183 bash[20824]: cluster 2024-02-13T03:30:45.319631+0000 mgr.y (mgr.24473) 392 : cluster [DBG] pgmap v509: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:47.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[27879]: cluster 2024-02-13T03:30:45.777010+0000 mon.a (mon.0) 1945 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-02-13T03:30:47.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[27879]: audit 2024-02-13T03:30:45.779151+0000 mon.c (mon.2) 481 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:47.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[27879]: audit 2024-02-13T03:30:45.780199+0000 mon.a (mon.0) 1946 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:47.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[27879]: audit 2024-02-13T03:30:46.004460+0000 mgr.y (mgr.24473) 393 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:47.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[27879]: audit 2024-02-13T03:30:46.780019+0000 mon.a (mon.0) 1947 : audit [INF] from='client.48317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]': finished 2024-02-13T03:30:47.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[27879]: cluster 2024-02-13T03:30:46.780212+0000 mon.a (mon.0) 1948 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-02-13T03:30:47.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[27879]: audit 2024-02-13T03:30:46.781117+0000 mon.c (mon.2) 482 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:47.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[27879]: audit 2024-02-13T03:30:46.781635+0000 mon.a (mon.0) 1949 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:47.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[19769]: cluster 2024-02-13T03:30:45.777010+0000 mon.a (mon.0) 1945 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-02-13T03:30:47.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[19769]: audit 2024-02-13T03:30:45.779151+0000 mon.c (mon.2) 481 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:47.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[19769]: audit 2024-02-13T03:30:45.780199+0000 mon.a (mon.0) 1946 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:47.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[19769]: audit 2024-02-13T03:30:46.004460+0000 mgr.y (mgr.24473) 393 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:47.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[19769]: audit 2024-02-13T03:30:46.780019+0000 mon.a (mon.0) 1947 : audit [INF] from='client.48317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]': finished 2024-02-13T03:30:47.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[19769]: cluster 2024-02-13T03:30:46.780212+0000 mon.a (mon.0) 1948 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-02-13T03:30:47.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[19769]: audit 2024-02-13T03:30:46.781117+0000 mon.c (mon.2) 482 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:47.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:46 smithi161 bash[19769]: audit 2024-02-13T03:30:46.781635+0000 mon.a (mon.0) 1949 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:47.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:46 smithi183 bash[20824]: cluster 2024-02-13T03:30:45.777010+0000 mon.a (mon.0) 1945 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-02-13T03:30:47.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:46 smithi183 bash[20824]: audit 2024-02-13T03:30:45.779151+0000 mon.c (mon.2) 481 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:46 smithi183 bash[20824]: audit 2024-02-13T03:30:45.780199+0000 mon.a (mon.0) 1946 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:46 smithi183 bash[20824]: audit 2024-02-13T03:30:46.004460+0000 mgr.y (mgr.24473) 393 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:46 smithi183 bash[20824]: audit 2024-02-13T03:30:46.780019+0000 mon.a (mon.0) 1947 : audit [INF] from='client.48317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi161-61726-56"}]': finished 2024-02-13T03:30:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:46 smithi183 bash[20824]: cluster 2024-02-13T03:30:46.780212+0000 mon.a (mon.0) 1948 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-02-13T03:30:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:46 smithi183 bash[20824]: audit 2024-02-13T03:30:46.781117+0000 mon.c (mon.2) 482 : audit [INF] from='client.? v1:172.21.15.161:0/3096267513' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:46 smithi183 bash[20824]: audit 2024-02-13T03:30:46.781635+0000 mon.a (mon.0) 1949 : audit [INF] from='client.48317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]: dispatch 2024-02-13T03:30:48.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:47 smithi161 bash[19769]: cluster 2024-02-13T03:30:47.320552+0000 mgr.y (mgr.24473) 394 : cluster [DBG] pgmap v512: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:30:48.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:47 smithi161 bash[27879]: cluster 2024-02-13T03:30:47.320552+0000 mgr.y (mgr.24473) 394 : cluster [DBG] pgmap v512: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:30:48.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:47 smithi183 bash[20824]: cluster 2024-02-13T03:30:47.320552+0000 mgr.y (mgr.24473) 394 : cluster [DBG] pgmap v512: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:30:49.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[19769]: audit 2024-02-13T03:30:47.782827+0000 mon.a (mon.0) 1950 : audit [INF] from='client.48317 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]': finished 2024-02-13T03:30:49.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[19769]: cluster 2024-02-13T03:30:47.783011+0000 mon.a (mon.0) 1951 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-02-13T03:30:49.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[19769]: audit 2024-02-13T03:30:47.808093+0000 mon.c (mon.2) 483 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:49.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[19769]: audit 2024-02-13T03:30:47.808470+0000 mon.a (mon.0) 1952 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:49.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[19769]: audit 2024-02-13T03:30:47.809482+0000 mon.c (mon.2) 484 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:49.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[19769]: audit 2024-02-13T03:30:47.809821+0000 mon.a (mon.0) 1953 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:49.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[19769]: audit 2024-02-13T03:30:47.810390+0000 mon.c (mon.2) 485 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi161-61726-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:49.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[19769]: audit 2024-02-13T03:30:47.810709+0000 mon.a (mon.0) 1954 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi161-61726-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:49.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[27879]: audit 2024-02-13T03:30:47.782827+0000 mon.a (mon.0) 1950 : audit [INF] from='client.48317 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]': finished 2024-02-13T03:30:49.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[27879]: cluster 2024-02-13T03:30:47.783011+0000 mon.a (mon.0) 1951 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-02-13T03:30:49.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[27879]: audit 2024-02-13T03:30:47.808093+0000 mon.c (mon.2) 483 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:49.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[27879]: audit 2024-02-13T03:30:47.808470+0000 mon.a (mon.0) 1952 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:49.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[27879]: audit 2024-02-13T03:30:47.809482+0000 mon.c (mon.2) 484 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:49.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[27879]: audit 2024-02-13T03:30:47.809821+0000 mon.a (mon.0) 1953 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:49.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[27879]: audit 2024-02-13T03:30:47.810390+0000 mon.c (mon.2) 485 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi161-61726-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:49.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:48 smithi161 bash[27879]: audit 2024-02-13T03:30:47.810709+0000 mon.a (mon.0) 1954 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi161-61726-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:49.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:48 smithi183 bash[20824]: audit 2024-02-13T03:30:47.782827+0000 mon.a (mon.0) 1950 : audit [INF] from='client.48317 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi161-61726-56"}]': finished 2024-02-13T03:30:49.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:48 smithi183 bash[20824]: cluster 2024-02-13T03:30:47.783011+0000 mon.a (mon.0) 1951 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-02-13T03:30:49.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:48 smithi183 bash[20824]: audit 2024-02-13T03:30:47.808093+0000 mon.c (mon.2) 483 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:49.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:48 smithi183 bash[20824]: audit 2024-02-13T03:30:47.808470+0000 mon.a (mon.0) 1952 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:49.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:48 smithi183 bash[20824]: audit 2024-02-13T03:30:47.809482+0000 mon.c (mon.2) 484 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:49.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:48 smithi183 bash[20824]: audit 2024-02-13T03:30:47.809821+0000 mon.a (mon.0) 1953 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:49.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:48 smithi183 bash[20824]: audit 2024-02-13T03:30:47.810390+0000 mon.c (mon.2) 485 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi161-61726-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:49.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:48 smithi183 bash[20824]: audit 2024-02-13T03:30:47.810709+0000 mon.a (mon.0) 1954 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi161-61726-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:49.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:30:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:30:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:30:50.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:49 smithi161 bash[27879]: audit 2024-02-13T03:30:48.801099+0000 mon.a (mon.0) 1955 : audit [INF] from='client.48323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi161-61726-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:50.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:49 smithi161 bash[27879]: cluster 2024-02-13T03:30:48.801355+0000 mon.a (mon.0) 1956 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-02-13T03:30:50.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:49 smithi161 bash[27879]: audit 2024-02-13T03:30:48.802873+0000 mon.c (mon.2) 486 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi161-61726-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:50.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:49 smithi161 bash[27879]: audit 2024-02-13T03:30:48.809666+0000 mon.a (mon.0) 1957 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi161-61726-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:50.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:49 smithi161 bash[27879]: cluster 2024-02-13T03:30:49.321465+0000 mgr.y (mgr.24473) 395 : cluster [DBG] pgmap v515: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:30:50.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:49 smithi161 bash[19769]: audit 2024-02-13T03:30:48.801099+0000 mon.a (mon.0) 1955 : audit [INF] from='client.48323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi161-61726-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:50.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:49 smithi161 bash[19769]: cluster 2024-02-13T03:30:48.801355+0000 mon.a (mon.0) 1956 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-02-13T03:30:50.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:49 smithi161 bash[19769]: audit 2024-02-13T03:30:48.802873+0000 mon.c (mon.2) 486 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi161-61726-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:50.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:49 smithi161 bash[19769]: audit 2024-02-13T03:30:48.809666+0000 mon.a (mon.0) 1957 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi161-61726-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:50.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:49 smithi161 bash[19769]: cluster 2024-02-13T03:30:49.321465+0000 mgr.y (mgr.24473) 395 : cluster [DBG] pgmap v515: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:30:50.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:49 smithi183 bash[20824]: audit 2024-02-13T03:30:48.801099+0000 mon.a (mon.0) 1955 : audit [INF] from='client.48323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi161-61726-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:50.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:49 smithi183 bash[20824]: cluster 2024-02-13T03:30:48.801355+0000 mon.a (mon.0) 1956 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-02-13T03:30:50.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:49 smithi183 bash[20824]: audit 2024-02-13T03:30:48.802873+0000 mon.c (mon.2) 486 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi161-61726-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:50.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:49 smithi183 bash[20824]: audit 2024-02-13T03:30:48.809666+0000 mon.a (mon.0) 1957 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi161-61726-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:50.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:49 smithi183 bash[20824]: cluster 2024-02-13T03:30:49.321465+0000 mgr.y (mgr.24473) 395 : cluster [DBG] pgmap v515: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:30:51.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:50 smithi161 bash[27879]: cluster 2024-02-13T03:30:50.244223+0000 mon.a (mon.0) 1958 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-02-13T03:30:51.098 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:50 smithi161 bash[19769]: cluster 2024-02-13T03:30:50.244223+0000 mon.a (mon.0) 1958 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-02-13T03:30:51.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:50 smithi183 bash[20824]: cluster 2024-02-13T03:30:50.244223+0000 mon.a (mon.0) 1958 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-02-13T03:30:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:30:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:30:51] "GET /metrics HTTP/1.1" 200 186518 "" "Prometheus/2.33.4" 2024-02-13T03:30:52.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:51 smithi161 bash[27879]: audit 2024-02-13T03:30:50.828933+0000 mon.a (mon.0) 1959 : audit [INF] from='client.48323 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi161-61726-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi161-61726-57"}]': finished 2024-02-13T03:30:52.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:51 smithi161 bash[27879]: cluster 2024-02-13T03:30:50.829038+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-02-13T03:30:52.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:51 smithi161 bash[27879]: cluster 2024-02-13T03:30:51.322359+0000 mgr.y (mgr.24473) 396 : cluster [DBG] pgmap v518: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-13T03:30:52.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:51 smithi161 bash[19769]: audit 2024-02-13T03:30:50.828933+0000 mon.a (mon.0) 1959 : audit [INF] from='client.48323 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi161-61726-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi161-61726-57"}]': finished 2024-02-13T03:30:52.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:51 smithi161 bash[19769]: cluster 2024-02-13T03:30:50.829038+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-02-13T03:30:52.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:51 smithi161 bash[19769]: cluster 2024-02-13T03:30:51.322359+0000 mgr.y (mgr.24473) 396 : cluster [DBG] pgmap v518: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-13T03:30:52.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:51 smithi183 bash[20824]: audit 2024-02-13T03:30:50.828933+0000 mon.a (mon.0) 1959 : audit [INF] from='client.48323 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi161-61726-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi161-61726-57"}]': finished 2024-02-13T03:30:52.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:51 smithi183 bash[20824]: cluster 2024-02-13T03:30:50.829038+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-02-13T03:30:52.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:51 smithi183 bash[20824]: cluster 2024-02-13T03:30:51.322359+0000 mgr.y (mgr.24473) 396 : cluster [DBG] pgmap v518: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-13T03:30:53.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:52 smithi161 bash[19769]: cluster 2024-02-13T03:30:51.834674+0000 mon.a (mon.0) 1961 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-02-13T03:30:53.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:52 smithi161 bash[27879]: cluster 2024-02-13T03:30:51.834674+0000 mon.a (mon.0) 1961 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-02-13T03:30:53.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:52 smithi183 bash[20824]: cluster 2024-02-13T03:30:51.834674+0000 mon.a (mon.0) 1961 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-02-13T03:30:54.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[27879]: cluster 2024-02-13T03:30:52.859664+0000 mon.a (mon.0) 1962 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-02-13T03:30:54.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[27879]: audit 2024-02-13T03:30:52.861250+0000 mon.c (mon.2) 487 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:54.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[27879]: audit 2024-02-13T03:30:52.862160+0000 mon.a (mon.0) 1963 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:54.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[27879]: cluster 2024-02-13T03:30:53.323333+0000 mgr.y (mgr.24473) 397 : cluster [DBG] pgmap v521: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-13T03:30:54.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[27879]: audit 2024-02-13T03:30:53.844134+0000 mon.a (mon.0) 1964 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:54.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[27879]: audit 2024-02-13T03:30:53.845173+0000 mon.a (mon.0) 1965 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43"}]: dispatch 2024-02-13T03:30:54.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[27879]: audit 2024-02-13T03:30:53.846481+0000 mon.b (mon.1) 265 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:54.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[27879]: audit 2024-02-13T03:30:53.847649+0000 mon.b (mon.1) 266 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43"}]: dispatch 2024-02-13T03:30:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[19769]: cluster 2024-02-13T03:30:52.859664+0000 mon.a (mon.0) 1962 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-02-13T03:30:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[19769]: audit 2024-02-13T03:30:52.861250+0000 mon.c (mon.2) 487 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[19769]: audit 2024-02-13T03:30:52.862160+0000 mon.a (mon.0) 1963 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[19769]: cluster 2024-02-13T03:30:53.323333+0000 mgr.y (mgr.24473) 397 : cluster [DBG] pgmap v521: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-13T03:30:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[19769]: audit 2024-02-13T03:30:53.844134+0000 mon.a (mon.0) 1964 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[19769]: audit 2024-02-13T03:30:53.845173+0000 mon.a (mon.0) 1965 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43"}]: dispatch 2024-02-13T03:30:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[19769]: audit 2024-02-13T03:30:53.846481+0000 mon.b (mon.1) 265 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:53 smithi161 bash[19769]: audit 2024-02-13T03:30:53.847649+0000 mon.b (mon.1) 266 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43"}]: dispatch 2024-02-13T03:30:54.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:53 smithi183 bash[20824]: cluster 2024-02-13T03:30:52.859664+0000 mon.a (mon.0) 1962 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-02-13T03:30:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:53 smithi183 bash[20824]: audit 2024-02-13T03:30:52.861250+0000 mon.c (mon.2) 487 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:53 smithi183 bash[20824]: audit 2024-02-13T03:30:52.862160+0000 mon.a (mon.0) 1963 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:53 smithi183 bash[20824]: cluster 2024-02-13T03:30:53.323333+0000 mgr.y (mgr.24473) 397 : cluster [DBG] pgmap v521: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-13T03:30:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:53 smithi183 bash[20824]: audit 2024-02-13T03:30:53.844134+0000 mon.a (mon.0) 1964 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:53 smithi183 bash[20824]: audit 2024-02-13T03:30:53.845173+0000 mon.a (mon.0) 1965 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43"}]: dispatch 2024-02-13T03:30:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:53 smithi183 bash[20824]: audit 2024-02-13T03:30:53.846481+0000 mon.b (mon.1) 265 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:30:54.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:53 smithi183 bash[20824]: audit 2024-02-13T03:30:53.847649+0000 mon.b (mon.1) 266 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43"}]: dispatch 2024-02-13T03:30:55.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:55 smithi161 bash[19769]: audit 2024-02-13T03:30:53.866795+0000 mon.a (mon.0) 1966 : audit [INF] from='client.48323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]': finished 2024-02-13T03:30:55.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:55 smithi161 bash[19769]: audit 2024-02-13T03:30:53.866884+0000 mon.a (mon.0) 1967 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43"}]': finished 2024-02-13T03:30:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:55 smithi161 bash[19769]: cluster 2024-02-13T03:30:53.866955+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-02-13T03:30:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:55 smithi161 bash[19769]: audit 2024-02-13T03:30:53.870079+0000 mon.c (mon.2) 488 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:55 smithi161 bash[19769]: audit 2024-02-13T03:30:53.870644+0000 mon.a (mon.0) 1969 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:55 smithi161 bash[27879]: audit 2024-02-13T03:30:53.866795+0000 mon.a (mon.0) 1966 : audit [INF] from='client.48323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]': finished 2024-02-13T03:30:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:55 smithi161 bash[27879]: audit 2024-02-13T03:30:53.866884+0000 mon.a (mon.0) 1967 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43"}]': finished 2024-02-13T03:30:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:55 smithi161 bash[27879]: cluster 2024-02-13T03:30:53.866955+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-02-13T03:30:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:55 smithi161 bash[27879]: audit 2024-02-13T03:30:53.870079+0000 mon.c (mon.2) 488 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:55 smithi161 bash[27879]: audit 2024-02-13T03:30:53.870644+0000 mon.a (mon.0) 1969 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:55.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:55 smithi183 bash[20824]: audit 2024-02-13T03:30:53.866795+0000 mon.a (mon.0) 1966 : audit [INF] from='client.48323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi161-61726-57"}]': finished 2024-02-13T03:30:55.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:55 smithi183 bash[20824]: audit 2024-02-13T03:30:53.866884+0000 mon.a (mon.0) 1967 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-43"}]': finished 2024-02-13T03:30:55.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:55 smithi183 bash[20824]: cluster 2024-02-13T03:30:53.866955+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-02-13T03:30:55.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:55 smithi183 bash[20824]: audit 2024-02-13T03:30:53.870079+0000 mon.c (mon.2) 488 : audit [INF] from='client.? v1:172.21.15.161:0/1915617383' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:55 smithi183 bash[20824]: audit 2024-02-13T03:30:53.870644+0000 mon.a (mon.0) 1969 : audit [INF] from='client.48323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]: dispatch 2024-02-13T03:30:56.326 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:30:56 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:30:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[27879]: audit 2024-02-13T03:30:54.872134+0000 mon.a (mon.0) 1970 : audit [INF] from='client.48323 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]': finished 2024-02-13T03:30:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[27879]: cluster 2024-02-13T03:30:54.872318+0000 mon.a (mon.0) 1971 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-02-13T03:30:56.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[27879]: audit 2024-02-13T03:30:55.321667+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:30:56.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[27879]: audit 2024-02-13T03:30:55.323631+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:30:56.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[27879]: cluster 2024-02-13T03:30:55.323799+0000 mgr.y (mgr.24473) 398 : cluster [DBG] pgmap v524: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:56.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[27879]: audit 2024-02-13T03:30:55.324108+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:56.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[27879]: audit 2024-02-13T03:30:55.876880+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:56.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[27879]: cluster 2024-02-13T03:30:55.876995+0000 mon.a (mon.0) 1976 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-02-13T03:30:56.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[27879]: audit 2024-02-13T03:30:55.880811+0000 mon.a (mon.0) 1977 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:56.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[27879]: audit 2024-02-13T03:30:55.881665+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi161-61726-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:30:56.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[27879]: audit 2024-02-13T03:30:55.882437+0000 mon.b (mon.1) 267 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:56.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[19769]: audit 2024-02-13T03:30:54.872134+0000 mon.a (mon.0) 1970 : audit [INF] from='client.48323 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]': finished 2024-02-13T03:30:56.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[19769]: cluster 2024-02-13T03:30:54.872318+0000 mon.a (mon.0) 1971 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-02-13T03:30:56.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[19769]: audit 2024-02-13T03:30:55.321667+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:30:56.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[19769]: audit 2024-02-13T03:30:55.323631+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:30:56.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[19769]: cluster 2024-02-13T03:30:55.323799+0000 mgr.y (mgr.24473) 398 : cluster [DBG] pgmap v524: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:56.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[19769]: audit 2024-02-13T03:30:55.324108+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:56.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[19769]: audit 2024-02-13T03:30:55.876880+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:56.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[19769]: cluster 2024-02-13T03:30:55.876995+0000 mon.a (mon.0) 1976 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-02-13T03:30:56.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[19769]: audit 2024-02-13T03:30:55.880811+0000 mon.a (mon.0) 1977 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:56.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[19769]: audit 2024-02-13T03:30:55.881665+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi161-61726-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:30:56.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:56 smithi161 bash[19769]: audit 2024-02-13T03:30:55.882437+0000 mon.b (mon.1) 267 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:56 smithi183 bash[20824]: audit 2024-02-13T03:30:54.872134+0000 mon.a (mon.0) 1970 : audit [INF] from='client.48323 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi161-61726-57"}]': finished 2024-02-13T03:30:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:56 smithi183 bash[20824]: cluster 2024-02-13T03:30:54.872318+0000 mon.a (mon.0) 1971 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-02-13T03:30:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:56 smithi183 bash[20824]: audit 2024-02-13T03:30:55.321667+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:30:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:56 smithi183 bash[20824]: audit 2024-02-13T03:30:55.323631+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:30:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:56 smithi183 bash[20824]: cluster 2024-02-13T03:30:55.323799+0000 mgr.y (mgr.24473) 398 : cluster [DBG] pgmap v524: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:30:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:56 smithi183 bash[20824]: audit 2024-02-13T03:30:55.324108+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:30:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:56 smithi183 bash[20824]: audit 2024-02-13T03:30:55.876880+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:30:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:56 smithi183 bash[20824]: cluster 2024-02-13T03:30:55.876995+0000 mon.a (mon.0) 1976 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-02-13T03:30:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:56 smithi183 bash[20824]: audit 2024-02-13T03:30:55.880811+0000 mon.a (mon.0) 1977 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:56 smithi183 bash[20824]: audit 2024-02-13T03:30:55.881665+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi161-61726-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:30:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:56 smithi183 bash[20824]: audit 2024-02-13T03:30:55.882437+0000 mon.b (mon.1) 267 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:30:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:57 smithi161 bash[27879]: audit 2024-02-13T03:30:56.007682+0000 mgr.y (mgr.24473) 399 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:57 smithi161 bash[27879]: audit 2024-02-13T03:30:56.878477+0000 mon.a (mon.0) 1979 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:57 smithi161 bash[27879]: audit 2024-02-13T03:30:56.977614+0000 mon.b (mon.1) 268 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:57 smithi161 bash[27879]: cluster 2024-02-13T03:30:57.298671+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-02-13T03:30:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:57 smithi161 bash[27879]: audit 2024-02-13T03:30:57.303395+0000 mon.a (mon.0) 1981 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:57 smithi161 bash[19769]: audit 2024-02-13T03:30:56.007682+0000 mgr.y (mgr.24473) 399 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:57 smithi161 bash[19769]: audit 2024-02-13T03:30:56.878477+0000 mon.a (mon.0) 1979 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:57 smithi161 bash[19769]: audit 2024-02-13T03:30:56.977614+0000 mon.b (mon.1) 268 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:57 smithi161 bash[19769]: cluster 2024-02-13T03:30:57.298671+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-02-13T03:30:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:57 smithi161 bash[19769]: audit 2024-02-13T03:30:57.303395+0000 mon.a (mon.0) 1981 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:57 smithi183 bash[20824]: audit 2024-02-13T03:30:56.007682+0000 mgr.y (mgr.24473) 399 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:30:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:57 smithi183 bash[20824]: audit 2024-02-13T03:30:56.878477+0000 mon.a (mon.0) 1979 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:30:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:57 smithi183 bash[20824]: audit 2024-02-13T03:30:56.977614+0000 mon.b (mon.1) 268 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:57 smithi183 bash[20824]: cluster 2024-02-13T03:30:57.298671+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-02-13T03:30:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:57 smithi183 bash[20824]: audit 2024-02-13T03:30:57.303395+0000 mon.a (mon.0) 1981 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:30:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:58 smithi161 bash[27879]: cluster 2024-02-13T03:30:57.324508+0000 mgr.y (mgr.24473) 400 : cluster [DBG] pgmap v527: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:58 smithi161 bash[27879]: audit 2024-02-13T03:30:57.882116+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi161-61726-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]': finished 2024-02-13T03:30:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:58 smithi161 bash[27879]: audit 2024-02-13T03:30:57.882325+0000 mon.a (mon.0) 1983 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:58 smithi161 bash[27879]: cluster 2024-02-13T03:30:57.882471+0000 mon.a (mon.0) 1984 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-02-13T03:30:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:58 smithi161 bash[27879]: audit 2024-02-13T03:30:57.888242+0000 mon.a (mon.0) 1985 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:30:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:58 smithi161 bash[27879]: audit 2024-02-13T03:30:57.888992+0000 mon.b (mon.1) 269 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:30:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:58 smithi161 bash[19769]: cluster 2024-02-13T03:30:57.324508+0000 mgr.y (mgr.24473) 400 : cluster [DBG] pgmap v527: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:58 smithi161 bash[19769]: audit 2024-02-13T03:30:57.882116+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi161-61726-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]': finished 2024-02-13T03:30:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:58 smithi161 bash[19769]: audit 2024-02-13T03:30:57.882325+0000 mon.a (mon.0) 1983 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:58 smithi161 bash[19769]: cluster 2024-02-13T03:30:57.882471+0000 mon.a (mon.0) 1984 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-02-13T03:30:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:58 smithi161 bash[19769]: audit 2024-02-13T03:30:57.888242+0000 mon.a (mon.0) 1985 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:30:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:58 smithi161 bash[19769]: audit 2024-02-13T03:30:57.888992+0000 mon.b (mon.1) 269 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:30:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:58 smithi183 bash[20824]: cluster 2024-02-13T03:30:57.324508+0000 mgr.y (mgr.24473) 400 : cluster [DBG] pgmap v527: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:30:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:58 smithi183 bash[20824]: audit 2024-02-13T03:30:57.882116+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi161-61726-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]': finished 2024-02-13T03:30:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:58 smithi183 bash[20824]: audit 2024-02-13T03:30:57.882325+0000 mon.a (mon.0) 1983 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:30:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:58 smithi183 bash[20824]: cluster 2024-02-13T03:30:57.882471+0000 mon.a (mon.0) 1984 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-02-13T03:30:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:58 smithi183 bash[20824]: audit 2024-02-13T03:30:57.888242+0000 mon.a (mon.0) 1985 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:30:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:58 smithi183 bash[20824]: audit 2024-02-13T03:30:57.888992+0000 mon.b (mon.1) 269 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:30:59.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:30:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:30:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:31:00.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:59 smithi161 bash[27879]: audit 2024-02-13T03:30:58.884918+0000 mon.a (mon.0) 1986 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-45"}]': finished 2024-02-13T03:31:00.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:59 smithi161 bash[27879]: cluster 2024-02-13T03:30:58.885113+0000 mon.a (mon.0) 1987 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-02-13T03:31:00.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:59 smithi161 bash[27879]: audit 2024-02-13T03:30:58.889771+0000 mon.b (mon.1) 270 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-45", "mode": "writeback"}]: dispatch 2024-02-13T03:31:00.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:59 smithi161 bash[27879]: audit 2024-02-13T03:30:58.892039+0000 mon.a (mon.0) 1988 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-45", "mode": "writeback"}]: dispatch 2024-02-13T03:31:00.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:30:59 smithi161 bash[27879]: cluster 2024-02-13T03:30:59.325452+0000 mgr.y (mgr.24473) 401 : cluster [DBG] pgmap v530: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:59 smithi161 bash[19769]: audit 2024-02-13T03:30:58.884918+0000 mon.a (mon.0) 1986 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-45"}]': finished 2024-02-13T03:31:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:59 smithi161 bash[19769]: cluster 2024-02-13T03:30:58.885113+0000 mon.a (mon.0) 1987 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-02-13T03:31:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:59 smithi161 bash[19769]: audit 2024-02-13T03:30:58.889771+0000 mon.b (mon.1) 270 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-45", "mode": "writeback"}]: dispatch 2024-02-13T03:31:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:59 smithi161 bash[19769]: audit 2024-02-13T03:30:58.892039+0000 mon.a (mon.0) 1988 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-45", "mode": "writeback"}]: dispatch 2024-02-13T03:31:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:30:59 smithi161 bash[19769]: cluster 2024-02-13T03:30:59.325452+0000 mgr.y (mgr.24473) 401 : cluster [DBG] pgmap v530: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:00.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:59 smithi183 bash[20824]: audit 2024-02-13T03:30:58.884918+0000 mon.a (mon.0) 1986 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-45"}]': finished 2024-02-13T03:31:00.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:59 smithi183 bash[20824]: cluster 2024-02-13T03:30:58.885113+0000 mon.a (mon.0) 1987 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-02-13T03:31:00.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:59 smithi183 bash[20824]: audit 2024-02-13T03:30:58.889771+0000 mon.b (mon.1) 270 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-45", "mode": "writeback"}]: dispatch 2024-02-13T03:31:00.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:59 smithi183 bash[20824]: audit 2024-02-13T03:30:58.892039+0000 mon.a (mon.0) 1988 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-45", "mode": "writeback"}]: dispatch 2024-02-13T03:31:00.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:30:59 smithi183 bash[20824]: cluster 2024-02-13T03:30:59.325452+0000 mgr.y (mgr.24473) 401 : cluster [DBG] pgmap v530: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:01.176 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:31:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:31:01] "GET /metrics HTTP/1.1" 200 186518 "" "Prometheus/2.33.4" 2024-02-13T03:31:01.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[27879]: cluster 2024-02-13T03:30:59.878953+0000 mon.a (mon.0) 1989 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:31:01.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[27879]: audit 2024-02-13T03:30:59.887213+0000 mon.a (mon.0) 1990 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-45", "mode": "writeback"}]': finished 2024-02-13T03:31:01.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[27879]: cluster 2024-02-13T03:30:59.887410+0000 mon.a (mon.0) 1991 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-02-13T03:31:01.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[27879]: audit 2024-02-13T03:30:59.889308+0000 mon.a (mon.0) 1992 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:31:01.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[27879]: audit 2024-02-13T03:30:59.894621+0000 mon.b (mon.1) 271 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:31:01.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[27879]: audit 2024-02-13T03:30:59.897352+0000 mon.a (mon.0) 1993 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:31:01.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[27879]: audit 2024-02-13T03:31:00.890572+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]': finished 2024-02-13T03:31:01.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[27879]: audit 2024-02-13T03:31:00.890688+0000 mon.a (mon.0) 1995 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:31:01.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[27879]: cluster 2024-02-13T03:31:00.890799+0000 mon.a (mon.0) 1996 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-02-13T03:31:01.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[27879]: audit 2024-02-13T03:31:00.891653+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:31:01.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[19769]: cluster 2024-02-13T03:30:59.878953+0000 mon.a (mon.0) 1989 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:31:01.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[19769]: audit 2024-02-13T03:30:59.887213+0000 mon.a (mon.0) 1990 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-45", "mode": "writeback"}]': finished 2024-02-13T03:31:01.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[19769]: cluster 2024-02-13T03:30:59.887410+0000 mon.a (mon.0) 1991 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-02-13T03:31:01.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[19769]: audit 2024-02-13T03:30:59.889308+0000 mon.a (mon.0) 1992 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:31:01.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[19769]: audit 2024-02-13T03:30:59.894621+0000 mon.b (mon.1) 271 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:31:01.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[19769]: audit 2024-02-13T03:30:59.897352+0000 mon.a (mon.0) 1993 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:31:01.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[19769]: audit 2024-02-13T03:31:00.890572+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]': finished 2024-02-13T03:31:01.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[19769]: audit 2024-02-13T03:31:00.890688+0000 mon.a (mon.0) 1995 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:31:01.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[19769]: cluster 2024-02-13T03:31:00.890799+0000 mon.a (mon.0) 1996 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-02-13T03:31:01.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:00 smithi161 bash[19769]: audit 2024-02-13T03:31:00.891653+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:31:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:00 smithi183 bash[20824]: cluster 2024-02-13T03:30:59.878953+0000 mon.a (mon.0) 1989 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:31:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:00 smithi183 bash[20824]: audit 2024-02-13T03:30:59.887213+0000 mon.a (mon.0) 1990 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-45", "mode": "writeback"}]': finished 2024-02-13T03:31:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:00 smithi183 bash[20824]: cluster 2024-02-13T03:30:59.887410+0000 mon.a (mon.0) 1991 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-02-13T03:31:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:00 smithi183 bash[20824]: audit 2024-02-13T03:30:59.889308+0000 mon.a (mon.0) 1992 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:31:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:00 smithi183 bash[20824]: audit 2024-02-13T03:30:59.894621+0000 mon.b (mon.1) 271 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:31:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:00 smithi183 bash[20824]: audit 2024-02-13T03:30:59.897352+0000 mon.a (mon.0) 1993 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:31:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:00 smithi183 bash[20824]: audit 2024-02-13T03:31:00.890572+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi161-61726-58"}]': finished 2024-02-13T03:31:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:00 smithi183 bash[20824]: audit 2024-02-13T03:31:00.890688+0000 mon.a (mon.0) 1995 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:31:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:00 smithi183 bash[20824]: cluster 2024-02-13T03:31:00.890799+0000 mon.a (mon.0) 1996 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-02-13T03:31:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:00 smithi183 bash[20824]: audit 2024-02-13T03:31:00.891653+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi161-61726-58"}]: dispatch 2024-02-13T03:31:02.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:01 smithi161 bash[19769]: audit 2024-02-13T03:31:00.893369+0000 mon.a (mon.0) 1998 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:31:02.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:01 smithi161 bash[19769]: audit 2024-02-13T03:31:00.894791+0000 mon.b (mon.1) 272 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:31:02.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:01 smithi161 bash[19769]: cluster 2024-02-13T03:31:01.326380+0000 mgr.y (mgr.24473) 402 : cluster [DBG] pgmap v533: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-13T03:31:02.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:01 smithi161 bash[27879]: audit 2024-02-13T03:31:00.893369+0000 mon.a (mon.0) 1998 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:31:02.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:01 smithi161 bash[27879]: audit 2024-02-13T03:31:00.894791+0000 mon.b (mon.1) 272 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:31:02.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:01 smithi161 bash[27879]: cluster 2024-02-13T03:31:01.326380+0000 mgr.y (mgr.24473) 402 : cluster [DBG] pgmap v533: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-13T03:31:02.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:01 smithi183 bash[20824]: audit 2024-02-13T03:31:00.893369+0000 mon.a (mon.0) 1998 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:31:02.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:01 smithi183 bash[20824]: audit 2024-02-13T03:31:00.894791+0000 mon.b (mon.1) 272 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:31:02.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:01 smithi183 bash[20824]: cluster 2024-02-13T03:31:01.326380+0000 mgr.y (mgr.24473) 402 : cluster [DBG] pgmap v533: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-13T03:31:03.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[27879]: audit 2024-02-13T03:31:01.897261+0000 mon.a (mon.0) 1999 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi161-61726-58"}]': finished 2024-02-13T03:31:03.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[27879]: audit 2024-02-13T03:31:01.897411+0000 mon.a (mon.0) 2000 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:31:03.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[27879]: cluster 2024-02-13T03:31:01.897558+0000 mon.a (mon.0) 2001 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-02-13T03:31:03.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[27879]: audit 2024-02-13T03:31:01.899328+0000 mon.a (mon.0) 2002 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:31:03.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[27879]: audit 2024-02-13T03:31:01.901259+0000 mon.b (mon.1) 273 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:31:03.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[27879]: cluster 2024-02-13T03:31:02.888848+0000 mon.a (mon.0) 2003 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:31:03.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[27879]: audit 2024-02-13T03:31:02.900783+0000 mon.a (mon.0) 2004 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:31:03.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[19769]: audit 2024-02-13T03:31:01.897261+0000 mon.a (mon.0) 1999 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi161-61726-58"}]': finished 2024-02-13T03:31:03.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[19769]: audit 2024-02-13T03:31:01.897411+0000 mon.a (mon.0) 2000 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:31:03.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[19769]: cluster 2024-02-13T03:31:01.897558+0000 mon.a (mon.0) 2001 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-02-13T03:31:03.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[19769]: audit 2024-02-13T03:31:01.899328+0000 mon.a (mon.0) 2002 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:31:03.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[19769]: audit 2024-02-13T03:31:01.901259+0000 mon.b (mon.1) 273 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:31:03.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[19769]: cluster 2024-02-13T03:31:02.888848+0000 mon.a (mon.0) 2003 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:31:03.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:02 smithi161 bash[19769]: audit 2024-02-13T03:31:02.900783+0000 mon.a (mon.0) 2004 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:31:03.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:02 smithi183 bash[20824]: audit 2024-02-13T03:31:01.897261+0000 mon.a (mon.0) 1999 : audit [INF] from='client.? v1:172.21.15.161:0/647962299' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi161-61726-58"}]': finished 2024-02-13T03:31:03.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:02 smithi183 bash[20824]: audit 2024-02-13T03:31:01.897411+0000 mon.a (mon.0) 2000 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:31:03.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:02 smithi183 bash[20824]: cluster 2024-02-13T03:31:01.897558+0000 mon.a (mon.0) 2001 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-02-13T03:31:03.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:02 smithi183 bash[20824]: audit 2024-02-13T03:31:01.899328+0000 mon.a (mon.0) 2002 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:31:03.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:02 smithi183 bash[20824]: audit 2024-02-13T03:31:01.901259+0000 mon.b (mon.1) 273 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:31:03.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:02 smithi183 bash[20824]: cluster 2024-02-13T03:31:02.888848+0000 mon.a (mon.0) 2003 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:31:03.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:02 smithi183 bash[20824]: audit 2024-02-13T03:31:02.900783+0000 mon.a (mon.0) 2004 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:31:04.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[27879]: cluster 2024-02-13T03:31:02.901023+0000 mon.a (mon.0) 2005 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-02-13T03:31:04.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[27879]: audit 2024-02-13T03:31:02.902852+0000 mon.c (mon.2) 489 : audit [INF] from='client.? v1:172.21.15.161:0/1164360435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:04.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[27879]: audit 2024-02-13T03:31:02.907949+0000 mon.a (mon.0) 2006 : audit [INF] from='client.48332 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:04.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[27879]: audit 2024-02-13T03:31:02.908437+0000 mon.a (mon.0) 2007 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:31:04.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[27879]: audit 2024-02-13T03:31:02.908700+0000 mon.b (mon.1) 274 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:31:04.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[27879]: cluster 2024-02-13T03:31:03.327307+0000 mgr.y (mgr.24473) 403 : cluster [DBG] pgmap v536: 260 pgs: 32 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-13T03:31:04.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[27879]: audit 2024-02-13T03:31:03.899555+0000 mon.a (mon.0) 2008 : audit [INF] from='client.48332 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:04.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[27879]: audit 2024-02-13T03:31:03.899688+0000 mon.a (mon.0) 2009 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-13T03:31:04.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[27879]: cluster 2024-02-13T03:31:03.899850+0000 mon.a (mon.0) 2010 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-02-13T03:31:04.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[27879]: audit 2024-02-13T03:31:03.903521+0000 mon.a (mon.0) 2011 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-13T03:31:04.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[27879]: audit 2024-02-13T03:31:03.905663+0000 mon.b (mon.1) 275 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-13T03:31:04.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[19769]: cluster 2024-02-13T03:31:02.901023+0000 mon.a (mon.0) 2005 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-02-13T03:31:04.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[19769]: audit 2024-02-13T03:31:02.902852+0000 mon.c (mon.2) 489 : audit [INF] from='client.? v1:172.21.15.161:0/1164360435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:04.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[19769]: audit 2024-02-13T03:31:02.907949+0000 mon.a (mon.0) 2006 : audit [INF] from='client.48332 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:04.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[19769]: audit 2024-02-13T03:31:02.908437+0000 mon.a (mon.0) 2007 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:31:04.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[19769]: audit 2024-02-13T03:31:02.908700+0000 mon.b (mon.1) 274 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:31:04.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[19769]: cluster 2024-02-13T03:31:03.327307+0000 mgr.y (mgr.24473) 403 : cluster [DBG] pgmap v536: 260 pgs: 32 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-13T03:31:04.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[19769]: audit 2024-02-13T03:31:03.899555+0000 mon.a (mon.0) 2008 : audit [INF] from='client.48332 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:04.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[19769]: audit 2024-02-13T03:31:03.899688+0000 mon.a (mon.0) 2009 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-13T03:31:04.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[19769]: cluster 2024-02-13T03:31:03.899850+0000 mon.a (mon.0) 2010 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-02-13T03:31:04.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[19769]: audit 2024-02-13T03:31:03.903521+0000 mon.a (mon.0) 2011 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-13T03:31:04.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:03 smithi161 bash[19769]: audit 2024-02-13T03:31:03.905663+0000 mon.b (mon.1) 275 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-13T03:31:04.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:03 smithi183 bash[20824]: cluster 2024-02-13T03:31:02.901023+0000 mon.a (mon.0) 2005 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-02-13T03:31:04.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:03 smithi183 bash[20824]: audit 2024-02-13T03:31:02.902852+0000 mon.c (mon.2) 489 : audit [INF] from='client.? v1:172.21.15.161:0/1164360435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:03 smithi183 bash[20824]: audit 2024-02-13T03:31:02.907949+0000 mon.a (mon.0) 2006 : audit [INF] from='client.48332 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:03 smithi183 bash[20824]: audit 2024-02-13T03:31:02.908437+0000 mon.a (mon.0) 2007 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:31:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:03 smithi183 bash[20824]: audit 2024-02-13T03:31:02.908700+0000 mon.b (mon.1) 274 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:31:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:03 smithi183 bash[20824]: cluster 2024-02-13T03:31:03.327307+0000 mgr.y (mgr.24473) 403 : cluster [DBG] pgmap v536: 260 pgs: 32 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-13T03:31:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:03 smithi183 bash[20824]: audit 2024-02-13T03:31:03.899555+0000 mon.a (mon.0) 2008 : audit [INF] from='client.48332 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi161-61726-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:03 smithi183 bash[20824]: audit 2024-02-13T03:31:03.899688+0000 mon.a (mon.0) 2009 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-13T03:31:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:03 smithi183 bash[20824]: cluster 2024-02-13T03:31:03.899850+0000 mon.a (mon.0) 2010 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-02-13T03:31:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:03 smithi183 bash[20824]: audit 2024-02-13T03:31:03.903521+0000 mon.a (mon.0) 2011 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-13T03:31:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:03 smithi183 bash[20824]: audit 2024-02-13T03:31:03.905663+0000 mon.b (mon.1) 275 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-13T03:31:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[27879]: audit 2024-02-13T03:31:04.904138+0000 mon.a (mon.0) 2012 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-13T03:31:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[27879]: cluster 2024-02-13T03:31:04.904681+0000 mon.a (mon.0) 2013 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-02-13T03:31:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[27879]: audit 2024-02-13T03:31:04.906282+0000 mon.a (mon.0) 2014 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-13T03:31:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[27879]: audit 2024-02-13T03:31:04.908216+0000 mon.b (mon.1) 276 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-13T03:31:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[27879]: audit 2024-02-13T03:31:04.924596+0000 mon.c (mon.2) 490 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[27879]: audit 2024-02-13T03:31:04.924909+0000 mon.a (mon.0) 2015 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[27879]: audit 2024-02-13T03:31:04.925854+0000 mon.c (mon.2) 491 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[27879]: audit 2024-02-13T03:31:04.926105+0000 mon.a (mon.0) 2016 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[27879]: audit 2024-02-13T03:31:04.926557+0000 mon.c (mon.2) 492 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi161-61726-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[27879]: audit 2024-02-13T03:31:04.926795+0000 mon.a (mon.0) 2017 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi161-61726-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[27879]: cluster 2024-02-13T03:31:05.328468+0000 mgr.y (mgr.24473) 404 : cluster [DBG] pgmap v539: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:06.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[19769]: audit 2024-02-13T03:31:04.904138+0000 mon.a (mon.0) 2012 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-13T03:31:06.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[19769]: cluster 2024-02-13T03:31:04.904681+0000 mon.a (mon.0) 2013 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-02-13T03:31:06.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[19769]: audit 2024-02-13T03:31:04.906282+0000 mon.a (mon.0) 2014 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-13T03:31:06.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[19769]: audit 2024-02-13T03:31:04.908216+0000 mon.b (mon.1) 276 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-13T03:31:06.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[19769]: audit 2024-02-13T03:31:04.924596+0000 mon.c (mon.2) 490 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:06.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[19769]: audit 2024-02-13T03:31:04.924909+0000 mon.a (mon.0) 2015 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:06.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[19769]: audit 2024-02-13T03:31:04.925854+0000 mon.c (mon.2) 491 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:06.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[19769]: audit 2024-02-13T03:31:04.926105+0000 mon.a (mon.0) 2016 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:06.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[19769]: audit 2024-02-13T03:31:04.926557+0000 mon.c (mon.2) 492 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi161-61726-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:06.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[19769]: audit 2024-02-13T03:31:04.926795+0000 mon.a (mon.0) 2017 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi161-61726-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:06.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:05 smithi161 bash[19769]: cluster 2024-02-13T03:31:05.328468+0000 mgr.y (mgr.24473) 404 : cluster [DBG] pgmap v539: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:06.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:31:06 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:31:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:05 smithi183 bash[20824]: audit 2024-02-13T03:31:04.904138+0000 mon.a (mon.0) 2012 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-13T03:31:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:05 smithi183 bash[20824]: cluster 2024-02-13T03:31:04.904681+0000 mon.a (mon.0) 2013 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-02-13T03:31:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:05 smithi183 bash[20824]: audit 2024-02-13T03:31:04.906282+0000 mon.a (mon.0) 2014 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-13T03:31:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:05 smithi183 bash[20824]: audit 2024-02-13T03:31:04.908216+0000 mon.b (mon.1) 276 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-13T03:31:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:05 smithi183 bash[20824]: audit 2024-02-13T03:31:04.924596+0000 mon.c (mon.2) 490 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:05 smithi183 bash[20824]: audit 2024-02-13T03:31:04.924909+0000 mon.a (mon.0) 2015 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:05 smithi183 bash[20824]: audit 2024-02-13T03:31:04.925854+0000 mon.c (mon.2) 491 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:05 smithi183 bash[20824]: audit 2024-02-13T03:31:04.926105+0000 mon.a (mon.0) 2016 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:05 smithi183 bash[20824]: audit 2024-02-13T03:31:04.926557+0000 mon.c (mon.2) 492 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi161-61726-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:06.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:05 smithi183 bash[20824]: audit 2024-02-13T03:31:04.926795+0000 mon.a (mon.0) 2017 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi161-61726-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:06.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:05 smithi183 bash[20824]: cluster 2024-02-13T03:31:05.328468+0000 mgr.y (mgr.24473) 404 : cluster [DBG] pgmap v539: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:07.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[27879]: audit 2024-02-13T03:31:05.905693+0000 mon.a (mon.0) 2018 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-13T03:31:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[27879]: audit 2024-02-13T03:31:05.905995+0000 mon.a (mon.0) 2019 : audit [INF] from='client.48338 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi161-61726-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[27879]: cluster 2024-02-13T03:31:05.906245+0000 mon.a (mon.0) 2020 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-02-13T03:31:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[27879]: audit 2024-02-13T03:31:05.910846+0000 mon.c (mon.2) 493 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi161-61726-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[27879]: audit 2024-02-13T03:31:05.916519+0000 mon.a (mon.0) 2021 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi161-61726-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[27879]: audit 2024-02-13T03:31:05.916576+0000 mon.c (mon.2) 494 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:31:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[27879]: audit 2024-02-13T03:31:05.960994+0000 mon.a (mon.0) 2022 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[27879]: audit 2024-02-13T03:31:05.963310+0000 mon.b (mon.1) 277 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[27879]: audit 2024-02-13T03:31:06.019065+0000 mgr.y (mgr.24473) 405 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[27879]: audit 2024-02-13T03:31:06.218148+0000 mon.c (mon.2) 495 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:31:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[27879]: audit 2024-02-13T03:31:06.219551+0000 mon.c (mon.2) 496 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:31:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[27879]: audit 2024-02-13T03:31:06.226394+0000 mon.a (mon.0) 2023 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:31:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[19769]: audit 2024-02-13T03:31:05.905693+0000 mon.a (mon.0) 2018 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-13T03:31:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[19769]: audit 2024-02-13T03:31:05.905995+0000 mon.a (mon.0) 2019 : audit [INF] from='client.48338 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi161-61726-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[19769]: cluster 2024-02-13T03:31:05.906245+0000 mon.a (mon.0) 2020 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-02-13T03:31:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[19769]: audit 2024-02-13T03:31:05.910846+0000 mon.c (mon.2) 493 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi161-61726-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[19769]: audit 2024-02-13T03:31:05.916519+0000 mon.a (mon.0) 2021 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi161-61726-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[19769]: audit 2024-02-13T03:31:05.916576+0000 mon.c (mon.2) 494 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:31:07.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[19769]: audit 2024-02-13T03:31:05.960994+0000 mon.a (mon.0) 2022 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:07.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[19769]: audit 2024-02-13T03:31:05.963310+0000 mon.b (mon.1) 277 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:07.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[19769]: audit 2024-02-13T03:31:06.019065+0000 mgr.y (mgr.24473) 405 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:07.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[19769]: audit 2024-02-13T03:31:06.218148+0000 mon.c (mon.2) 495 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:31:07.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[19769]: audit 2024-02-13T03:31:06.219551+0000 mon.c (mon.2) 496 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:31:07.179 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:06 smithi161 bash[19769]: audit 2024-02-13T03:31:06.226394+0000 mon.a (mon.0) 2023 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:31:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:06 smithi183 bash[20824]: audit 2024-02-13T03:31:05.905693+0000 mon.a (mon.0) 2018 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-13T03:31:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:06 smithi183 bash[20824]: audit 2024-02-13T03:31:05.905995+0000 mon.a (mon.0) 2019 : audit [INF] from='client.48338 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi161-61726-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:06 smithi183 bash[20824]: cluster 2024-02-13T03:31:05.906245+0000 mon.a (mon.0) 2020 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-02-13T03:31:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:06 smithi183 bash[20824]: audit 2024-02-13T03:31:05.910846+0000 mon.c (mon.2) 493 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi161-61726-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:06 smithi183 bash[20824]: audit 2024-02-13T03:31:05.916519+0000 mon.a (mon.0) 2021 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi161-61726-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:06 smithi183 bash[20824]: audit 2024-02-13T03:31:05.916576+0000 mon.c (mon.2) 494 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:31:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:06 smithi183 bash[20824]: audit 2024-02-13T03:31:05.960994+0000 mon.a (mon.0) 2022 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:06 smithi183 bash[20824]: audit 2024-02-13T03:31:05.963310+0000 mon.b (mon.1) 277 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:06 smithi183 bash[20824]: audit 2024-02-13T03:31:06.019065+0000 mgr.y (mgr.24473) 405 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:06 smithi183 bash[20824]: audit 2024-02-13T03:31:06.218148+0000 mon.c (mon.2) 495 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:31:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:06 smithi183 bash[20824]: audit 2024-02-13T03:31:06.219551+0000 mon.c (mon.2) 496 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:31:07.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:06 smithi183 bash[20824]: audit 2024-02-13T03:31:06.226394+0000 mon.a (mon.0) 2023 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:31:08.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:07 smithi183 bash[20824]: cluster 2024-02-13T03:31:07.329276+0000 mgr.y (mgr.24473) 406 : cluster [DBG] pgmap v542: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:07 smithi183 bash[20824]: audit 2024-02-13T03:31:07.348963+0000 mon.a (mon.0) 2024 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:31:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:07 smithi183 bash[20824]: cluster 2024-02-13T03:31:07.349150+0000 mon.a (mon.0) 2025 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-02-13T03:31:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:07 smithi183 bash[20824]: audit 2024-02-13T03:31:07.353094+0000 mon.a (mon.0) 2026 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:31:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:07 smithi183 bash[20824]: audit 2024-02-13T03:31:07.353259+0000 mon.b (mon.1) 278 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:31:08.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:07 smithi161 bash[27879]: cluster 2024-02-13T03:31:07.329276+0000 mgr.y (mgr.24473) 406 : cluster [DBG] pgmap v542: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:08.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:07 smithi161 bash[27879]: audit 2024-02-13T03:31:07.348963+0000 mon.a (mon.0) 2024 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:31:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:07 smithi161 bash[27879]: cluster 2024-02-13T03:31:07.349150+0000 mon.a (mon.0) 2025 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-02-13T03:31:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:07 smithi161 bash[27879]: audit 2024-02-13T03:31:07.353094+0000 mon.a (mon.0) 2026 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:31:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:07 smithi161 bash[27879]: audit 2024-02-13T03:31:07.353259+0000 mon.b (mon.1) 278 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:31:08.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:07 smithi161 bash[19769]: cluster 2024-02-13T03:31:07.329276+0000 mgr.y (mgr.24473) 406 : cluster [DBG] pgmap v542: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:08.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:07 smithi161 bash[19769]: audit 2024-02-13T03:31:07.348963+0000 mon.a (mon.0) 2024 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:31:08.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:07 smithi161 bash[19769]: cluster 2024-02-13T03:31:07.349150+0000 mon.a (mon.0) 2025 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-02-13T03:31:08.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:07 smithi161 bash[19769]: audit 2024-02-13T03:31:07.353094+0000 mon.a (mon.0) 2026 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:31:08.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:07 smithi161 bash[19769]: audit 2024-02-13T03:31:07.353259+0000 mon.b (mon.1) 278 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:31:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[27879]: audit 2024-02-13T03:31:07.945612+0000 mon.a (mon.0) 2027 : audit [INF] from='client.48338 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi161-61726-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi161-61726-60"}]': finished 2024-02-13T03:31:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[27879]: audit 2024-02-13T03:31:07.945719+0000 mon.a (mon.0) 2028 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]': finished 2024-02-13T03:31:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[27879]: cluster 2024-02-13T03:31:07.945803+0000 mon.a (mon.0) 2029 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-02-13T03:31:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[27879]: audit 2024-02-13T03:31:07.970132+0000 mon.a (mon.0) 2030 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[27879]: audit 2024-02-13T03:31:07.971110+0000 mon.a (mon.0) 2031 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:31:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[27879]: audit 2024-02-13T03:31:07.972405+0000 mon.b (mon.1) 279 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[27879]: audit 2024-02-13T03:31:07.973538+0000 mon.b (mon.1) 280 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:31:09.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[19769]: audit 2024-02-13T03:31:07.945612+0000 mon.a (mon.0) 2027 : audit [INF] from='client.48338 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi161-61726-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi161-61726-60"}]': finished 2024-02-13T03:31:09.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[19769]: audit 2024-02-13T03:31:07.945719+0000 mon.a (mon.0) 2028 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]': finished 2024-02-13T03:31:09.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[19769]: cluster 2024-02-13T03:31:07.945803+0000 mon.a (mon.0) 2029 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-02-13T03:31:09.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[19769]: audit 2024-02-13T03:31:07.970132+0000 mon.a (mon.0) 2030 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:09.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[19769]: audit 2024-02-13T03:31:07.971110+0000 mon.a (mon.0) 2031 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:31:09.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[19769]: audit 2024-02-13T03:31:07.972405+0000 mon.b (mon.1) 279 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:09.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:09 smithi161 bash[19769]: audit 2024-02-13T03:31:07.973538+0000 mon.b (mon.1) 280 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:31:09.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:31:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:31:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:31:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:09 smithi183 bash[20824]: audit 2024-02-13T03:31:07.945612+0000 mon.a (mon.0) 2027 : audit [INF] from='client.48338 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi161-61726-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi161-61726-60"}]': finished 2024-02-13T03:31:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:09 smithi183 bash[20824]: audit 2024-02-13T03:31:07.945719+0000 mon.a (mon.0) 2028 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]': finished 2024-02-13T03:31:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:09 smithi183 bash[20824]: cluster 2024-02-13T03:31:07.945803+0000 mon.a (mon.0) 2029 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-02-13T03:31:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:09 smithi183 bash[20824]: audit 2024-02-13T03:31:07.970132+0000 mon.a (mon.0) 2030 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:09 smithi183 bash[20824]: audit 2024-02-13T03:31:07.971110+0000 mon.a (mon.0) 2031 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:31:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:09 smithi183 bash[20824]: audit 2024-02-13T03:31:07.972405+0000 mon.b (mon.1) 279 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:09 smithi183 bash[20824]: audit 2024-02-13T03:31:07.973538+0000 mon.b (mon.1) 280 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-45"}]: dispatch 2024-02-13T03:31:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[27879]: cluster 2024-02-13T03:31:08.954140+0000 mon.a (mon.0) 2032 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-02-13T03:31:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[27879]: cluster 2024-02-13T03:31:09.330079+0000 mgr.y (mgr.24473) 407 : cluster [DBG] pgmap v545: 204 pgs: 8 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[27879]: cluster 2024-02-13T03:31:09.952943+0000 mon.a (mon.0) 2033 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-02-13T03:31:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[27879]: audit 2024-02-13T03:31:09.955334+0000 mon.a (mon.0) 2034 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[27879]: audit 2024-02-13T03:31:09.955566+0000 mon.c (mon.2) 497 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[27879]: audit 2024-02-13T03:31:09.957107+0000 mon.a (mon.0) 2035 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[27879]: audit 2024-02-13T03:31:09.957134+0000 mon.b (mon.1) 281 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[19769]: cluster 2024-02-13T03:31:08.954140+0000 mon.a (mon.0) 2032 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-02-13T03:31:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[19769]: cluster 2024-02-13T03:31:09.330079+0000 mgr.y (mgr.24473) 407 : cluster [DBG] pgmap v545: 204 pgs: 8 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[19769]: cluster 2024-02-13T03:31:09.952943+0000 mon.a (mon.0) 2033 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-02-13T03:31:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[19769]: audit 2024-02-13T03:31:09.955334+0000 mon.a (mon.0) 2034 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[19769]: audit 2024-02-13T03:31:09.955566+0000 mon.c (mon.2) 497 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[19769]: audit 2024-02-13T03:31:09.957107+0000 mon.a (mon.0) 2035 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:10 smithi161 bash[19769]: audit 2024-02-13T03:31:09.957134+0000 mon.b (mon.1) 281 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:10 smithi183 bash[20824]: cluster 2024-02-13T03:31:08.954140+0000 mon.a (mon.0) 2032 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-02-13T03:31:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:10 smithi183 bash[20824]: cluster 2024-02-13T03:31:09.330079+0000 mgr.y (mgr.24473) 407 : cluster [DBG] pgmap v545: 204 pgs: 8 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:10 smithi183 bash[20824]: cluster 2024-02-13T03:31:09.952943+0000 mon.a (mon.0) 2033 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-02-13T03:31:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:10 smithi183 bash[20824]: audit 2024-02-13T03:31:09.955334+0000 mon.a (mon.0) 2034 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:10 smithi183 bash[20824]: audit 2024-02-13T03:31:09.955566+0000 mon.c (mon.2) 497 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:10 smithi183 bash[20824]: audit 2024-02-13T03:31:09.957107+0000 mon.a (mon.0) 2035 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:10 smithi183 bash[20824]: audit 2024-02-13T03:31:09.957134+0000 mon.b (mon.1) 281 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:11.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:31:11] "GET /metrics HTTP/1.1" 200 189152 "" "Prometheus/2.33.4" 2024-02-13T03:31:11.970 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:o.RoundTripAppendPP (3434 ms) 2024-02-13T03:31:11.970 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-02-13T03:31:11.970 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (2558 ms) 2024-02-13T03:31:11.970 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-02-13T03:31:11.970 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3011 ms) 2024-02-13T03:31:11.970 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-02-13T03:31:11.970 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3018 ms) 2024-02-13T03:31:11.970 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-02-13T03:31:11.970 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3006 ms) 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [----------] 31 tests from LibRadosAio (128333 ms total) 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3053 ms) 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9051 ms) 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14905 ms) 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3428 ms) 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (30438 ms total) 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: 2024-02-13T03:31:11.971 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-02-13T03:31:11.972 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-13T03:31:11.972 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (2608 ms) 2024-02-13T03:31:11.972 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (2608 ms total) 2024-02-13T03:31:11.972 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: 2024-02-13T03:31:11.972 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-02-13T03:31:11.972 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-02-13T03:31:11.972 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13995 ms) 2024-02-13T03:31:11.972 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-02-13T03:31:11.972 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (7445 ms) 2024-02-13T03:31:11.972 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-02-13T03:31:11.972 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (6600 ms) 2024-02-13T03:31:11.972 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-02-13T03:31:11.973 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7031 ms) 2024-02-13T03:31:11.973 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-02-13T03:31:11.973 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (2886 ms) 2024-02-13T03:31:11.973 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-02-13T03:31:11.973 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7029 ms) 2024-02-13T03:31:11.973 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-02-13T03:31:11.973 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (7036 ms) 2024-02-13T03:31:11.973 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-02-13T03:31:11.973 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7037 ms) 2024-02-13T03:31:11.973 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-02-13T03:31:11.975 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7033 ms) 2024-02-13T03:31:11.975 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-02-13T03:31:11.975 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7088 ms) 2024-02-13T03:31:11.975 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-02-13T03:31:11.975 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7020 ms) 2024-02-13T03:31:11.975 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-02-13T03:31:11.975 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7091 ms) 2024-02-13T03:31:11.975 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-02-13T03:31:11.976 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7023 ms) 2024-02-13T03:31:11.976 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-02-13T03:31:11.976 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3006 ms) 2024-02-13T03:31:11.976 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-02-13T03:31:11.976 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7062 ms) 2024-02-13T03:31:12.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:11 smithi183 bash[20824]: audit 2024-02-13T03:31:10.956183+0000 mon.a (mon.0) 2036 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:12.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:11 smithi183 bash[20824]: audit 2024-02-13T03:31:10.956702+0000 mon.a (mon.0) 2037 : audit [INF] from='client.48338 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]': finished 2024-02-13T03:31:12.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:11 smithi183 bash[20824]: cluster 2024-02-13T03:31:10.956895+0000 mon.a (mon.0) 2038 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-02-13T03:31:12.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:11 smithi183 bash[20824]: audit 2024-02-13T03:31:10.958640+0000 mon.c (mon.2) 498 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:12.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:11 smithi183 bash[20824]: audit 2024-02-13T03:31:10.960152+0000 mon.a (mon.0) 2039 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:12.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:11 smithi183 bash[20824]: audit 2024-02-13T03:31:10.979293+0000 mon.a (mon.0) 2040 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:31:12.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:11 smithi183 bash[20824]: audit 2024-02-13T03:31:10.981591+0000 mon.b (mon.1) 282 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:31:12.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:11 smithi183 bash[20824]: cluster 2024-02-13T03:31:11.330930+0000 mgr.y (mgr.24473) 408 : cluster [DBG] pgmap v548: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-13T03:31:12.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[27879]: audit 2024-02-13T03:31:10.956183+0000 mon.a (mon.0) 2036 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:12.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[27879]: audit 2024-02-13T03:31:10.956702+0000 mon.a (mon.0) 2037 : audit [INF] from='client.48338 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]': finished 2024-02-13T03:31:12.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[27879]: cluster 2024-02-13T03:31:10.956895+0000 mon.a (mon.0) 2038 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-02-13T03:31:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[27879]: audit 2024-02-13T03:31:10.958640+0000 mon.c (mon.2) 498 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[27879]: audit 2024-02-13T03:31:10.960152+0000 mon.a (mon.0) 2039 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[27879]: audit 2024-02-13T03:31:10.979293+0000 mon.a (mon.0) 2040 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:31:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[27879]: audit 2024-02-13T03:31:10.981591+0000 mon.b (mon.1) 282 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:31:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[27879]: cluster 2024-02-13T03:31:11.330930+0000 mgr.y (mgr.24473) 408 : cluster [DBG] pgmap v548: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-13T03:31:12.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[19769]: audit 2024-02-13T03:31:10.956183+0000 mon.a (mon.0) 2036 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:12.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[19769]: audit 2024-02-13T03:31:10.956702+0000 mon.a (mon.0) 2037 : audit [INF] from='client.48338 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi161-61726-60"}]': finished 2024-02-13T03:31:12.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[19769]: cluster 2024-02-13T03:31:10.956895+0000 mon.a (mon.0) 2038 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-02-13T03:31:12.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[19769]: audit 2024-02-13T03:31:10.958640+0000 mon.c (mon.2) 498 : audit [INF] from='client.? v1:172.21.15.161:0/4096340150' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:12.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[19769]: audit 2024-02-13T03:31:10.960152+0000 mon.a (mon.0) 2039 : audit [INF] from='client.48338 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]: dispatch 2024-02-13T03:31:12.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[19769]: audit 2024-02-13T03:31:10.979293+0000 mon.a (mon.0) 2040 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:31:12.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[19769]: audit 2024-02-13T03:31:10.981591+0000 mon.b (mon.1) 282 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:31:12.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:11 smithi161 bash[19769]: cluster 2024-02-13T03:31:11.330930+0000 mgr.y (mgr.24473) 408 : cluster [DBG] pgmap v548: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-13T03:31:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:12 smithi183 bash[20824]: audit 2024-02-13T03:31:11.959112+0000 mon.a (mon.0) 2041 : audit [INF] from='client.48338 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]': finished 2024-02-13T03:31:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:12 smithi183 bash[20824]: audit 2024-02-13T03:31:11.959390+0000 mon.a (mon.0) 2042 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:31:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:12 smithi183 bash[20824]: cluster 2024-02-13T03:31:11.959630+0000 mon.a (mon.0) 2043 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-02-13T03:31:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:12 smithi183 bash[20824]: audit 2024-02-13T03:31:11.961107+0000 mon.a (mon.0) 2044 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:12 smithi183 bash[20824]: audit 2024-02-13T03:31:11.963342+0000 mon.b (mon.1) 283 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:12 smithi183 bash[20824]: audit 2024-02-13T03:31:11.986407+0000 mon.a (mon.0) 2045 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:12 smithi183 bash[20824]: audit 2024-02-13T03:31:11.988422+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:12 smithi183 bash[20824]: audit 2024-02-13T03:31:11.988925+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:12 smithi183 bash[20824]: audit 2024-02-13T03:31:12.961424+0000 mon.a (mon.0) 2048 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-47"}]': finished 2024-02-13T03:31:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[27879]: audit 2024-02-13T03:31:11.959112+0000 mon.a (mon.0) 2041 : audit [INF] from='client.48338 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]': finished 2024-02-13T03:31:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[27879]: audit 2024-02-13T03:31:11.959390+0000 mon.a (mon.0) 2042 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:31:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[27879]: cluster 2024-02-13T03:31:11.959630+0000 mon.a (mon.0) 2043 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-02-13T03:31:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[27879]: audit 2024-02-13T03:31:11.961107+0000 mon.a (mon.0) 2044 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[27879]: audit 2024-02-13T03:31:11.963342+0000 mon.b (mon.1) 283 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[27879]: audit 2024-02-13T03:31:11.986407+0000 mon.a (mon.0) 2045 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[27879]: audit 2024-02-13T03:31:11.988422+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:13.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[27879]: audit 2024-02-13T03:31:11.988925+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:13.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[27879]: audit 2024-02-13T03:31:12.961424+0000 mon.a (mon.0) 2048 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-47"}]': finished 2024-02-13T03:31:13.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[19769]: audit 2024-02-13T03:31:11.959112+0000 mon.a (mon.0) 2041 : audit [INF] from='client.48338 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi161-61726-60"}]': finished 2024-02-13T03:31:13.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[19769]: audit 2024-02-13T03:31:11.959390+0000 mon.a (mon.0) 2042 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:31:13.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[19769]: cluster 2024-02-13T03:31:11.959630+0000 mon.a (mon.0) 2043 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-02-13T03:31:13.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[19769]: audit 2024-02-13T03:31:11.961107+0000 mon.a (mon.0) 2044 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:13.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[19769]: audit 2024-02-13T03:31:11.963342+0000 mon.b (mon.1) 283 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:13.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[19769]: audit 2024-02-13T03:31:11.986407+0000 mon.a (mon.0) 2045 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:13.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[19769]: audit 2024-02-13T03:31:11.988422+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:13.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[19769]: audit 2024-02-13T03:31:11.988925+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:13.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:12 smithi161 bash[19769]: audit 2024-02-13T03:31:12.961424+0000 mon.a (mon.0) 2048 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-47"}]': finished 2024-02-13T03:31:14.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[27879]: audit 2024-02-13T03:31:12.961738+0000 mon.a (mon.0) 2049 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[27879]: cluster 2024-02-13T03:31:12.961996+0000 mon.a (mon.0) 2050 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-02-13T03:31:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[27879]: audit 2024-02-13T03:31:12.964517+0000 mon.a (mon.0) 2051 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-47", "mode": "readproxy"}]: dispatch 2024-02-13T03:31:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[27879]: audit 2024-02-13T03:31:12.965500+0000 mon.b (mon.1) 284 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-47", "mode": "readproxy"}]: dispatch 2024-02-13T03:31:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[27879]: audit 2024-02-13T03:31:12.969242+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi161-61726-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[27879]: cluster 2024-02-13T03:31:13.331779+0000 mgr.y (mgr.24473) 409 : cluster [DBG] pgmap v551: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-13T03:31:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[27879]: cluster 2024-02-13T03:31:13.955552+0000 mon.a (mon.0) 2053 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:31:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[27879]: audit 2024-02-13T03:31:13.964608+0000 mon.a (mon.0) 2054 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-47", "mode": "readproxy"}]': finished 2024-02-13T03:31:14.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[19769]: audit 2024-02-13T03:31:12.961738+0000 mon.a (mon.0) 2049 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:14.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[19769]: cluster 2024-02-13T03:31:12.961996+0000 mon.a (mon.0) 2050 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-02-13T03:31:14.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[19769]: audit 2024-02-13T03:31:12.964517+0000 mon.a (mon.0) 2051 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-47", "mode": "readproxy"}]: dispatch 2024-02-13T03:31:14.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[19769]: audit 2024-02-13T03:31:12.965500+0000 mon.b (mon.1) 284 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-47", "mode": "readproxy"}]: dispatch 2024-02-13T03:31:14.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[19769]: audit 2024-02-13T03:31:12.969242+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi161-61726-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:14.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[19769]: cluster 2024-02-13T03:31:13.331779+0000 mgr.y (mgr.24473) 409 : cluster [DBG] pgmap v551: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-13T03:31:14.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[19769]: cluster 2024-02-13T03:31:13.955552+0000 mon.a (mon.0) 2053 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:31:14.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:14 smithi161 bash[19769]: audit 2024-02-13T03:31:13.964608+0000 mon.a (mon.0) 2054 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-47", "mode": "readproxy"}]': finished 2024-02-13T03:31:14.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:14 smithi183 bash[20824]: audit 2024-02-13T03:31:12.961738+0000 mon.a (mon.0) 2049 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:14 smithi183 bash[20824]: cluster 2024-02-13T03:31:12.961996+0000 mon.a (mon.0) 2050 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-02-13T03:31:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:14 smithi183 bash[20824]: audit 2024-02-13T03:31:12.964517+0000 mon.a (mon.0) 2051 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-47", "mode": "readproxy"}]: dispatch 2024-02-13T03:31:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:14 smithi183 bash[20824]: audit 2024-02-13T03:31:12.965500+0000 mon.b (mon.1) 284 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-47", "mode": "readproxy"}]: dispatch 2024-02-13T03:31:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:14 smithi183 bash[20824]: audit 2024-02-13T03:31:12.969242+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi161-61726-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:14 smithi183 bash[20824]: cluster 2024-02-13T03:31:13.331779+0000 mgr.y (mgr.24473) 409 : cluster [DBG] pgmap v551: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-13T03:31:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:14 smithi183 bash[20824]: cluster 2024-02-13T03:31:13.955552+0000 mon.a (mon.0) 2053 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:31:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:14 smithi183 bash[20824]: audit 2024-02-13T03:31:13.964608+0000 mon.a (mon.0) 2054 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-47", "mode": "readproxy"}]': finished 2024-02-13T03:31:15.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:15 smithi161 bash[27879]: cluster 2024-02-13T03:31:14.391259+0000 mon.a (mon.0) 2055 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-02-13T03:31:15.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:15 smithi161 bash[27879]: audit 2024-02-13T03:31:14.967586+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi161-61726-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi161-61726-61"}]': finished 2024-02-13T03:31:15.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:15 smithi161 bash[27879]: cluster 2024-02-13T03:31:14.967740+0000 mon.a (mon.0) 2057 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-02-13T03:31:15.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:15 smithi161 bash[27879]: cluster 2024-02-13T03:31:15.332623+0000 mgr.y (mgr.24473) 410 : cluster [DBG] pgmap v554: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:15.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:15 smithi161 bash[19769]: cluster 2024-02-13T03:31:14.391259+0000 mon.a (mon.0) 2055 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-02-13T03:31:15.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:15 smithi161 bash[19769]: audit 2024-02-13T03:31:14.967586+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi161-61726-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi161-61726-61"}]': finished 2024-02-13T03:31:15.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:15 smithi161 bash[19769]: cluster 2024-02-13T03:31:14.967740+0000 mon.a (mon.0) 2057 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-02-13T03:31:15.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:15 smithi161 bash[19769]: cluster 2024-02-13T03:31:15.332623+0000 mgr.y (mgr.24473) 410 : cluster [DBG] pgmap v554: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:15 smithi183 bash[20824]: cluster 2024-02-13T03:31:14.391259+0000 mon.a (mon.0) 2055 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-02-13T03:31:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:15 smithi183 bash[20824]: audit 2024-02-13T03:31:14.967586+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi161-61726-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi161-61726-61"}]': finished 2024-02-13T03:31:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:15 smithi183 bash[20824]: cluster 2024-02-13T03:31:14.967740+0000 mon.a (mon.0) 2057 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-02-13T03:31:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:15 smithi183 bash[20824]: cluster 2024-02-13T03:31:15.332623+0000 mgr.y (mgr.24473) 410 : cluster [DBG] pgmap v554: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:16.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:31:16 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:31:17.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:16 smithi183 bash[20824]: cluster 2024-02-13T03:31:15.970891+0000 mon.a (mon.0) 2058 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-02-13T03:31:17.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:16 smithi183 bash[20824]: audit 2024-02-13T03:31:16.029466+0000 mgr.y (mgr.24473) 411 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:17.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:16 smithi161 bash[19769]: cluster 2024-02-13T03:31:15.970891+0000 mon.a (mon.0) 2058 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-02-13T03:31:17.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:16 smithi161 bash[19769]: audit 2024-02-13T03:31:16.029466+0000 mgr.y (mgr.24473) 411 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:16 smithi161 bash[27879]: cluster 2024-02-13T03:31:15.970891+0000 mon.a (mon.0) 2058 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-02-13T03:31:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:16 smithi161 bash[27879]: audit 2024-02-13T03:31:16.029466+0000 mgr.y (mgr.24473) 411 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:17 smithi161 bash[27879]: cluster 2024-02-13T03:31:16.983719+0000 mon.a (mon.0) 2059 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-02-13T03:31:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:17 smithi161 bash[27879]: audit 2024-02-13T03:31:16.984797+0000 mon.a (mon.0) 2060 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:18.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:17 smithi161 bash[19769]: cluster 2024-02-13T03:31:16.983719+0000 mon.a (mon.0) 2059 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-02-13T03:31:18.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:17 smithi161 bash[19769]: audit 2024-02-13T03:31:16.984797+0000 mon.a (mon.0) 2060 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:18.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:17 smithi183 bash[20824]: cluster 2024-02-13T03:31:16.983719+0000 mon.a (mon.0) 2059 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-02-13T03:31:18.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:17 smithi183 bash[20824]: audit 2024-02-13T03:31:16.984797+0000 mon.a (mon.0) 2060 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:19.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:19 smithi183 bash[20824]: cluster 2024-02-13T03:31:17.333182+0000 mgr.y (mgr.24473) 412 : cluster [DBG] pgmap v557: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:31:19.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:19 smithi183 bash[20824]: audit 2024-02-13T03:31:17.992238+0000 mon.a (mon.0) 2061 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61"}]': finished 2024-02-13T03:31:19.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:19 smithi183 bash[20824]: cluster 2024-02-13T03:31:17.992390+0000 mon.a (mon.0) 2062 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-02-13T03:31:19.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:19 smithi183 bash[20824]: audit 2024-02-13T03:31:17.993118+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:18 smithi161 bash[27879]: cluster 2024-02-13T03:31:17.333182+0000 mgr.y (mgr.24473) 412 : cluster [DBG] pgmap v557: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:31:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:18 smithi161 bash[27879]: audit 2024-02-13T03:31:17.992238+0000 mon.a (mon.0) 2061 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61"}]': finished 2024-02-13T03:31:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:18 smithi161 bash[27879]: cluster 2024-02-13T03:31:17.992390+0000 mon.a (mon.0) 2062 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-02-13T03:31:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:18 smithi161 bash[27879]: audit 2024-02-13T03:31:17.993118+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:19.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:18 smithi161 bash[19769]: cluster 2024-02-13T03:31:17.333182+0000 mgr.y (mgr.24473) 412 : cluster [DBG] pgmap v557: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:31:19.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:18 smithi161 bash[19769]: audit 2024-02-13T03:31:17.992238+0000 mon.a (mon.0) 2061 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi161-61726-61"}]': finished 2024-02-13T03:31:19.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:18 smithi161 bash[19769]: cluster 2024-02-13T03:31:17.992390+0000 mon.a (mon.0) 2062 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-02-13T03:31:19.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:18 smithi161 bash[19769]: audit 2024-02-13T03:31:17.993118+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi161-61726-61"}]: dispatch 2024-02-13T03:31:19.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:31:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:31:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:31:20.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[27879]: audit 2024-02-13T03:31:18.995775+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi161-61726-61"}]': finished 2024-02-13T03:31:20.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[27879]: cluster 2024-02-13T03:31:18.995882+0000 mon.a (mon.0) 2065 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-02-13T03:31:20.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[27879]: audit 2024-02-13T03:31:19.020166+0000 mon.a (mon.0) 2066 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:20.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[27879]: audit 2024-02-13T03:31:19.022138+0000 mon.a (mon.0) 2067 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:20.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[27879]: audit 2024-02-13T03:31:19.022430+0000 mon.b (mon.1) 285 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:20.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[27879]: audit 2024-02-13T03:31:19.023487+0000 mon.a (mon.0) 2068 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi161-61726-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:20.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[27879]: audit 2024-02-13T03:31:19.024335+0000 mon.b (mon.1) 286 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:20.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[27879]: audit 2024-02-13T03:31:19.025676+0000 mon.b (mon.1) 287 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi161-61726-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:20.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[27879]: audit 2024-02-13T03:31:19.784713+0000 mon.a (mon.0) 2069 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:31:20.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[19769]: audit 2024-02-13T03:31:18.995775+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi161-61726-61"}]': finished 2024-02-13T03:31:20.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[19769]: cluster 2024-02-13T03:31:18.995882+0000 mon.a (mon.0) 2065 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-02-13T03:31:20.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[19769]: audit 2024-02-13T03:31:19.020166+0000 mon.a (mon.0) 2066 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:20.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[19769]: audit 2024-02-13T03:31:19.022138+0000 mon.a (mon.0) 2067 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:20.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[19769]: audit 2024-02-13T03:31:19.022430+0000 mon.b (mon.1) 285 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:20.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[19769]: audit 2024-02-13T03:31:19.023487+0000 mon.a (mon.0) 2068 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi161-61726-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:20.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[19769]: audit 2024-02-13T03:31:19.024335+0000 mon.b (mon.1) 286 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:20.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[19769]: audit 2024-02-13T03:31:19.025676+0000 mon.b (mon.1) 287 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi161-61726-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:20.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:20 smithi161 bash[19769]: audit 2024-02-13T03:31:19.784713+0000 mon.a (mon.0) 2069 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:31:20.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:20 smithi183 bash[20824]: audit 2024-02-13T03:31:18.995775+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? v1:172.21.15.161:0/2772940038' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi161-61726-61"}]': finished 2024-02-13T03:31:20.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:20 smithi183 bash[20824]: cluster 2024-02-13T03:31:18.995882+0000 mon.a (mon.0) 2065 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-02-13T03:31:20.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:20 smithi183 bash[20824]: audit 2024-02-13T03:31:19.020166+0000 mon.a (mon.0) 2066 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:20.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:20 smithi183 bash[20824]: audit 2024-02-13T03:31:19.022138+0000 mon.a (mon.0) 2067 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:20.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:20 smithi183 bash[20824]: audit 2024-02-13T03:31:19.022430+0000 mon.b (mon.1) 285 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:20.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:20 smithi183 bash[20824]: audit 2024-02-13T03:31:19.023487+0000 mon.a (mon.0) 2068 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi161-61726-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:20.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:20 smithi183 bash[20824]: audit 2024-02-13T03:31:19.024335+0000 mon.b (mon.1) 286 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:20.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:20 smithi183 bash[20824]: audit 2024-02-13T03:31:19.025676+0000 mon.b (mon.1) 287 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi161-61726-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:20.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:20 smithi183 bash[20824]: audit 2024-02-13T03:31:19.784713+0000 mon.a (mon.0) 2069 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:31:21.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:31:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:31:21] "GET /metrics HTTP/1.1" 200 186843 "" "Prometheus/2.33.4" 2024-02-13T03:31:21.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:21 smithi183 bash[20824]: cluster 2024-02-13T03:31:19.334001+0000 mgr.y (mgr.24473) 413 : cluster [DBG] pgmap v560: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:31:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:21 smithi183 bash[20824]: audit 2024-02-13T03:31:20.006395+0000 mon.a (mon.0) 2070 : audit [INF] from='client.49015 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi161-61726-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:21 smithi183 bash[20824]: cluster 2024-02-13T03:31:20.006488+0000 mon.a (mon.0) 2071 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-02-13T03:31:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:21 smithi183 bash[20824]: audit 2024-02-13T03:31:20.008813+0000 mon.a (mon.0) 2072 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi161-61726-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:21 smithi183 bash[20824]: audit 2024-02-13T03:31:20.010147+0000 mon.b (mon.1) 288 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi161-61726-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:21.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:21 smithi161 bash[27879]: cluster 2024-02-13T03:31:19.334001+0000 mgr.y (mgr.24473) 413 : cluster [DBG] pgmap v560: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:31:21.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:21 smithi161 bash[27879]: audit 2024-02-13T03:31:20.006395+0000 mon.a (mon.0) 2070 : audit [INF] from='client.49015 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi161-61726-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:21.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:21 smithi161 bash[27879]: cluster 2024-02-13T03:31:20.006488+0000 mon.a (mon.0) 2071 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-02-13T03:31:21.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:21 smithi161 bash[27879]: audit 2024-02-13T03:31:20.008813+0000 mon.a (mon.0) 2072 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi161-61726-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:21 smithi161 bash[27879]: audit 2024-02-13T03:31:20.010147+0000 mon.b (mon.1) 288 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi161-61726-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:21 smithi161 bash[19769]: cluster 2024-02-13T03:31:19.334001+0000 mgr.y (mgr.24473) 413 : cluster [DBG] pgmap v560: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:31:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:21 smithi161 bash[19769]: audit 2024-02-13T03:31:20.006395+0000 mon.a (mon.0) 2070 : audit [INF] from='client.49015 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi161-61726-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:21 smithi161 bash[19769]: cluster 2024-02-13T03:31:20.006488+0000 mon.a (mon.0) 2071 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-02-13T03:31:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:21 smithi161 bash[19769]: audit 2024-02-13T03:31:20.008813+0000 mon.a (mon.0) 2072 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi161-61726-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:21 smithi161 bash[19769]: audit 2024-02-13T03:31:20.010147+0000 mon.b (mon.1) 288 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi161-61726-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:22.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:22 smithi183 bash[20824]: cluster 2024-02-13T03:31:21.334727+0000 mgr.y (mgr.24473) 414 : cluster [DBG] pgmap v563: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 108 KiB/s rd, 0 B/s wr, 179 op/s 2024-02-13T03:31:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:22 smithi183 bash[20824]: cluster 2024-02-13T03:31:21.444976+0000 mon.a (mon.0) 2073 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-02-13T03:31:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:22 smithi183 bash[20824]: audit 2024-02-13T03:31:22.017962+0000 mon.a (mon.0) 2074 : audit [INF] from='client.49015 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi161-61726-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi161-61726-62"}]': finished 2024-02-13T03:31:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:22 smithi183 bash[20824]: cluster 2024-02-13T03:31:22.018778+0000 mon.a (mon.0) 2075 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-02-13T03:31:22.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:22 smithi161 bash[27879]: cluster 2024-02-13T03:31:21.334727+0000 mgr.y (mgr.24473) 414 : cluster [DBG] pgmap v563: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 108 KiB/s rd, 0 B/s wr, 179 op/s 2024-02-13T03:31:22.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:22 smithi161 bash[27879]: cluster 2024-02-13T03:31:21.444976+0000 mon.a (mon.0) 2073 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-02-13T03:31:22.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:22 smithi161 bash[27879]: audit 2024-02-13T03:31:22.017962+0000 mon.a (mon.0) 2074 : audit [INF] from='client.49015 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi161-61726-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi161-61726-62"}]': finished 2024-02-13T03:31:22.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:22 smithi161 bash[27879]: cluster 2024-02-13T03:31:22.018778+0000 mon.a (mon.0) 2075 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-02-13T03:31:22.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:22 smithi161 bash[19769]: cluster 2024-02-13T03:31:21.334727+0000 mgr.y (mgr.24473) 414 : cluster [DBG] pgmap v563: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 108 KiB/s rd, 0 B/s wr, 179 op/s 2024-02-13T03:31:22.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:22 smithi161 bash[19769]: cluster 2024-02-13T03:31:21.444976+0000 mon.a (mon.0) 2073 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-02-13T03:31:22.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:22 smithi161 bash[19769]: audit 2024-02-13T03:31:22.017962+0000 mon.a (mon.0) 2074 : audit [INF] from='client.49015 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi161-61726-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi161-61726-62"}]': finished 2024-02-13T03:31:22.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:22 smithi161 bash[19769]: cluster 2024-02-13T03:31:22.018778+0000 mon.a (mon.0) 2075 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-02-13T03:31:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:24 smithi161 bash[27879]: cluster 2024-02-13T03:31:23.023004+0000 mon.a (mon.0) 2076 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-02-13T03:31:24.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:24 smithi161 bash[19769]: cluster 2024-02-13T03:31:23.023004+0000 mon.a (mon.0) 2076 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-02-13T03:31:24.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:24 smithi183 bash[20824]: cluster 2024-02-13T03:31:23.023004+0000 mon.a (mon.0) 2076 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-02-13T03:31:25.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:25 smithi161 bash[27879]: cluster 2024-02-13T03:31:23.335529+0000 mgr.y (mgr.24473) 415 : cluster [DBG] pgmap v566: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 108 KiB/s rd, 0 B/s wr, 179 op/s 2024-02-13T03:31:25.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:25 smithi161 bash[27879]: cluster 2024-02-13T03:31:24.025074+0000 mon.a (mon.0) 2077 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-02-13T03:31:25.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:25 smithi161 bash[27879]: audit 2024-02-13T03:31:24.030788+0000 mon.b (mon.1) 289 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:25.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:25 smithi161 bash[27879]: audit 2024-02-13T03:31:24.033846+0000 mon.a (mon.0) 2078 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:25.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:25 smithi161 bash[27879]: audit 2024-02-13T03:31:24.248377+0000 mon.a (mon.0) 2079 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:25.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:25 smithi161 bash[27879]: audit 2024-02-13T03:31:24.250269+0000 mon.b (mon.1) 290 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:25.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:25 smithi161 bash[19769]: cluster 2024-02-13T03:31:23.335529+0000 mgr.y (mgr.24473) 415 : cluster [DBG] pgmap v566: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 108 KiB/s rd, 0 B/s wr, 179 op/s 2024-02-13T03:31:25.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:25 smithi161 bash[19769]: cluster 2024-02-13T03:31:24.025074+0000 mon.a (mon.0) 2077 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-02-13T03:31:25.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:25 smithi161 bash[19769]: audit 2024-02-13T03:31:24.030788+0000 mon.b (mon.1) 289 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:25.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:25 smithi161 bash[19769]: audit 2024-02-13T03:31:24.033846+0000 mon.a (mon.0) 2078 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:25.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:25 smithi161 bash[19769]: audit 2024-02-13T03:31:24.248377+0000 mon.a (mon.0) 2079 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:25.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:25 smithi161 bash[19769]: audit 2024-02-13T03:31:24.250269+0000 mon.b (mon.1) 290 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:25 smithi183 bash[20824]: cluster 2024-02-13T03:31:23.335529+0000 mgr.y (mgr.24473) 415 : cluster [DBG] pgmap v566: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 108 KiB/s rd, 0 B/s wr, 179 op/s 2024-02-13T03:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:25 smithi183 bash[20824]: cluster 2024-02-13T03:31:24.025074+0000 mon.a (mon.0) 2077 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-02-13T03:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:25 smithi183 bash[20824]: audit 2024-02-13T03:31:24.030788+0000 mon.b (mon.1) 289 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:25.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:25 smithi183 bash[20824]: audit 2024-02-13T03:31:24.033846+0000 mon.a (mon.0) 2078 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:25.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:25 smithi183 bash[20824]: audit 2024-02-13T03:31:24.248377+0000 mon.a (mon.0) 2079 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:25.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:25 smithi183 bash[20824]: audit 2024-02-13T03:31:24.250269+0000 mon.b (mon.1) 290 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[19769]: audit 2024-02-13T03:31:25.043588+0000 mon.a (mon.0) 2080 : audit [INF] from='client.49015 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]': finished 2024-02-13T03:31:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[19769]: audit 2024-02-13T03:31:25.043685+0000 mon.a (mon.0) 2081 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:31:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[19769]: cluster 2024-02-13T03:31:25.043753+0000 mon.a (mon.0) 2082 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-02-13T03:31:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[19769]: audit 2024-02-13T03:31:25.046141+0000 mon.a (mon.0) 2083 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:26.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[19769]: audit 2024-02-13T03:31:25.046981+0000 mon.a (mon.0) 2084 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:26.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[19769]: audit 2024-02-13T03:31:25.048028+0000 mon.b (mon.1) 291 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:26.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[19769]: audit 2024-02-13T03:31:25.048984+0000 mon.b (mon.1) 292 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:26.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[27879]: audit 2024-02-13T03:31:25.043588+0000 mon.a (mon.0) 2080 : audit [INF] from='client.49015 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]': finished 2024-02-13T03:31:26.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[27879]: audit 2024-02-13T03:31:25.043685+0000 mon.a (mon.0) 2081 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:31:26.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[27879]: cluster 2024-02-13T03:31:25.043753+0000 mon.a (mon.0) 2082 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-02-13T03:31:26.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[27879]: audit 2024-02-13T03:31:25.046141+0000 mon.a (mon.0) 2083 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:26.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[27879]: audit 2024-02-13T03:31:25.046981+0000 mon.a (mon.0) 2084 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:26.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[27879]: audit 2024-02-13T03:31:25.048028+0000 mon.b (mon.1) 291 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:26.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:26 smithi161 bash[27879]: audit 2024-02-13T03:31:25.048984+0000 mon.b (mon.1) 292 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:26.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:31:26 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:31:26.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:26 smithi183 bash[20824]: audit 2024-02-13T03:31:25.043588+0000 mon.a (mon.0) 2080 : audit [INF] from='client.49015 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi161-61726-62"}]': finished 2024-02-13T03:31:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:26 smithi183 bash[20824]: audit 2024-02-13T03:31:25.043685+0000 mon.a (mon.0) 2081 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:31:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:26 smithi183 bash[20824]: cluster 2024-02-13T03:31:25.043753+0000 mon.a (mon.0) 2082 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-02-13T03:31:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:26 smithi183 bash[20824]: audit 2024-02-13T03:31:25.046141+0000 mon.a (mon.0) 2083 : audit [INF] from='client.49015 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:26 smithi183 bash[20824]: audit 2024-02-13T03:31:25.046981+0000 mon.a (mon.0) 2084 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:26 smithi183 bash[20824]: audit 2024-02-13T03:31:25.048028+0000 mon.b (mon.1) 291 : audit [INF] from='client.? v1:172.21.15.161:0/2760782558' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]: dispatch 2024-02-13T03:31:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:26 smithi183 bash[20824]: audit 2024-02-13T03:31:25.048984+0000 mon.b (mon.1) 292 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:27.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: cluster 2024-02-13T03:31:25.336332+0000 mgr.y (mgr.24473) 416 : cluster [DBG] pgmap v569: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:27.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: audit 2024-02-13T03:31:26.040689+0000 mgr.y (mgr.24473) 417 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: cluster 2024-02-13T03:31:26.041701+0000 mon.a (mon.0) 2085 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:31:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: audit 2024-02-13T03:31:26.051674+0000 mon.a (mon.0) 2086 : audit [INF] from='client.49015 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]': finished 2024-02-13T03:31:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: audit 2024-02-13T03:31:26.051924+0000 mon.a (mon.0) 2087 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]': finished 2024-02-13T03:31:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: cluster 2024-02-13T03:31:26.052013+0000 mon.a (mon.0) 2088 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-02-13T03:31:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: audit 2024-02-13T03:31:26.074093+0000 mon.c (mon.2) 499 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: audit 2024-02-13T03:31:26.074577+0000 mon.a (mon.0) 2089 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: audit 2024-02-13T03:31:26.076072+0000 mon.c (mon.2) 500 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: audit 2024-02-13T03:31:26.076466+0000 mon.a (mon.0) 2090 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: audit 2024-02-13T03:31:26.077051+0000 mon.c (mon.2) 501 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi161-61726-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: audit 2024-02-13T03:31:26.077376+0000 mon.a (mon.0) 2091 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi161-61726-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: audit 2024-02-13T03:31:26.084181+0000 mon.a (mon.0) 2092 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: audit 2024-02-13T03:31:26.085118+0000 mon.a (mon.0) 2093 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: audit 2024-02-13T03:31:26.086531+0000 mon.b (mon.1) 293 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:27.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[27879]: audit 2024-02-13T03:31:26.087500+0000 mon.b (mon.1) 294 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:27.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: cluster 2024-02-13T03:31:25.336332+0000 mgr.y (mgr.24473) 416 : cluster [DBG] pgmap v569: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:27.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: audit 2024-02-13T03:31:26.040689+0000 mgr.y (mgr.24473) 417 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:27.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: cluster 2024-02-13T03:31:26.041701+0000 mon.a (mon.0) 2085 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:31:27.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: audit 2024-02-13T03:31:26.051674+0000 mon.a (mon.0) 2086 : audit [INF] from='client.49015 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]': finished 2024-02-13T03:31:27.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: audit 2024-02-13T03:31:26.051924+0000 mon.a (mon.0) 2087 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]': finished 2024-02-13T03:31:27.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: cluster 2024-02-13T03:31:26.052013+0000 mon.a (mon.0) 2088 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-02-13T03:31:27.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: audit 2024-02-13T03:31:26.074093+0000 mon.c (mon.2) 499 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:27.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: audit 2024-02-13T03:31:26.074577+0000 mon.a (mon.0) 2089 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:27.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: audit 2024-02-13T03:31:26.076072+0000 mon.c (mon.2) 500 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:27.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: audit 2024-02-13T03:31:26.076466+0000 mon.a (mon.0) 2090 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:27.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: audit 2024-02-13T03:31:26.077051+0000 mon.c (mon.2) 501 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi161-61726-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:27.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: audit 2024-02-13T03:31:26.077376+0000 mon.a (mon.0) 2091 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi161-61726-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:27.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: audit 2024-02-13T03:31:26.084181+0000 mon.a (mon.0) 2092 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:27.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: audit 2024-02-13T03:31:26.085118+0000 mon.a (mon.0) 2093 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:27.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: audit 2024-02-13T03:31:26.086531+0000 mon.b (mon.1) 293 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:27.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:27 smithi161 bash[19769]: audit 2024-02-13T03:31:26.087500+0000 mon.b (mon.1) 294 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: cluster 2024-02-13T03:31:25.336332+0000 mgr.y (mgr.24473) 416 : cluster [DBG] pgmap v569: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: audit 2024-02-13T03:31:26.040689+0000 mgr.y (mgr.24473) 417 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: cluster 2024-02-13T03:31:26.041701+0000 mon.a (mon.0) 2085 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:31:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: audit 2024-02-13T03:31:26.051674+0000 mon.a (mon.0) 2086 : audit [INF] from='client.49015 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi161-61726-62"}]': finished 2024-02-13T03:31:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: audit 2024-02-13T03:31:26.051924+0000 mon.a (mon.0) 2087 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]': finished 2024-02-13T03:31:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: cluster 2024-02-13T03:31:26.052013+0000 mon.a (mon.0) 2088 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-02-13T03:31:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: audit 2024-02-13T03:31:26.074093+0000 mon.c (mon.2) 499 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: audit 2024-02-13T03:31:26.074577+0000 mon.a (mon.0) 2089 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: audit 2024-02-13T03:31:26.076072+0000 mon.c (mon.2) 500 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:27.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: audit 2024-02-13T03:31:26.076466+0000 mon.a (mon.0) 2090 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:27.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: audit 2024-02-13T03:31:26.077051+0000 mon.c (mon.2) 501 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi161-61726-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:27.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: audit 2024-02-13T03:31:26.077376+0000 mon.a (mon.0) 2091 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi161-61726-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:27.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: audit 2024-02-13T03:31:26.084181+0000 mon.a (mon.0) 2092 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:27.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: audit 2024-02-13T03:31:26.085118+0000 mon.a (mon.0) 2093 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:27.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: audit 2024-02-13T03:31:26.086531+0000 mon.b (mon.1) 293 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:27.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:27 smithi183 bash[20824]: audit 2024-02-13T03:31:26.087500+0000 mon.b (mon.1) 294 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-47"}]: dispatch 2024-02-13T03:31:28.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:28 smithi161 bash[27879]: audit 2024-02-13T03:31:27.064146+0000 mon.a (mon.0) 2094 : audit [INF] from='client.48356 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi161-61726-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:28.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:28 smithi161 bash[27879]: cluster 2024-02-13T03:31:27.064327+0000 mon.a (mon.0) 2095 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-02-13T03:31:28.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:28 smithi161 bash[27879]: audit 2024-02-13T03:31:27.070577+0000 mon.c (mon.2) 502 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi161-61726-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:28.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:28 smithi161 bash[27879]: audit 2024-02-13T03:31:27.509697+0000 mon.a (mon.0) 2096 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi161-61726-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:28.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:28 smithi161 bash[19769]: audit 2024-02-13T03:31:27.064146+0000 mon.a (mon.0) 2094 : audit [INF] from='client.48356 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi161-61726-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:28.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:28 smithi161 bash[19769]: cluster 2024-02-13T03:31:27.064327+0000 mon.a (mon.0) 2095 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-02-13T03:31:28.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:28 smithi161 bash[19769]: audit 2024-02-13T03:31:27.070577+0000 mon.c (mon.2) 502 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi161-61726-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:28.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:28 smithi161 bash[19769]: audit 2024-02-13T03:31:27.509697+0000 mon.a (mon.0) 2096 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi161-61726-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:28 smithi183 bash[20824]: audit 2024-02-13T03:31:27.064146+0000 mon.a (mon.0) 2094 : audit [INF] from='client.48356 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi161-61726-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:28 smithi183 bash[20824]: cluster 2024-02-13T03:31:27.064327+0000 mon.a (mon.0) 2095 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-02-13T03:31:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:28 smithi183 bash[20824]: audit 2024-02-13T03:31:27.070577+0000 mon.c (mon.2) 502 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi161-61726-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:28 smithi183 bash[20824]: audit 2024-02-13T03:31:27.509697+0000 mon.a (mon.0) 2096 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi161-61726-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:29.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:29 smithi161 bash[27879]: cluster 2024-02-13T03:31:27.337036+0000 mgr.y (mgr.24473) 418 : cluster [DBG] pgmap v572: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:29.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:29 smithi161 bash[27879]: audit 2024-02-13T03:31:28.101486+0000 mon.b (mon.1) 295 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:29.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:29 smithi161 bash[27879]: cluster 2024-02-13T03:31:28.512970+0000 mon.a (mon.0) 2097 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-02-13T03:31:29.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:29 smithi161 bash[27879]: audit 2024-02-13T03:31:28.516193+0000 mon.a (mon.0) 2098 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:29.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:29 smithi161 bash[19769]: cluster 2024-02-13T03:31:27.337036+0000 mgr.y (mgr.24473) 418 : cluster [DBG] pgmap v572: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:29.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:29 smithi161 bash[19769]: audit 2024-02-13T03:31:28.101486+0000 mon.b (mon.1) 295 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:29.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:29 smithi161 bash[19769]: cluster 2024-02-13T03:31:28.512970+0000 mon.a (mon.0) 2097 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-02-13T03:31:29.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:29 smithi161 bash[19769]: audit 2024-02-13T03:31:28.516193+0000 mon.a (mon.0) 2098 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:29.486 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:31:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:31:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:31:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:29 smithi183 bash[20824]: cluster 2024-02-13T03:31:27.337036+0000 mgr.y (mgr.24473) 418 : cluster [DBG] pgmap v572: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:29 smithi183 bash[20824]: audit 2024-02-13T03:31:28.101486+0000 mon.b (mon.1) 295 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:29 smithi183 bash[20824]: cluster 2024-02-13T03:31:28.512970+0000 mon.a (mon.0) 2097 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-02-13T03:31:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:29 smithi183 bash[20824]: audit 2024-02-13T03:31:28.516193+0000 mon.a (mon.0) 2098 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:30.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:30 smithi161 bash[27879]: audit 2024-02-13T03:31:29.100314+0000 mon.a (mon.0) 2099 : audit [INF] from='client.48356 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi161-61726-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi161-61726-63"}]': finished 2024-02-13T03:31:30.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:30 smithi161 bash[27879]: audit 2024-02-13T03:31:29.100526+0000 mon.a (mon.0) 2100 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:30.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:30 smithi161 bash[27879]: cluster 2024-02-13T03:31:29.100947+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-02-13T03:31:30.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:30 smithi161 bash[27879]: audit 2024-02-13T03:31:29.129354+0000 mon.a (mon.0) 2102 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:31:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:30 smithi161 bash[27879]: audit 2024-02-13T03:31:29.131592+0000 mon.b (mon.1) 296 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:31:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:30 smithi161 bash[19769]: audit 2024-02-13T03:31:29.100314+0000 mon.a (mon.0) 2099 : audit [INF] from='client.48356 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi161-61726-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi161-61726-63"}]': finished 2024-02-13T03:31:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:30 smithi161 bash[19769]: audit 2024-02-13T03:31:29.100526+0000 mon.a (mon.0) 2100 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:30 smithi161 bash[19769]: cluster 2024-02-13T03:31:29.100947+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-02-13T03:31:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:30 smithi161 bash[19769]: audit 2024-02-13T03:31:29.129354+0000 mon.a (mon.0) 2102 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:31:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:30 smithi161 bash[19769]: audit 2024-02-13T03:31:29.131592+0000 mon.b (mon.1) 296 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:31:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:30 smithi183 bash[20824]: audit 2024-02-13T03:31:29.100314+0000 mon.a (mon.0) 2099 : audit [INF] from='client.48356 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi161-61726-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi161-61726-63"}]': finished 2024-02-13T03:31:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:30 smithi183 bash[20824]: audit 2024-02-13T03:31:29.100526+0000 mon.a (mon.0) 2100 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:30 smithi183 bash[20824]: cluster 2024-02-13T03:31:29.100947+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-02-13T03:31:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:30 smithi183 bash[20824]: audit 2024-02-13T03:31:29.129354+0000 mon.a (mon.0) 2102 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:31:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:30 smithi183 bash[20824]: audit 2024-02-13T03:31:29.131592+0000 mon.b (mon.1) 296 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:31:31.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:31:31] "GET /metrics HTTP/1.1" 200 186843 "" "Prometheus/2.33.4" 2024-02-13T03:31:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[27879]: cluster 2024-02-13T03:31:29.337990+0000 mgr.y (mgr.24473) 419 : cluster [DBG] pgmap v575: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[27879]: audit 2024-02-13T03:31:30.114194+0000 mon.a (mon.0) 2103 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:31:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[27879]: cluster 2024-02-13T03:31:30.114637+0000 mon.a (mon.0) 2104 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-02-13T03:31:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[27879]: audit 2024-02-13T03:31:30.117069+0000 mon.a (mon.0) 2105 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[27879]: audit 2024-02-13T03:31:30.118283+0000 mon.b (mon.1) 297 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[27879]: audit 2024-02-13T03:31:31.116502+0000 mon.a (mon.0) 2106 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-49"}]': finished 2024-02-13T03:31:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[27879]: cluster 2024-02-13T03:31:31.117131+0000 mon.a (mon.0) 2107 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-02-13T03:31:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[27879]: audit 2024-02-13T03:31:31.119696+0000 mon.c (mon.2) 503 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[27879]: audit 2024-02-13T03:31:31.119781+0000 mon.a (mon.0) 2108 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-49", "mode": "writeback"}]: dispatch 2024-02-13T03:31:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[27879]: audit 2024-02-13T03:31:31.120917+0000 mon.b (mon.1) 298 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-49", "mode": "writeback"}]: dispatch 2024-02-13T03:31:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[27879]: audit 2024-02-13T03:31:31.120960+0000 mon.a (mon.0) 2109 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[19769]: cluster 2024-02-13T03:31:29.337990+0000 mgr.y (mgr.24473) 419 : cluster [DBG] pgmap v575: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[19769]: audit 2024-02-13T03:31:30.114194+0000 mon.a (mon.0) 2103 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:31:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[19769]: cluster 2024-02-13T03:31:30.114637+0000 mon.a (mon.0) 2104 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-02-13T03:31:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[19769]: audit 2024-02-13T03:31:30.117069+0000 mon.a (mon.0) 2105 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[19769]: audit 2024-02-13T03:31:30.118283+0000 mon.b (mon.1) 297 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[19769]: audit 2024-02-13T03:31:31.116502+0000 mon.a (mon.0) 2106 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-49"}]': finished 2024-02-13T03:31:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[19769]: cluster 2024-02-13T03:31:31.117131+0000 mon.a (mon.0) 2107 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-02-13T03:31:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[19769]: audit 2024-02-13T03:31:31.119696+0000 mon.c (mon.2) 503 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[19769]: audit 2024-02-13T03:31:31.119781+0000 mon.a (mon.0) 2108 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-49", "mode": "writeback"}]: dispatch 2024-02-13T03:31:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[19769]: audit 2024-02-13T03:31:31.120917+0000 mon.b (mon.1) 298 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-49", "mode": "writeback"}]: dispatch 2024-02-13T03:31:31.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:31 smithi161 bash[19769]: audit 2024-02-13T03:31:31.120960+0000 mon.a (mon.0) 2109 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:31 smithi183 bash[20824]: cluster 2024-02-13T03:31:29.337990+0000 mgr.y (mgr.24473) 419 : cluster [DBG] pgmap v575: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:31 smithi183 bash[20824]: audit 2024-02-13T03:31:30.114194+0000 mon.a (mon.0) 2103 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:31:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:31 smithi183 bash[20824]: cluster 2024-02-13T03:31:30.114637+0000 mon.a (mon.0) 2104 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-02-13T03:31:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:31 smithi183 bash[20824]: audit 2024-02-13T03:31:30.117069+0000 mon.a (mon.0) 2105 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:31 smithi183 bash[20824]: audit 2024-02-13T03:31:30.118283+0000 mon.b (mon.1) 297 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:31 smithi183 bash[20824]: audit 2024-02-13T03:31:31.116502+0000 mon.a (mon.0) 2106 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-49"}]': finished 2024-02-13T03:31:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:31 smithi183 bash[20824]: cluster 2024-02-13T03:31:31.117131+0000 mon.a (mon.0) 2107 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-02-13T03:31:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:31 smithi183 bash[20824]: audit 2024-02-13T03:31:31.119696+0000 mon.c (mon.2) 503 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:31 smithi183 bash[20824]: audit 2024-02-13T03:31:31.119781+0000 mon.a (mon.0) 2108 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-49", "mode": "writeback"}]: dispatch 2024-02-13T03:31:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:31 smithi183 bash[20824]: audit 2024-02-13T03:31:31.120917+0000 mon.b (mon.1) 298 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-49", "mode": "writeback"}]: dispatch 2024-02-13T03:31:31.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:31 smithi183 bash[20824]: audit 2024-02-13T03:31:31.120960+0000 mon.a (mon.0) 2109 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:32.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:32 smithi161 bash[27879]: cluster 2024-02-13T03:31:32.109886+0000 mon.a (mon.0) 2110 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:31:32.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:32 smithi161 bash[27879]: audit 2024-02-13T03:31:32.117376+0000 mon.a (mon.0) 2111 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-49", "mode": "writeback"}]': finished 2024-02-13T03:31:32.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:32 smithi161 bash[27879]: audit 2024-02-13T03:31:32.117678+0000 mon.a (mon.0) 2112 : audit [INF] from='client.48356 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]': finished 2024-02-13T03:31:32.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:32 smithi161 bash[27879]: cluster 2024-02-13T03:31:32.117885+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-02-13T03:31:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:32 smithi161 bash[27879]: audit 2024-02-13T03:31:32.120183+0000 mon.c (mon.2) 504 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:32 smithi161 bash[27879]: audit 2024-02-13T03:31:32.120704+0000 mon.a (mon.0) 2114 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:32 smithi161 bash[19769]: cluster 2024-02-13T03:31:32.109886+0000 mon.a (mon.0) 2110 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:31:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:32 smithi161 bash[19769]: audit 2024-02-13T03:31:32.117376+0000 mon.a (mon.0) 2111 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-49", "mode": "writeback"}]': finished 2024-02-13T03:31:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:32 smithi161 bash[19769]: audit 2024-02-13T03:31:32.117678+0000 mon.a (mon.0) 2112 : audit [INF] from='client.48356 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]': finished 2024-02-13T03:31:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:32 smithi161 bash[19769]: cluster 2024-02-13T03:31:32.117885+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-02-13T03:31:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:32 smithi161 bash[19769]: audit 2024-02-13T03:31:32.120183+0000 mon.c (mon.2) 504 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:32 smithi161 bash[19769]: audit 2024-02-13T03:31:32.120704+0000 mon.a (mon.0) 2114 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:32 smithi183 bash[20824]: cluster 2024-02-13T03:31:32.109886+0000 mon.a (mon.0) 2110 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:31:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:32 smithi183 bash[20824]: audit 2024-02-13T03:31:32.117376+0000 mon.a (mon.0) 2111 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-49", "mode": "writeback"}]': finished 2024-02-13T03:31:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:32 smithi183 bash[20824]: audit 2024-02-13T03:31:32.117678+0000 mon.a (mon.0) 2112 : audit [INF] from='client.48356 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi161-61726-63"}]': finished 2024-02-13T03:31:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:32 smithi183 bash[20824]: cluster 2024-02-13T03:31:32.117885+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-02-13T03:31:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:32 smithi183 bash[20824]: audit 2024-02-13T03:31:32.120183+0000 mon.c (mon.2) 504 : audit [INF] from='client.? v1:172.21.15.161:0/4285839623' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:32 smithi183 bash[20824]: audit 2024-02-13T03:31:32.120704+0000 mon.a (mon.0) 2114 : audit [INF] from='client.48356 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]: dispatch 2024-02-13T03:31:33.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[19769]: cluster 2024-02-13T03:31:31.338826+0000 mgr.y (mgr.24473) 420 : cluster [DBG] pgmap v578: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-13T03:31:33.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[19769]: audit 2024-02-13T03:31:32.171068+0000 mon.a (mon.0) 2115 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:31:33.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[19769]: audit 2024-02-13T03:31:32.173128+0000 mon.b (mon.1) 299 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:31:33.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[19769]: audit 2024-02-13T03:31:33.126489+0000 mon.a (mon.0) 2116 : audit [INF] from='client.48356 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]': finished 2024-02-13T03:31:33.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[19769]: audit 2024-02-13T03:31:33.126608+0000 mon.a (mon.0) 2117 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:31:33.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[19769]: cluster 2024-02-13T03:31:33.126704+0000 mon.a (mon.0) 2118 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-02-13T03:31:33.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[19769]: audit 2024-02-13T03:31:33.128282+0000 mon.a (mon.0) 2119 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:31:33.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[19769]: audit 2024-02-13T03:31:33.130217+0000 mon.b (mon.1) 300 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:31:33.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[27879]: cluster 2024-02-13T03:31:31.338826+0000 mgr.y (mgr.24473) 420 : cluster [DBG] pgmap v578: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-13T03:31:33.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[27879]: audit 2024-02-13T03:31:32.171068+0000 mon.a (mon.0) 2115 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:31:33.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[27879]: audit 2024-02-13T03:31:32.173128+0000 mon.b (mon.1) 299 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:31:33.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[27879]: audit 2024-02-13T03:31:33.126489+0000 mon.a (mon.0) 2116 : audit [INF] from='client.48356 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]': finished 2024-02-13T03:31:33.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[27879]: audit 2024-02-13T03:31:33.126608+0000 mon.a (mon.0) 2117 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:31:33.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[27879]: cluster 2024-02-13T03:31:33.126704+0000 mon.a (mon.0) 2118 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-02-13T03:31:33.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[27879]: audit 2024-02-13T03:31:33.128282+0000 mon.a (mon.0) 2119 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:31:33.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:33 smithi161 bash[27879]: audit 2024-02-13T03:31:33.130217+0000 mon.b (mon.1) 300 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:31:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:33 smithi183 bash[20824]: cluster 2024-02-13T03:31:31.338826+0000 mgr.y (mgr.24473) 420 : cluster [DBG] pgmap v578: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-13T03:31:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:33 smithi183 bash[20824]: audit 2024-02-13T03:31:32.171068+0000 mon.a (mon.0) 2115 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:31:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:33 smithi183 bash[20824]: audit 2024-02-13T03:31:32.173128+0000 mon.b (mon.1) 299 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:31:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:33 smithi183 bash[20824]: audit 2024-02-13T03:31:33.126489+0000 mon.a (mon.0) 2116 : audit [INF] from='client.48356 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi161-61726-63"}]': finished 2024-02-13T03:31:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:33 smithi183 bash[20824]: audit 2024-02-13T03:31:33.126608+0000 mon.a (mon.0) 2117 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:31:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:33 smithi183 bash[20824]: cluster 2024-02-13T03:31:33.126704+0000 mon.a (mon.0) 2118 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-02-13T03:31:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:33 smithi183 bash[20824]: audit 2024-02-13T03:31:33.128282+0000 mon.a (mon.0) 2119 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:31:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:33 smithi183 bash[20824]: audit 2024-02-13T03:31:33.130217+0000 mon.b (mon.1) 300 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:31:34.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[19769]: audit 2024-02-13T03:31:33.147950+0000 mon.c (mon.2) 505 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[19769]: audit 2024-02-13T03:31:33.148326+0000 mon.a (mon.0) 2120 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[19769]: audit 2024-02-13T03:31:33.149463+0000 mon.c (mon.2) 506 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[19769]: audit 2024-02-13T03:31:33.149789+0000 mon.a (mon.0) 2121 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[19769]: audit 2024-02-13T03:31:33.150326+0000 mon.c (mon.2) 507 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi161-61726-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:34.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[19769]: audit 2024-02-13T03:31:33.150627+0000 mon.a (mon.0) 2122 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi161-61726-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:34.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[19769]: audit 2024-02-13T03:31:34.123639+0000 mon.a (mon.0) 2123 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:31:34.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[19769]: audit 2024-02-13T03:31:34.123912+0000 mon.a (mon.0) 2124 : audit [INF] from='client.48362 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi161-61726-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:34.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[19769]: cluster 2024-02-13T03:31:34.124133+0000 mon.a (mon.0) 2125 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-02-13T03:31:34.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[19769]: audit 2024-02-13T03:31:34.126801+0000 mon.a (mon.0) 2126 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:31:34.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[19769]: audit 2024-02-13T03:31:34.128548+0000 mon.c (mon.2) 508 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi161-61726-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[19769]: audit 2024-02-13T03:31:34.128874+0000 mon.b (mon.1) 301 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:31:34.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[19769]: audit 2024-02-13T03:31:34.130609+0000 mon.a (mon.0) 2127 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi161-61726-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[27879]: audit 2024-02-13T03:31:33.147950+0000 mon.c (mon.2) 505 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[27879]: audit 2024-02-13T03:31:33.148326+0000 mon.a (mon.0) 2120 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[27879]: audit 2024-02-13T03:31:33.149463+0000 mon.c (mon.2) 506 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[27879]: audit 2024-02-13T03:31:33.149789+0000 mon.a (mon.0) 2121 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[27879]: audit 2024-02-13T03:31:33.150326+0000 mon.c (mon.2) 507 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi161-61726-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:34.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[27879]: audit 2024-02-13T03:31:33.150627+0000 mon.a (mon.0) 2122 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi161-61726-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:34.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[27879]: audit 2024-02-13T03:31:34.123639+0000 mon.a (mon.0) 2123 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:31:34.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[27879]: audit 2024-02-13T03:31:34.123912+0000 mon.a (mon.0) 2124 : audit [INF] from='client.48362 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi161-61726-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:34.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[27879]: cluster 2024-02-13T03:31:34.124133+0000 mon.a (mon.0) 2125 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-02-13T03:31:34.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[27879]: audit 2024-02-13T03:31:34.126801+0000 mon.a (mon.0) 2126 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:31:34.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[27879]: audit 2024-02-13T03:31:34.128548+0000 mon.c (mon.2) 508 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi161-61726-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[27879]: audit 2024-02-13T03:31:34.128874+0000 mon.b (mon.1) 301 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:31:34.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:34 smithi161 bash[27879]: audit 2024-02-13T03:31:34.130609+0000 mon.a (mon.0) 2127 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi161-61726-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:34 smithi183 bash[20824]: audit 2024-02-13T03:31:33.147950+0000 mon.c (mon.2) 505 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:34 smithi183 bash[20824]: audit 2024-02-13T03:31:33.148326+0000 mon.a (mon.0) 2120 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:34 smithi183 bash[20824]: audit 2024-02-13T03:31:33.149463+0000 mon.c (mon.2) 506 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:34 smithi183 bash[20824]: audit 2024-02-13T03:31:33.149789+0000 mon.a (mon.0) 2121 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:34 smithi183 bash[20824]: audit 2024-02-13T03:31:33.150326+0000 mon.c (mon.2) 507 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi161-61726-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:34 smithi183 bash[20824]: audit 2024-02-13T03:31:33.150627+0000 mon.a (mon.0) 2122 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi161-61726-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:34 smithi183 bash[20824]: audit 2024-02-13T03:31:34.123639+0000 mon.a (mon.0) 2123 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:31:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:34 smithi183 bash[20824]: audit 2024-02-13T03:31:34.123912+0000 mon.a (mon.0) 2124 : audit [INF] from='client.48362 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi161-61726-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:34.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:34 smithi183 bash[20824]: cluster 2024-02-13T03:31:34.124133+0000 mon.a (mon.0) 2125 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-02-13T03:31:34.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:34 smithi183 bash[20824]: audit 2024-02-13T03:31:34.126801+0000 mon.a (mon.0) 2126 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:31:34.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:34 smithi183 bash[20824]: audit 2024-02-13T03:31:34.128548+0000 mon.c (mon.2) 508 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi161-61726-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:34.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:34 smithi183 bash[20824]: audit 2024-02-13T03:31:34.128874+0000 mon.b (mon.1) 301 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:31:34.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:34 smithi183 bash[20824]: audit 2024-02-13T03:31:34.130609+0000 mon.a (mon.0) 2127 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi161-61726-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:35 smithi161 bash[27879]: cluster 2024-02-13T03:31:33.339660+0000 mgr.y (mgr.24473) 421 : cluster [DBG] pgmap v581: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-13T03:31:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:35 smithi161 bash[27879]: cluster 2024-02-13T03:31:35.118804+0000 mon.a (mon.0) 2128 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:31:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:35 smithi161 bash[27879]: audit 2024-02-13T03:31:35.127224+0000 mon.a (mon.0) 2129 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:31:35.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:35 smithi161 bash[19769]: cluster 2024-02-13T03:31:33.339660+0000 mgr.y (mgr.24473) 421 : cluster [DBG] pgmap v581: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-13T03:31:35.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:35 smithi161 bash[19769]: cluster 2024-02-13T03:31:35.118804+0000 mon.a (mon.0) 2128 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:31:35.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:35 smithi161 bash[19769]: audit 2024-02-13T03:31:35.127224+0000 mon.a (mon.0) 2129 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:31:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:35 smithi183 bash[20824]: cluster 2024-02-13T03:31:33.339660+0000 mgr.y (mgr.24473) 421 : cluster [DBG] pgmap v581: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-13T03:31:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:35 smithi183 bash[20824]: cluster 2024-02-13T03:31:35.118804+0000 mon.a (mon.0) 2128 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:31:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:35 smithi183 bash[20824]: audit 2024-02-13T03:31:35.127224+0000 mon.a (mon.0) 2129 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:31:36.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:31:36 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:31:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[27879]: audit 2024-02-13T03:31:35.133158+0000 mon.b (mon.1) 302 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:31:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[27879]: cluster 2024-02-13T03:31:35.340388+0000 mgr.y (mgr.24473) 422 : cluster [DBG] pgmap v584: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[27879]: cluster 2024-02-13T03:31:35.551587+0000 mon.a (mon.0) 2130 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-02-13T03:31:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[27879]: audit 2024-02-13T03:31:35.553020+0000 mon.a (mon.0) 2131 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:31:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[27879]: audit 2024-02-13T03:31:36.046685+0000 mgr.y (mgr.24473) 423 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[27879]: audit 2024-02-13T03:31:36.137074+0000 mon.a (mon.0) 2132 : audit [INF] from='client.48362 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi161-61726-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi161-61726-64"}]': finished 2024-02-13T03:31:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[27879]: audit 2024-02-13T03:31:36.137380+0000 mon.a (mon.0) 2133 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-13T03:31:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[27879]: cluster 2024-02-13T03:31:36.137523+0000 mon.a (mon.0) 2134 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-02-13T03:31:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[27879]: audit 2024-02-13T03:31:36.140577+0000 mon.a (mon.0) 2135 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-13T03:31:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[27879]: audit 2024-02-13T03:31:36.142621+0000 mon.b (mon.1) 303 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-13T03:31:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[19769]: audit 2024-02-13T03:31:35.133158+0000 mon.b (mon.1) 302 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:31:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[19769]: cluster 2024-02-13T03:31:35.340388+0000 mgr.y (mgr.24473) 422 : cluster [DBG] pgmap v584: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[19769]: cluster 2024-02-13T03:31:35.551587+0000 mon.a (mon.0) 2130 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-02-13T03:31:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[19769]: audit 2024-02-13T03:31:35.553020+0000 mon.a (mon.0) 2131 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:31:36.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[19769]: audit 2024-02-13T03:31:36.046685+0000 mgr.y (mgr.24473) 423 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:36.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[19769]: audit 2024-02-13T03:31:36.137074+0000 mon.a (mon.0) 2132 : audit [INF] from='client.48362 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi161-61726-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi161-61726-64"}]': finished 2024-02-13T03:31:36.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[19769]: audit 2024-02-13T03:31:36.137380+0000 mon.a (mon.0) 2133 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-13T03:31:36.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[19769]: cluster 2024-02-13T03:31:36.137523+0000 mon.a (mon.0) 2134 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-02-13T03:31:36.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[19769]: audit 2024-02-13T03:31:36.140577+0000 mon.a (mon.0) 2135 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-13T03:31:36.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:36 smithi161 bash[19769]: audit 2024-02-13T03:31:36.142621+0000 mon.b (mon.1) 303 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-13T03:31:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:36 smithi183 bash[20824]: audit 2024-02-13T03:31:35.133158+0000 mon.b (mon.1) 302 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:31:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:36 smithi183 bash[20824]: cluster 2024-02-13T03:31:35.340388+0000 mgr.y (mgr.24473) 422 : cluster [DBG] pgmap v584: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:36 smithi183 bash[20824]: cluster 2024-02-13T03:31:35.551587+0000 mon.a (mon.0) 2130 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-02-13T03:31:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:36 smithi183 bash[20824]: audit 2024-02-13T03:31:35.553020+0000 mon.a (mon.0) 2131 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:31:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:36 smithi183 bash[20824]: audit 2024-02-13T03:31:36.046685+0000 mgr.y (mgr.24473) 423 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:36 smithi183 bash[20824]: audit 2024-02-13T03:31:36.137074+0000 mon.a (mon.0) 2132 : audit [INF] from='client.48362 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi161-61726-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi161-61726-64"}]': finished 2024-02-13T03:31:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:36 smithi183 bash[20824]: audit 2024-02-13T03:31:36.137380+0000 mon.a (mon.0) 2133 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-13T03:31:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:36 smithi183 bash[20824]: cluster 2024-02-13T03:31:36.137523+0000 mon.a (mon.0) 2134 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-02-13T03:31:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:36 smithi183 bash[20824]: audit 2024-02-13T03:31:36.140577+0000 mon.a (mon.0) 2135 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-13T03:31:36.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:36 smithi183 bash[20824]: audit 2024-02-13T03:31:36.142621+0000 mon.b (mon.1) 303 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-13T03:31:38.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:38 smithi161 bash[27879]: audit 2024-02-13T03:31:37.142621+0000 mon.a (mon.0) 2136 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "target_max_objects","val": "1"}]': finished 2024-02-13T03:31:38.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:38 smithi161 bash[27879]: cluster 2024-02-13T03:31:37.142906+0000 mon.a (mon.0) 2137 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-02-13T03:31:38.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:38 smithi161 bash[19769]: audit 2024-02-13T03:31:37.142621+0000 mon.a (mon.0) 2136 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "target_max_objects","val": "1"}]': finished 2024-02-13T03:31:38.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:38 smithi161 bash[19769]: cluster 2024-02-13T03:31:37.142906+0000 mon.a (mon.0) 2137 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-02-13T03:31:38.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:38 smithi183 bash[20824]: audit 2024-02-13T03:31:37.142621+0000 mon.a (mon.0) 2136 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-49","var": "target_max_objects","val": "1"}]': finished 2024-02-13T03:31:38.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:38 smithi183 bash[20824]: cluster 2024-02-13T03:31:37.142906+0000 mon.a (mon.0) 2137 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-02-13T03:31:39.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:39 smithi161 bash[27879]: cluster 2024-02-13T03:31:37.341303+0000 mgr.y (mgr.24473) 424 : cluster [DBG] pgmap v587: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-13T03:31:39.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:39 smithi161 bash[27879]: cluster 2024-02-13T03:31:38.132803+0000 mon.a (mon.0) 2138 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-13T03:31:39.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:39 smithi161 bash[27879]: cluster 2024-02-13T03:31:38.145321+0000 mon.a (mon.0) 2139 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-02-13T03:31:39.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:39 smithi161 bash[27879]: audit 2024-02-13T03:31:38.148070+0000 mon.c (mon.2) 509 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:39.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:39 smithi161 bash[27879]: audit 2024-02-13T03:31:38.148514+0000 mon.a (mon.0) 2140 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:39 smithi161 bash[19769]: cluster 2024-02-13T03:31:37.341303+0000 mgr.y (mgr.24473) 424 : cluster [DBG] pgmap v587: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-13T03:31:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:39 smithi161 bash[19769]: cluster 2024-02-13T03:31:38.132803+0000 mon.a (mon.0) 2138 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-13T03:31:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:39 smithi161 bash[19769]: cluster 2024-02-13T03:31:38.145321+0000 mon.a (mon.0) 2139 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-02-13T03:31:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:39 smithi161 bash[19769]: audit 2024-02-13T03:31:38.148070+0000 mon.c (mon.2) 509 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:39 smithi161 bash[19769]: audit 2024-02-13T03:31:38.148514+0000 mon.a (mon.0) 2140 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:39 smithi183 bash[20824]: cluster 2024-02-13T03:31:37.341303+0000 mgr.y (mgr.24473) 424 : cluster [DBG] pgmap v587: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-13T03:31:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:39 smithi183 bash[20824]: cluster 2024-02-13T03:31:38.132803+0000 mon.a (mon.0) 2138 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-13T03:31:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:39 smithi183 bash[20824]: cluster 2024-02-13T03:31:38.145321+0000 mon.a (mon.0) 2139 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-02-13T03:31:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:39 smithi183 bash[20824]: audit 2024-02-13T03:31:38.148070+0000 mon.c (mon.2) 509 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:39 smithi183 bash[20824]: audit 2024-02-13T03:31:38.148514+0000 mon.a (mon.0) 2140 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:39.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:31:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:31:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:31:40.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:40 smithi161 bash[19769]: audit 2024-02-13T03:31:39.147869+0000 mon.a (mon.0) 2141 : audit [INF] from='client.48362 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]': finished 2024-02-13T03:31:40.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:40 smithi161 bash[19769]: cluster 2024-02-13T03:31:39.147970+0000 mon.a (mon.0) 2142 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-02-13T03:31:40.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:40 smithi161 bash[19769]: audit 2024-02-13T03:31:39.152333+0000 mon.c (mon.2) 510 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:40.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:40 smithi161 bash[19769]: audit 2024-02-13T03:31:39.164840+0000 mon.a (mon.0) 2143 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:40.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:40 smithi161 bash[27879]: audit 2024-02-13T03:31:39.147869+0000 mon.a (mon.0) 2141 : audit [INF] from='client.48362 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]': finished 2024-02-13T03:31:40.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:40 smithi161 bash[27879]: cluster 2024-02-13T03:31:39.147970+0000 mon.a (mon.0) 2142 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-02-13T03:31:40.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:40 smithi161 bash[27879]: audit 2024-02-13T03:31:39.152333+0000 mon.c (mon.2) 510 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:40.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:40 smithi161 bash[27879]: audit 2024-02-13T03:31:39.164840+0000 mon.a (mon.0) 2143 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:40.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:40 smithi183 bash[20824]: audit 2024-02-13T03:31:39.147869+0000 mon.a (mon.0) 2141 : audit [INF] from='client.48362 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi161-61726-64"}]': finished 2024-02-13T03:31:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:40 smithi183 bash[20824]: cluster 2024-02-13T03:31:39.147970+0000 mon.a (mon.0) 2142 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-02-13T03:31:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:40 smithi183 bash[20824]: audit 2024-02-13T03:31:39.152333+0000 mon.c (mon.2) 510 : audit [INF] from='client.? v1:172.21.15.161:0/3669905142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:40 smithi183 bash[20824]: audit 2024-02-13T03:31:39.164840+0000 mon.a (mon.0) 2143 : audit [INF] from='client.48362 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]: dispatch 2024-02-13T03:31:41.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:31:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:31:41] "GET /metrics HTTP/1.1" 200 186965 "" "Prometheus/2.33.4" 2024-02-13T03:31:41.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:41 smithi161 bash[19769]: cluster 2024-02-13T03:31:39.342249+0000 mgr.y (mgr.24473) 425 : cluster [DBG] pgmap v590: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-13T03:31:41.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:41 smithi161 bash[19769]: audit 2024-02-13T03:31:40.165167+0000 mon.a (mon.0) 2144 : audit [INF] from='client.48362 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]': finished 2024-02-13T03:31:41.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:41 smithi161 bash[19769]: cluster 2024-02-13T03:31:40.165257+0000 mon.a (mon.0) 2145 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-02-13T03:31:41.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:41 smithi161 bash[19769]: audit 2024-02-13T03:31:40.185342+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:41.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:41 smithi161 bash[19769]: audit 2024-02-13T03:31:40.186911+0000 mon.a (mon.0) 2147 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:41.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:41 smithi161 bash[19769]: audit 2024-02-13T03:31:40.187360+0000 mon.a (mon.0) 2148 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi161-61726-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:41.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:41 smithi161 bash[27879]: cluster 2024-02-13T03:31:39.342249+0000 mgr.y (mgr.24473) 425 : cluster [DBG] pgmap v590: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-13T03:31:41.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:41 smithi161 bash[27879]: audit 2024-02-13T03:31:40.165167+0000 mon.a (mon.0) 2144 : audit [INF] from='client.48362 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]': finished 2024-02-13T03:31:41.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:41 smithi161 bash[27879]: cluster 2024-02-13T03:31:40.165257+0000 mon.a (mon.0) 2145 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-02-13T03:31:41.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:41 smithi161 bash[27879]: audit 2024-02-13T03:31:40.185342+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:41.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:41 smithi161 bash[27879]: audit 2024-02-13T03:31:40.186911+0000 mon.a (mon.0) 2147 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:41.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:41 smithi161 bash[27879]: audit 2024-02-13T03:31:40.187360+0000 mon.a (mon.0) 2148 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi161-61726-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:41 smithi183 bash[20824]: cluster 2024-02-13T03:31:39.342249+0000 mgr.y (mgr.24473) 425 : cluster [DBG] pgmap v590: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-13T03:31:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:41 smithi183 bash[20824]: audit 2024-02-13T03:31:40.165167+0000 mon.a (mon.0) 2144 : audit [INF] from='client.48362 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi161-61726-64"}]': finished 2024-02-13T03:31:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:41 smithi183 bash[20824]: cluster 2024-02-13T03:31:40.165257+0000 mon.a (mon.0) 2145 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-02-13T03:31:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:41 smithi183 bash[20824]: audit 2024-02-13T03:31:40.185342+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:41 smithi183 bash[20824]: audit 2024-02-13T03:31:40.186911+0000 mon.a (mon.0) 2147 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:41 smithi183 bash[20824]: audit 2024-02-13T03:31:40.187360+0000 mon.a (mon.0) 2148 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi161-61726-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:31:42.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:42 smithi161 bash[19769]: audit 2024-02-13T03:31:41.179373+0000 mon.a (mon.0) 2149 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi161-61726-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:42.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:42 smithi161 bash[19769]: cluster 2024-02-13T03:31:41.179537+0000 mon.a (mon.0) 2150 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-02-13T03:31:42.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:42 smithi161 bash[19769]: audit 2024-02-13T03:31:41.180566+0000 mon.a (mon.0) 2151 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi161-61726-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:42.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:42 smithi161 bash[27879]: audit 2024-02-13T03:31:41.179373+0000 mon.a (mon.0) 2149 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi161-61726-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:42.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:42 smithi161 bash[27879]: cluster 2024-02-13T03:31:41.179537+0000 mon.a (mon.0) 2150 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-02-13T03:31:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:42 smithi161 bash[27879]: audit 2024-02-13T03:31:41.180566+0000 mon.a (mon.0) 2151 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi161-61726-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:42.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:42 smithi183 bash[20824]: audit 2024-02-13T03:31:41.179373+0000 mon.a (mon.0) 2149 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi161-61726-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:31:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:42 smithi183 bash[20824]: cluster 2024-02-13T03:31:41.179537+0000 mon.a (mon.0) 2150 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-02-13T03:31:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:42 smithi183 bash[20824]: audit 2024-02-13T03:31:41.180566+0000 mon.a (mon.0) 2151 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi161-61726-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:43.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:43 smithi161 bash[27879]: cluster 2024-02-13T03:31:41.342949+0000 mgr.y (mgr.24473) 426 : cluster [DBG] pgmap v593: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:31:43.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:43 smithi161 bash[27879]: cluster 2024-02-13T03:31:42.611288+0000 mon.a (mon.0) 2152 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-02-13T03:31:43.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:43 smithi161 bash[27879]: audit 2024-02-13T03:31:43.187964+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi161-61726-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi161-61726-65"}]': finished 2024-02-13T03:31:43.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:43 smithi161 bash[27879]: cluster 2024-02-13T03:31:43.188305+0000 mon.a (mon.0) 2154 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-02-13T03:31:43.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:43 smithi161 bash[19769]: cluster 2024-02-13T03:31:41.342949+0000 mgr.y (mgr.24473) 426 : cluster [DBG] pgmap v593: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:31:43.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:43 smithi161 bash[19769]: cluster 2024-02-13T03:31:42.611288+0000 mon.a (mon.0) 2152 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-02-13T03:31:43.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:43 smithi161 bash[19769]: audit 2024-02-13T03:31:43.187964+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi161-61726-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi161-61726-65"}]': finished 2024-02-13T03:31:43.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:43 smithi161 bash[19769]: cluster 2024-02-13T03:31:43.188305+0000 mon.a (mon.0) 2154 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-02-13T03:31:43.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:43 smithi183 bash[20824]: cluster 2024-02-13T03:31:41.342949+0000 mgr.y (mgr.24473) 426 : cluster [DBG] pgmap v593: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:31:43.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:43 smithi183 bash[20824]: cluster 2024-02-13T03:31:42.611288+0000 mon.a (mon.0) 2152 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-02-13T03:31:43.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:43 smithi183 bash[20824]: audit 2024-02-13T03:31:43.187964+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi161-61726-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi161-61726-65"}]': finished 2024-02-13T03:31:43.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:43 smithi183 bash[20824]: cluster 2024-02-13T03:31:43.188305+0000 mon.a (mon.0) 2154 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-02-13T03:31:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:45 smithi183 bash[20824]: cluster 2024-02-13T03:31:43.343759+0000 mgr.y (mgr.24473) 427 : cluster [DBG] pgmap v596: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:31:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:45 smithi183 bash[20824]: cluster 2024-02-13T03:31:44.189629+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-02-13T03:31:45.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:45 smithi161 bash[27879]: cluster 2024-02-13T03:31:43.343759+0000 mgr.y (mgr.24473) 427 : cluster [DBG] pgmap v596: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:31:45.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:45 smithi161 bash[27879]: cluster 2024-02-13T03:31:44.189629+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-02-13T03:31:45.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:45 smithi161 bash[19769]: cluster 2024-02-13T03:31:43.343759+0000 mgr.y (mgr.24473) 427 : cluster [DBG] pgmap v596: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:31:45.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:45 smithi161 bash[19769]: cluster 2024-02-13T03:31:44.189629+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-02-13T03:31:46.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:31:46 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:31:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:46 smithi183 bash[20824]: cluster 2024-02-13T03:31:45.204991+0000 mon.a (mon.0) 2156 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-02-13T03:31:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:46 smithi183 bash[20824]: audit 2024-02-13T03:31:45.206216+0000 mon.a (mon.0) 2157 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:46.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:46 smithi161 bash[27879]: cluster 2024-02-13T03:31:45.204991+0000 mon.a (mon.0) 2156 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-02-13T03:31:46.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:46 smithi161 bash[27879]: audit 2024-02-13T03:31:45.206216+0000 mon.a (mon.0) 2157 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:46.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:46 smithi161 bash[19769]: cluster 2024-02-13T03:31:45.204991+0000 mon.a (mon.0) 2156 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-02-13T03:31:46.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:46 smithi161 bash[19769]: audit 2024-02-13T03:31:45.206216+0000 mon.a (mon.0) 2157 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:47.227 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-02-13T03:31:47.227 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7034 ms) 2024-02-13T03:31:47.227 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-02-13T03:31:47.227 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7051 ms) 2024-02-13T03:31:47.227 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-02-13T03:31:47.227 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7075 ms) 2024-02-13T03:31:47.227 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-02-13T03:31:47.227 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7040 ms) 2024-02-13T03:31:47.228 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-02-13T03:31:47.228 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7054 ms) 2024-02-13T03:31:47.228 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139636 ms total) 2024-02-13T03:31:47.228 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: 2024-02-13T03:31:47.228 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-02-13T03:31:47.228 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [==========] 56 tests from 4 test suites ran. (301015 ms total) 2024-02-13T03:31:47.228 INFO:tasks.workunit.client.0.smithi161.stdout: api_aio_pp: [ PASSED ] 56 tests. 2024-02-13T03:31:47.230 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.230 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61738 2024-02-13T03:31:47.230 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61738 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61916 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61916 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61886 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61886 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61762 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61762 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62118 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 62118 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61730 2024-02-13T03:31:47.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61730 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62138 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 62138 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61996 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61996 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62244 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 62244 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62123 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 62123 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.232 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62308 2024-02-13T03:31:47.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 62308 2024-02-13T03:31:47.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61714 2024-02-13T03:31:47.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61714 2024-02-13T03:31:47.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62273 2024-02-13T03:31:47.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 62273 2024-02-13T03:31:47.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=62158 2024-02-13T03:31:47.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 62158 2024-02-13T03:31:47.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ for t in "${!pids[@]}" 2024-02-13T03:31:47.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=61898 2024-02-13T03:31:47.233 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 61898 2024-02-13T03:31:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:47 smithi183 bash[20824]: cluster 2024-02-13T03:31:45.344660+0000 mgr.y (mgr.24473) 428 : cluster [DBG] pgmap v599: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:47 smithi183 bash[20824]: audit 2024-02-13T03:31:46.058204+0000 mgr.y (mgr.24473) 429 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:47 smithi183 bash[20824]: audit 2024-02-13T03:31:46.215346+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi161-61726-65"}]': finished 2024-02-13T03:31:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:47 smithi183 bash[20824]: cluster 2024-02-13T03:31:46.215470+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-02-13T03:31:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:47 smithi183 bash[20824]: audit 2024-02-13T03:31:46.218243+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:47 smithi183 bash[20824]: audit 2024-02-13T03:31:47.172957+0000 mon.a (mon.0) 2161 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:47 smithi183 bash[20824]: audit 2024-02-13T03:31:47.174662+0000 mon.b (mon.1) 304 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[27879]: cluster 2024-02-13T03:31:45.344660+0000 mgr.y (mgr.24473) 428 : cluster [DBG] pgmap v599: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[27879]: audit 2024-02-13T03:31:46.058204+0000 mgr.y (mgr.24473) 429 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[27879]: audit 2024-02-13T03:31:46.215346+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi161-61726-65"}]': finished 2024-02-13T03:31:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[27879]: cluster 2024-02-13T03:31:46.215470+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-02-13T03:31:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[27879]: audit 2024-02-13T03:31:46.218243+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[27879]: audit 2024-02-13T03:31:47.172957+0000 mon.a (mon.0) 2161 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[27879]: audit 2024-02-13T03:31:47.174662+0000 mon.b (mon.1) 304 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[19769]: cluster 2024-02-13T03:31:45.344660+0000 mgr.y (mgr.24473) 428 : cluster [DBG] pgmap v599: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[19769]: audit 2024-02-13T03:31:46.058204+0000 mgr.y (mgr.24473) 429 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[19769]: audit 2024-02-13T03:31:46.215346+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi161-61726-65"}]': finished 2024-02-13T03:31:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[19769]: cluster 2024-02-13T03:31:46.215470+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-02-13T03:31:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[19769]: audit 2024-02-13T03:31:46.218243+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi161-61726-65"}]: dispatch 2024-02-13T03:31:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[19769]: audit 2024-02-13T03:31:47.172957+0000 mon.a (mon.0) 2161 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:47 smithi161 bash[19769]: audit 2024-02-13T03:31:47.174662+0000 mon.b (mon.1) 304 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:48.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:48 smithi161 bash[27879]: audit 2024-02-13T03:31:47.219427+0000 mon.a (mon.0) 2162 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi161-61726-65"}]': finished 2024-02-13T03:31:48.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:48 smithi161 bash[27879]: audit 2024-02-13T03:31:47.219838+0000 mon.a (mon.0) 2163 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:31:48.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:48 smithi161 bash[27879]: cluster 2024-02-13T03:31:47.219956+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-02-13T03:31:48.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:48 smithi161 bash[27879]: audit 2024-02-13T03:31:47.221642+0000 mon.a (mon.0) 2165 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:48.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:48 smithi161 bash[27879]: audit 2024-02-13T03:31:47.223540+0000 mon.b (mon.1) 305 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:48.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:48 smithi161 bash[27879]: audit 2024-02-13T03:31:48.222326+0000 mon.a (mon.0) 2166 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]': finished 2024-02-13T03:31:48.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:48 smithi161 bash[19769]: audit 2024-02-13T03:31:47.219427+0000 mon.a (mon.0) 2162 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi161-61726-65"}]': finished 2024-02-13T03:31:48.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:48 smithi161 bash[19769]: audit 2024-02-13T03:31:47.219838+0000 mon.a (mon.0) 2163 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:31:48.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:48 smithi161 bash[19769]: cluster 2024-02-13T03:31:47.219956+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-02-13T03:31:48.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:48 smithi161 bash[19769]: audit 2024-02-13T03:31:47.221642+0000 mon.a (mon.0) 2165 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:48.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:48 smithi161 bash[19769]: audit 2024-02-13T03:31:47.223540+0000 mon.b (mon.1) 305 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:48.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:48 smithi161 bash[19769]: audit 2024-02-13T03:31:48.222326+0000 mon.a (mon.0) 2166 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]': finished 2024-02-13T03:31:48.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:48 smithi183 bash[20824]: audit 2024-02-13T03:31:47.219427+0000 mon.a (mon.0) 2162 : audit [INF] from='client.? v1:172.21.15.161:0/2504147686' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi161-61726-65"}]': finished 2024-02-13T03:31:48.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:48 smithi183 bash[20824]: audit 2024-02-13T03:31:47.219838+0000 mon.a (mon.0) 2163 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:31:48.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:48 smithi183 bash[20824]: cluster 2024-02-13T03:31:47.219956+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-02-13T03:31:48.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:48 smithi183 bash[20824]: audit 2024-02-13T03:31:47.221642+0000 mon.a (mon.0) 2165 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:48.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:48 smithi183 bash[20824]: audit 2024-02-13T03:31:47.223540+0000 mon.b (mon.1) 305 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:48.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:48 smithi183 bash[20824]: audit 2024-02-13T03:31:48.222326+0000 mon.a (mon.0) 2166 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]': finished 2024-02-13T03:31:49.250 INFO:tasks.workunit.client.0.smithi161.stdout:b4c720e9:::166:head 2024-02-13T03:31:49.250 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:e694b040:::167:head 2024-02-13T03:31:49.250 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:afa38db2:::168:head 2024-02-13T03:31:49.250 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:77ba9f53:::169:head 2024-02-13T03:31:49.250 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:87495034:::170:head 2024-02-13T03:31:49.250 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:7c96bf0e:::171:head 2024-02-13T03:31:49.250 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:dbe346cc:::172:head 2024-02-13T03:31:49.250 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:e943ec24:::173:head 2024-02-13T03:31:49.250 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:f97a9c0c:::174:head 2024-02-13T03:31:49.251 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:6f26e74d:::175:head 2024-02-13T03:31:49.251 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:4f95e106:::176:head 2024-02-13T03:31:49.251 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:0e6f2f8f:::177:head 2024-02-13T03:31:49.251 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:05db05f1:::178:head 2024-02-13T03:31:49.251 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:38a78d66:::179:head 2024-02-13T03:31:49.251 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:d095610b:::180:head 2024-02-13T03:31:49.251 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:a1a9d709:::181:head 2024-02-13T03:31:49.251 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:1e5d39db:::182:head 2024-02-13T03:31:49.251 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:f7df4fb9:::183:head 2024-02-13T03:31:49.251 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:03a7f161:::184:head 2024-02-13T03:31:49.251 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:ba70721e:::185:head 2024-02-13T03:31:49.252 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:28e5662d:::186:head 2024-02-13T03:31:49.252 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:973d52de:::187:head 2024-02-13T03:31:49.252 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:4303eb1c:::188:head 2024-02-13T03:31:49.252 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:b990b48e:::189:head 2024-02-13T03:31:49.252 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:29b8165b:::190:head 2024-02-13T03:31:49.252 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:3547f197:::191:head 2024-02-13T03:31:49.252 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:7e260936:::192:head 2024-02-13T03:31:49.252 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:1abec7b1:::193:head 2024-02-13T03:31:49.252 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:10fdda93:::194:head 2024-02-13T03:31:49.252 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:15817eea:::195:head 2024-02-13T03:31:49.252 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:770bab57:::196:head 2024-02-13T03:31:49.252 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:ed9e13e7:::197:head 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:71471a8f:::198:head 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: checking for 190:10fb1d02:::199:head 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8068 ms) 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795042,0 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: first is 1707795042 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795042,0 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795042,0 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795042,1707795044,1707795045,0 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795042,1707795044,1707795045,0 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795042,1707795044,1707795045,0 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795042,1707795044,1707795045,1707795047,1707795048,0 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795042,1707795044,1707795045,1707795047,1707795048,0 2024-02-13T03:31:49.253 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795042,1707795044,1707795045,1707795047,1707795048,0 2024-02-13T03:31:49.254 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795042,1707795044,1707795045,1707795047,1707795048,1707795050,1707795051,0 2024-02-13T03:31:49.254 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795042,1707795044,1707795045,1707795047,1707795048,1707795050,1707795051,0 2024-02-13T03:31:49.254 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795042,1707795044,1707795045,1707795047,1707795048,1707795050,1707795051,0 2024-02-13T03:31:49.254 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795045,1707795047,1707795048,1707795050,1707795051,1707795053,1707795054,0 2024-02-13T03:31:49.254 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: first now 1707795045, trimmed 2024-02-13T03:31:49.254 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20173 ms) 2024-02-13T03:31:49.254 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-02-13T03:31:49.254 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: foo0 2024-02-13T03:31:49.254 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-13T03:31:49.254 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14082 ms) 2024-02-13T03:31:49.254 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-02-13T03:31:49.254 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ProxyRead (18110 ms) 2024-02-13T03:31:49.254 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-02-13T03:31:49.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:49 smithi161 bash[19769]: cluster 2024-02-13T03:31:47.345535+0000 mgr.y (mgr.24473) 430 : cluster [DBG] pgmap v602: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:49.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:49 smithi161 bash[19769]: cluster 2024-02-13T03:31:48.222438+0000 mon.a (mon.0) 2167 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-02-13T03:31:49.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:49 smithi161 bash[19769]: audit 2024-02-13T03:31:48.264414+0000 mon.a (mon.0) 2168 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:49.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:49 smithi161 bash[19769]: audit 2024-02-13T03:31:48.265899+0000 mon.a (mon.0) 2169 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:49.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:49 smithi161 bash[19769]: audit 2024-02-13T03:31:48.266293+0000 mon.b (mon.1) 306 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:49.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:49 smithi161 bash[19769]: audit 2024-02-13T03:31:48.268022+0000 mon.b (mon.1) 307 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:49 smithi161 bash[27879]: cluster 2024-02-13T03:31:47.345535+0000 mgr.y (mgr.24473) 430 : cluster [DBG] pgmap v602: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:49 smithi161 bash[27879]: cluster 2024-02-13T03:31:48.222438+0000 mon.a (mon.0) 2167 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-02-13T03:31:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:49 smithi161 bash[27879]: audit 2024-02-13T03:31:48.264414+0000 mon.a (mon.0) 2168 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:49 smithi161 bash[27879]: audit 2024-02-13T03:31:48.265899+0000 mon.a (mon.0) 2169 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:49 smithi161 bash[27879]: audit 2024-02-13T03:31:48.266293+0000 mon.b (mon.1) 306 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:49 smithi161 bash[27879]: audit 2024-02-13T03:31:48.268022+0000 mon.b (mon.1) 307 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:49.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:31:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:31:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:31:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:49 smithi183 bash[20824]: cluster 2024-02-13T03:31:47.345535+0000 mgr.y (mgr.24473) 430 : cluster [DBG] pgmap v602: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:49 smithi183 bash[20824]: cluster 2024-02-13T03:31:48.222438+0000 mon.a (mon.0) 2167 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-02-13T03:31:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:49 smithi183 bash[20824]: audit 2024-02-13T03:31:48.264414+0000 mon.a (mon.0) 2168 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:49 smithi183 bash[20824]: audit 2024-02-13T03:31:48.265899+0000 mon.a (mon.0) 2169 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:49 smithi183 bash[20824]: audit 2024-02-13T03:31:48.266293+0000 mon.b (mon.1) 306 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:49 smithi183 bash[20824]: audit 2024-02-13T03:31:48.268022+0000 mon.b (mon.1) 307 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-49"}]: dispatch 2024-02-13T03:31:50.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:50 smithi161 bash[27879]: cluster 2024-02-13T03:31:49.243901+0000 mon.a (mon.0) 2170 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-02-13T03:31:50.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:50 smithi161 bash[27879]: audit 2024-02-13T03:31:49.786170+0000 mon.a (mon.0) 2171 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:31:50.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:50 smithi161 bash[19769]: cluster 2024-02-13T03:31:49.243901+0000 mon.a (mon.0) 2170 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-02-13T03:31:50.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:50 smithi161 bash[19769]: audit 2024-02-13T03:31:49.786170+0000 mon.a (mon.0) 2171 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:31:50.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:50 smithi183 bash[20824]: cluster 2024-02-13T03:31:49.243901+0000 mon.a (mon.0) 2170 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-02-13T03:31:50.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:50 smithi183 bash[20824]: audit 2024-02-13T03:31:49.786170+0000 mon.a (mon.0) 2171 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:31:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:31:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:31:51] "GET /metrics HTTP/1.1" 200 184620 "" "Prometheus/2.33.4" 2024-02-13T03:31:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:51 smithi161 bash[27879]: cluster 2024-02-13T03:31:49.346290+0000 mgr.y (mgr.24473) 431 : cluster [DBG] pgmap v605: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-13T03:31:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:51 smithi161 bash[27879]: cluster 2024-02-13T03:31:50.249627+0000 mon.a (mon.0) 2172 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-13T03:31:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:51 smithi161 bash[27879]: cluster 2024-02-13T03:31:50.266543+0000 mon.a (mon.0) 2173 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-02-13T03:31:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:51 smithi161 bash[27879]: audit 2024-02-13T03:31:50.273898+0000 mon.a (mon.0) 2174 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:51 smithi161 bash[27879]: audit 2024-02-13T03:31:50.275337+0000 mon.b (mon.1) 308 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:51 smithi161 bash[19769]: cluster 2024-02-13T03:31:49.346290+0000 mgr.y (mgr.24473) 431 : cluster [DBG] pgmap v605: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-13T03:31:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:51 smithi161 bash[19769]: cluster 2024-02-13T03:31:50.249627+0000 mon.a (mon.0) 2172 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-13T03:31:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:51 smithi161 bash[19769]: cluster 2024-02-13T03:31:50.266543+0000 mon.a (mon.0) 2173 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-02-13T03:31:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:51 smithi161 bash[19769]: audit 2024-02-13T03:31:50.273898+0000 mon.a (mon.0) 2174 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:51 smithi161 bash[19769]: audit 2024-02-13T03:31:50.275337+0000 mon.b (mon.1) 308 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:51 smithi183 bash[20824]: cluster 2024-02-13T03:31:49.346290+0000 mgr.y (mgr.24473) 431 : cluster [DBG] pgmap v605: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-13T03:31:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:51 smithi183 bash[20824]: cluster 2024-02-13T03:31:50.249627+0000 mon.a (mon.0) 2172 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-13T03:31:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:51 smithi183 bash[20824]: cluster 2024-02-13T03:31:50.266543+0000 mon.a (mon.0) 2173 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-02-13T03:31:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:51 smithi183 bash[20824]: audit 2024-02-13T03:31:50.273898+0000 mon.a (mon.0) 2174 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:51 smithi183 bash[20824]: audit 2024-02-13T03:31:50.275337+0000 mon.b (mon.1) 308 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:52.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:52 smithi161 bash[27879]: audit 2024-02-13T03:31:51.273984+0000 mon.a (mon.0) 2175 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:52.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:52 smithi161 bash[27879]: cluster 2024-02-13T03:31:51.274140+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-02-13T03:31:52.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:52 smithi161 bash[27879]: audit 2024-02-13T03:31:51.314885+0000 mon.a (mon.0) 2177 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:52.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:52 smithi161 bash[27879]: audit 2024-02-13T03:31:51.315923+0000 mon.a (mon.0) 2178 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-51"}]: dispatch 2024-02-13T03:31:52.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:52 smithi161 bash[27879]: audit 2024-02-13T03:31:51.317053+0000 mon.b (mon.1) 309 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:52.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:52 smithi161 bash[27879]: audit 2024-02-13T03:31:51.318126+0000 mon.b (mon.1) 310 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-51"}]: dispatch 2024-02-13T03:31:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:52 smithi161 bash[19769]: audit 2024-02-13T03:31:51.273984+0000 mon.a (mon.0) 2175 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:52 smithi161 bash[19769]: cluster 2024-02-13T03:31:51.274140+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-02-13T03:31:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:52 smithi161 bash[19769]: audit 2024-02-13T03:31:51.314885+0000 mon.a (mon.0) 2177 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:52 smithi161 bash[19769]: audit 2024-02-13T03:31:51.315923+0000 mon.a (mon.0) 2178 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-51"}]: dispatch 2024-02-13T03:31:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:52 smithi161 bash[19769]: audit 2024-02-13T03:31:51.317053+0000 mon.b (mon.1) 309 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:52.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:52 smithi161 bash[19769]: audit 2024-02-13T03:31:51.318126+0000 mon.b (mon.1) 310 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-51"}]: dispatch 2024-02-13T03:31:52.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:52 smithi183 bash[20824]: audit 2024-02-13T03:31:51.273984+0000 mon.a (mon.0) 2175 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:52.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:52 smithi183 bash[20824]: cluster 2024-02-13T03:31:51.274140+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-02-13T03:31:52.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:52 smithi183 bash[20824]: audit 2024-02-13T03:31:51.314885+0000 mon.a (mon.0) 2177 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:52.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:52 smithi183 bash[20824]: audit 2024-02-13T03:31:51.315923+0000 mon.a (mon.0) 2178 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-51"}]: dispatch 2024-02-13T03:31:52.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:52 smithi183 bash[20824]: audit 2024-02-13T03:31:51.317053+0000 mon.b (mon.1) 309 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:52.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:52 smithi183 bash[20824]: audit 2024-02-13T03:31:51.318126+0000 mon.b (mon.1) 310 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-51"}]: dispatch 2024-02-13T03:31:53.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:53 smithi161 bash[27879]: cluster 2024-02-13T03:31:51.347114+0000 mgr.y (mgr.24473) 432 : cluster [DBG] pgmap v608: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-13T03:31:53.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:53 smithi161 bash[27879]: cluster 2024-02-13T03:31:52.284770+0000 mon.a (mon.0) 2179 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-02-13T03:31:53.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:53 smithi161 bash[19769]: cluster 2024-02-13T03:31:51.347114+0000 mgr.y (mgr.24473) 432 : cluster [DBG] pgmap v608: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-13T03:31:53.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:53 smithi161 bash[19769]: cluster 2024-02-13T03:31:52.284770+0000 mon.a (mon.0) 2179 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-02-13T03:31:53.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:53 smithi183 bash[20824]: cluster 2024-02-13T03:31:51.347114+0000 mgr.y (mgr.24473) 432 : cluster [DBG] pgmap v608: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-13T03:31:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:53 smithi183 bash[20824]: cluster 2024-02-13T03:31:52.284770+0000 mon.a (mon.0) 2179 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-02-13T03:31:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:54 smithi161 bash[27879]: cluster 2024-02-13T03:31:53.296962+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-02-13T03:31:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:54 smithi161 bash[27879]: audit 2024-02-13T03:31:53.302610+0000 mon.a (mon.0) 2181 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:54 smithi161 bash[27879]: audit 2024-02-13T03:31:53.303335+0000 mon.b (mon.1) 311 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:54.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:54 smithi161 bash[19769]: cluster 2024-02-13T03:31:53.296962+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-02-13T03:31:54.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:54 smithi161 bash[19769]: audit 2024-02-13T03:31:53.302610+0000 mon.a (mon.0) 2181 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:54.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:54 smithi161 bash[19769]: audit 2024-02-13T03:31:53.303335+0000 mon.b (mon.1) 311 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:54.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:54 smithi183 bash[20824]: cluster 2024-02-13T03:31:53.296962+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-02-13T03:31:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:54 smithi183 bash[20824]: audit 2024-02-13T03:31:53.302610+0000 mon.a (mon.0) 2181 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:54 smithi183 bash[20824]: audit 2024-02-13T03:31:53.303335+0000 mon.b (mon.1) 311 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:55.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[27879]: cluster 2024-02-13T03:31:53.347915+0000 mgr.y (mgr.24473) 433 : cluster [DBG] pgmap v611: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:55.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[27879]: audit 2024-02-13T03:31:54.306590+0000 mon.a (mon.0) 2182 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:55.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[27879]: cluster 2024-02-13T03:31:54.306773+0000 mon.a (mon.0) 2183 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-02-13T03:31:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[27879]: audit 2024-02-13T03:31:54.313090+0000 mon.b (mon.1) 312 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:31:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[27879]: audit 2024-02-13T03:31:54.361035+0000 mon.a (mon.0) 2184 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[27879]: audit 2024-02-13T03:31:54.362212+0000 mon.a (mon.0) 2185 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-53"}]: dispatch 2024-02-13T03:31:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[27879]: audit 2024-02-13T03:31:54.363046+0000 mon.b (mon.1) 313 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[27879]: audit 2024-02-13T03:31:54.364385+0000 mon.b (mon.1) 314 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-53"}]: dispatch 2024-02-13T03:31:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[19769]: cluster 2024-02-13T03:31:53.347915+0000 mgr.y (mgr.24473) 433 : cluster [DBG] pgmap v611: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[19769]: audit 2024-02-13T03:31:54.306590+0000 mon.a (mon.0) 2182 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[19769]: cluster 2024-02-13T03:31:54.306773+0000 mon.a (mon.0) 2183 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-02-13T03:31:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[19769]: audit 2024-02-13T03:31:54.313090+0000 mon.b (mon.1) 312 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:31:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[19769]: audit 2024-02-13T03:31:54.361035+0000 mon.a (mon.0) 2184 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[19769]: audit 2024-02-13T03:31:54.362212+0000 mon.a (mon.0) 2185 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-53"}]: dispatch 2024-02-13T03:31:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[19769]: audit 2024-02-13T03:31:54.363046+0000 mon.b (mon.1) 313 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:55.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:55 smithi161 bash[19769]: audit 2024-02-13T03:31:54.364385+0000 mon.b (mon.1) 314 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-53"}]: dispatch 2024-02-13T03:31:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:55 smithi183 bash[20824]: cluster 2024-02-13T03:31:53.347915+0000 mgr.y (mgr.24473) 433 : cluster [DBG] pgmap v611: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:31:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:55 smithi183 bash[20824]: audit 2024-02-13T03:31:54.306590+0000 mon.a (mon.0) 2182 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:55 smithi183 bash[20824]: cluster 2024-02-13T03:31:54.306773+0000 mon.a (mon.0) 2183 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-02-13T03:31:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:55 smithi183 bash[20824]: audit 2024-02-13T03:31:54.313090+0000 mon.b (mon.1) 312 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:31:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:55 smithi183 bash[20824]: audit 2024-02-13T03:31:54.361035+0000 mon.a (mon.0) 2184 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:55 smithi183 bash[20824]: audit 2024-02-13T03:31:54.362212+0000 mon.a (mon.0) 2185 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-53"}]: dispatch 2024-02-13T03:31:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:55 smithi183 bash[20824]: audit 2024-02-13T03:31:54.363046+0000 mon.b (mon.1) 313 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:55 smithi183 bash[20824]: audit 2024-02-13T03:31:54.364385+0000 mon.b (mon.1) 314 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-53"}]: dispatch 2024-02-13T03:31:56.331 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:31:56 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:31:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:56 smithi161 bash[27879]: cluster 2024-02-13T03:31:55.307431+0000 mon.a (mon.0) 2186 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-02-13T03:31:56.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:56 smithi161 bash[19769]: cluster 2024-02-13T03:31:55.307431+0000 mon.a (mon.0) 2186 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-02-13T03:31:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:56 smithi183 bash[20824]: cluster 2024-02-13T03:31:55.307431+0000 mon.a (mon.0) 2186 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-02-13T03:31:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:57 smithi161 bash[27879]: cluster 2024-02-13T03:31:55.348442+0000 mgr.y (mgr.24473) 434 : cluster [DBG] pgmap v614: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:57 smithi161 bash[27879]: audit 2024-02-13T03:31:56.061370+0000 mgr.y (mgr.24473) 435 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:57 smithi161 bash[27879]: cluster 2024-02-13T03:31:56.335756+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-02-13T03:31:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:57 smithi161 bash[27879]: audit 2024-02-13T03:31:56.339148+0000 mon.a (mon.0) 2188 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:57 smithi161 bash[27879]: audit 2024-02-13T03:31:56.339770+0000 mon.b (mon.1) 315 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:57 smithi161 bash[19769]: cluster 2024-02-13T03:31:55.348442+0000 mgr.y (mgr.24473) 434 : cluster [DBG] pgmap v614: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:57 smithi161 bash[19769]: audit 2024-02-13T03:31:56.061370+0000 mgr.y (mgr.24473) 435 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:57 smithi161 bash[19769]: cluster 2024-02-13T03:31:56.335756+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-02-13T03:31:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:57 smithi161 bash[19769]: audit 2024-02-13T03:31:56.339148+0000 mon.a (mon.0) 2188 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:57 smithi161 bash[19769]: audit 2024-02-13T03:31:56.339770+0000 mon.b (mon.1) 315 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:57 smithi183 bash[20824]: cluster 2024-02-13T03:31:55.348442+0000 mgr.y (mgr.24473) 434 : cluster [DBG] pgmap v614: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:31:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:57 smithi183 bash[20824]: audit 2024-02-13T03:31:56.061370+0000 mgr.y (mgr.24473) 435 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:31:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:57 smithi183 bash[20824]: cluster 2024-02-13T03:31:56.335756+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-02-13T03:31:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:57 smithi183 bash[20824]: audit 2024-02-13T03:31:56.339148+0000 mon.a (mon.0) 2188 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:57 smithi183 bash[20824]: audit 2024-02-13T03:31:56.339770+0000 mon.b (mon.1) 315 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:31:59.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[27879]: audit 2024-02-13T03:31:57.343463+0000 mon.a (mon.0) 2189 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:59.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[27879]: cluster 2024-02-13T03:31:57.344357+0000 mon.a (mon.0) 2190 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-02-13T03:31:59.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[27879]: audit 2024-02-13T03:31:57.392418+0000 mon.a (mon.0) 2191 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:59.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[27879]: audit 2024-02-13T03:31:57.393526+0000 mon.a (mon.0) 2192 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-55"}]: dispatch 2024-02-13T03:31:59.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[27879]: audit 2024-02-13T03:31:57.394452+0000 mon.b (mon.1) 316 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:59.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[27879]: audit 2024-02-13T03:31:57.395703+0000 mon.b (mon.1) 317 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-55"}]: dispatch 2024-02-13T03:31:59.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[27879]: cluster 2024-02-13T03:31:58.338762+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-02-13T03:31:59.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[19769]: audit 2024-02-13T03:31:57.343463+0000 mon.a (mon.0) 2189 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:59.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[19769]: cluster 2024-02-13T03:31:57.344357+0000 mon.a (mon.0) 2190 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-02-13T03:31:59.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[19769]: audit 2024-02-13T03:31:57.392418+0000 mon.a (mon.0) 2191 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:59.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[19769]: audit 2024-02-13T03:31:57.393526+0000 mon.a (mon.0) 2192 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-55"}]: dispatch 2024-02-13T03:31:59.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[19769]: audit 2024-02-13T03:31:57.394452+0000 mon.b (mon.1) 316 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:59.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[19769]: audit 2024-02-13T03:31:57.395703+0000 mon.b (mon.1) 317 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-55"}]: dispatch 2024-02-13T03:31:59.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:58 smithi161 bash[19769]: cluster 2024-02-13T03:31:58.338762+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-02-13T03:31:59.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:58 smithi183 bash[20824]: audit 2024-02-13T03:31:57.343463+0000 mon.a (mon.0) 2189 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:31:59.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:58 smithi183 bash[20824]: cluster 2024-02-13T03:31:57.344357+0000 mon.a (mon.0) 2190 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-02-13T03:31:59.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:58 smithi183 bash[20824]: audit 2024-02-13T03:31:57.392418+0000 mon.a (mon.0) 2191 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:59.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:58 smithi183 bash[20824]: audit 2024-02-13T03:31:57.393526+0000 mon.a (mon.0) 2192 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-55"}]: dispatch 2024-02-13T03:31:59.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:58 smithi183 bash[20824]: audit 2024-02-13T03:31:57.394452+0000 mon.b (mon.1) 316 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:31:59.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:58 smithi183 bash[20824]: audit 2024-02-13T03:31:57.395703+0000 mon.b (mon.1) 317 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-55"}]: dispatch 2024-02-13T03:31:59.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:58 smithi183 bash[20824]: cluster 2024-02-13T03:31:58.338762+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-02-13T03:31:59.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:31:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:31:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:32:00.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:59 smithi161 bash[27879]: cluster 2024-02-13T03:31:57.349339+0000 mgr.y (mgr.24473) 436 : cluster [DBG] pgmap v617: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:32:00.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:59 smithi161 bash[27879]: cluster 2024-02-13T03:31:59.342666+0000 mon.a (mon.0) 2194 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-02-13T03:32:00.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:59 smithi161 bash[27879]: audit 2024-02-13T03:31:59.347164+0000 mon.a (mon.0) 2195 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:00.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:31:59 smithi161 bash[27879]: audit 2024-02-13T03:31:59.348964+0000 mon.b (mon.1) 318 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:59 smithi161 bash[19769]: cluster 2024-02-13T03:31:57.349339+0000 mgr.y (mgr.24473) 436 : cluster [DBG] pgmap v617: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:32:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:59 smithi161 bash[19769]: cluster 2024-02-13T03:31:59.342666+0000 mon.a (mon.0) 2194 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-02-13T03:32:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:59 smithi161 bash[19769]: audit 2024-02-13T03:31:59.347164+0000 mon.a (mon.0) 2195 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:31:59 smithi161 bash[19769]: audit 2024-02-13T03:31:59.348964+0000 mon.b (mon.1) 318 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:00.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:59 smithi183 bash[20824]: cluster 2024-02-13T03:31:57.349339+0000 mgr.y (mgr.24473) 436 : cluster [DBG] pgmap v617: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:32:00.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:59 smithi183 bash[20824]: cluster 2024-02-13T03:31:59.342666+0000 mon.a (mon.0) 2194 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-02-13T03:32:00.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:59 smithi183 bash[20824]: audit 2024-02-13T03:31:59.347164+0000 mon.a (mon.0) 2195 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:00.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:31:59 smithi183 bash[20824]: audit 2024-02-13T03:31:59.348964+0000 mon.b (mon.1) 318 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[19769]: cluster 2024-02-13T03:31:59.350095+0000 mgr.y (mgr.24473) 437 : cluster [DBG] pgmap v620: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:32:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[19769]: audit 2024-02-13T03:32:00.351184+0000 mon.a (mon.0) 2196 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:32:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[19769]: cluster 2024-02-13T03:32:00.351731+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-02-13T03:32:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[19769]: audit 2024-02-13T03:32:00.358084+0000 mon.b (mon.1) 319 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:32:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[19769]: audit 2024-02-13T03:32:00.402591+0000 mon.a (mon.0) 2198 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[19769]: audit 2024-02-13T03:32:00.403604+0000 mon.a (mon.0) 2199 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-57"}]: dispatch 2024-02-13T03:32:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[19769]: audit 2024-02-13T03:32:00.404608+0000 mon.b (mon.1) 320 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[19769]: audit 2024-02-13T03:32:00.405813+0000 mon.b (mon.1) 321 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-57"}]: dispatch 2024-02-13T03:32:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[27879]: cluster 2024-02-13T03:31:59.350095+0000 mgr.y (mgr.24473) 437 : cluster [DBG] pgmap v620: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:32:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[27879]: audit 2024-02-13T03:32:00.351184+0000 mon.a (mon.0) 2196 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:32:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[27879]: cluster 2024-02-13T03:32:00.351731+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-02-13T03:32:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[27879]: audit 2024-02-13T03:32:00.358084+0000 mon.b (mon.1) 319 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:32:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[27879]: audit 2024-02-13T03:32:00.402591+0000 mon.a (mon.0) 2198 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[27879]: audit 2024-02-13T03:32:00.403604+0000 mon.a (mon.0) 2199 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-57"}]: dispatch 2024-02-13T03:32:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[27879]: audit 2024-02-13T03:32:00.404608+0000 mon.b (mon.1) 320 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[27879]: audit 2024-02-13T03:32:00.405813+0000 mon.b (mon.1) 321 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-57"}]: dispatch 2024-02-13T03:32:01.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:32:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:32:01] "GET /metrics HTTP/1.1" 200 184620 "" "Prometheus/2.33.4" 2024-02-13T03:32:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:01 smithi183 bash[20824]: cluster 2024-02-13T03:31:59.350095+0000 mgr.y (mgr.24473) 437 : cluster [DBG] pgmap v620: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:32:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:01 smithi183 bash[20824]: audit 2024-02-13T03:32:00.351184+0000 mon.a (mon.0) 2196 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:32:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:01 smithi183 bash[20824]: cluster 2024-02-13T03:32:00.351731+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-02-13T03:32:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:01 smithi183 bash[20824]: audit 2024-02-13T03:32:00.358084+0000 mon.b (mon.1) 319 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:32:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:01 smithi183 bash[20824]: audit 2024-02-13T03:32:00.402591+0000 mon.a (mon.0) 2198 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:01 smithi183 bash[20824]: audit 2024-02-13T03:32:00.403604+0000 mon.a (mon.0) 2199 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-57"}]: dispatch 2024-02-13T03:32:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:01 smithi183 bash[20824]: audit 2024-02-13T03:32:00.404608+0000 mon.b (mon.1) 320 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:01 smithi183 bash[20824]: audit 2024-02-13T03:32:00.405813+0000 mon.b (mon.1) 321 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-57"}]: dispatch 2024-02-13T03:32:02.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:02 smithi161 bash[27879]: cluster 2024-02-13T03:32:01.351741+0000 mgr.y (mgr.24473) 438 : cluster [DBG] pgmap v622: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-02-13T03:32:02.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:02 smithi161 bash[27879]: cluster 2024-02-13T03:32:01.354401+0000 mon.a (mon.0) 2200 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-02-13T03:32:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:02 smithi161 bash[19769]: cluster 2024-02-13T03:32:01.351741+0000 mgr.y (mgr.24473) 438 : cluster [DBG] pgmap v622: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-02-13T03:32:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:02 smithi161 bash[19769]: cluster 2024-02-13T03:32:01.354401+0000 mon.a (mon.0) 2200 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-02-13T03:32:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:02 smithi183 bash[20824]: cluster 2024-02-13T03:32:01.351741+0000 mgr.y (mgr.24473) 438 : cluster [DBG] pgmap v622: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-02-13T03:32:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:02 smithi183 bash[20824]: cluster 2024-02-13T03:32:01.354401+0000 mon.a (mon.0) 2200 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-02-13T03:32:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:03 smithi161 bash[27879]: cluster 2024-02-13T03:32:02.370151+0000 mon.a (mon.0) 2201 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-02-13T03:32:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:03 smithi161 bash[27879]: audit 2024-02-13T03:32:02.373494+0000 mon.a (mon.0) 2202 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:03 smithi161 bash[27879]: audit 2024-02-13T03:32:02.375450+0000 mon.b (mon.1) 322 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:03 smithi161 bash[19769]: cluster 2024-02-13T03:32:02.370151+0000 mon.a (mon.0) 2201 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-02-13T03:32:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:03 smithi161 bash[19769]: audit 2024-02-13T03:32:02.373494+0000 mon.a (mon.0) 2202 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:03 smithi161 bash[19769]: audit 2024-02-13T03:32:02.375450+0000 mon.b (mon.1) 322 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:03 smithi183 bash[20824]: cluster 2024-02-13T03:32:02.370151+0000 mon.a (mon.0) 2201 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-02-13T03:32:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:03 smithi183 bash[20824]: audit 2024-02-13T03:32:02.373494+0000 mon.a (mon.0) 2202 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:03 smithi183 bash[20824]: audit 2024-02-13T03:32:02.375450+0000 mon.b (mon.1) 322 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:04.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:04 smithi161 bash[19769]: cluster 2024-02-13T03:32:03.352594+0000 mgr.y (mgr.24473) 439 : cluster [DBG] pgmap v625: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:32:04.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:04 smithi161 bash[19769]: audit 2024-02-13T03:32:03.370431+0000 mon.a (mon.0) 2203 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:32:04.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:04 smithi161 bash[19769]: cluster 2024-02-13T03:32:03.371083+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-02-13T03:32:04.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:04 smithi161 bash[19769]: audit 2024-02-13T03:32:03.389502+0000 mon.b (mon.1) 323 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:32:04.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:04 smithi161 bash[19769]: audit 2024-02-13T03:32:03.393808+0000 mon.a (mon.0) 2205 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:04.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:04 smithi161 bash[19769]: audit 2024-02-13T03:32:03.394168+0000 mon.b (mon.1) 324 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:04 smithi161 bash[27879]: cluster 2024-02-13T03:32:03.352594+0000 mgr.y (mgr.24473) 439 : cluster [DBG] pgmap v625: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:32:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:04 smithi161 bash[27879]: audit 2024-02-13T03:32:03.370431+0000 mon.a (mon.0) 2203 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:32:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:04 smithi161 bash[27879]: cluster 2024-02-13T03:32:03.371083+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-02-13T03:32:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:04 smithi161 bash[27879]: audit 2024-02-13T03:32:03.389502+0000 mon.b (mon.1) 323 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:32:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:04 smithi161 bash[27879]: audit 2024-02-13T03:32:03.393808+0000 mon.a (mon.0) 2205 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:04 smithi161 bash[27879]: audit 2024-02-13T03:32:03.394168+0000 mon.b (mon.1) 324 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:04 smithi183 bash[20824]: cluster 2024-02-13T03:32:03.352594+0000 mgr.y (mgr.24473) 439 : cluster [DBG] pgmap v625: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:32:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:04 smithi183 bash[20824]: audit 2024-02-13T03:32:03.370431+0000 mon.a (mon.0) 2203 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:32:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:04 smithi183 bash[20824]: cluster 2024-02-13T03:32:03.371083+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-02-13T03:32:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:04 smithi183 bash[20824]: audit 2024-02-13T03:32:03.389502+0000 mon.b (mon.1) 323 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:32:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:04 smithi183 bash[20824]: audit 2024-02-13T03:32:03.393808+0000 mon.a (mon.0) 2205 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:04 smithi183 bash[20824]: audit 2024-02-13T03:32:03.394168+0000 mon.b (mon.1) 324 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[27879]: audit 2024-02-13T03:32:04.384691+0000 mon.a (mon.0) 2206 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:32:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[27879]: cluster 2024-02-13T03:32:04.384913+0000 mon.a (mon.0) 2207 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-02-13T03:32:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[27879]: audit 2024-02-13T03:32:04.435961+0000 mon.a (mon.0) 2208 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[27879]: audit 2024-02-13T03:32:04.437411+0000 mon.a (mon.0) 2209 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-59"}]: dispatch 2024-02-13T03:32:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[27879]: audit 2024-02-13T03:32:04.437759+0000 mon.b (mon.1) 325 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[27879]: audit 2024-02-13T03:32:04.439534+0000 mon.b (mon.1) 326 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-59"}]: dispatch 2024-02-13T03:32:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[27879]: audit 2024-02-13T03:32:04.785404+0000 mon.a (mon.0) 2210 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:32:06.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[19769]: audit 2024-02-13T03:32:04.384691+0000 mon.a (mon.0) 2206 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:32:06.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[19769]: cluster 2024-02-13T03:32:04.384913+0000 mon.a (mon.0) 2207 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-02-13T03:32:06.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[19769]: audit 2024-02-13T03:32:04.435961+0000 mon.a (mon.0) 2208 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:06.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[19769]: audit 2024-02-13T03:32:04.437411+0000 mon.a (mon.0) 2209 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-59"}]: dispatch 2024-02-13T03:32:06.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[19769]: audit 2024-02-13T03:32:04.437759+0000 mon.b (mon.1) 325 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:06.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[19769]: audit 2024-02-13T03:32:04.439534+0000 mon.b (mon.1) 326 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-59"}]: dispatch 2024-02-13T03:32:06.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:05 smithi161 bash[19769]: audit 2024-02-13T03:32:04.785404+0000 mon.a (mon.0) 2210 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:32:06.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:32:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:05 smithi183 bash[20824]: audit 2024-02-13T03:32:04.384691+0000 mon.a (mon.0) 2206 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:32:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:05 smithi183 bash[20824]: cluster 2024-02-13T03:32:04.384913+0000 mon.a (mon.0) 2207 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-02-13T03:32:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:05 smithi183 bash[20824]: audit 2024-02-13T03:32:04.435961+0000 mon.a (mon.0) 2208 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:05 smithi183 bash[20824]: audit 2024-02-13T03:32:04.437411+0000 mon.a (mon.0) 2209 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-59"}]: dispatch 2024-02-13T03:32:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:05 smithi183 bash[20824]: audit 2024-02-13T03:32:04.437759+0000 mon.b (mon.1) 325 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:05 smithi183 bash[20824]: audit 2024-02-13T03:32:04.439534+0000 mon.b (mon.1) 326 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-59"}]: dispatch 2024-02-13T03:32:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:05 smithi183 bash[20824]: audit 2024-02-13T03:32:04.785404+0000 mon.a (mon.0) 2210 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:32:07.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: cluster 2024-02-13T03:32:05.353464+0000 mgr.y (mgr.24473) 440 : cluster [DBG] pgmap v628: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:32:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: cluster 2024-02-13T03:32:05.401841+0000 mon.a (mon.0) 2211 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-02-13T03:32:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.064233+0000 mgr.y (mgr.24473) 441 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.238254+0000 mon.c (mon.2) 511 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:32:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: cluster 2024-02-13T03:32:06.403525+0000 mon.a (mon.0) 2212 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-02-13T03:32:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.407813+0000 mon.b (mon.1) 327 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.409172+0000 mon.a (mon.0) 2213 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.538532+0000 mon.c (mon.2) 512 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.539214+0000 mon.a (mon.0) 2214 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.541135+0000 mon.c (mon.2) 513 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.541811+0000 mon.a (mon.0) 2215 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.543561+0000 mon.c (mon.2) 514 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.544234+0000 mon.a (mon.0) 2216 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.545764+0000 mon.c (mon.2) 515 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.546289+0000 mon.a (mon.0) 2217 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.556100+0000 mon.a (mon.0) 2218 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:32:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.568499+0000 mon.c (mon.2) 516 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: cluster 2024-02-13T03:32:05.353464+0000 mgr.y (mgr.24473) 440 : cluster [DBG] pgmap v628: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:32:07.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: cluster 2024-02-13T03:32:05.401841+0000 mon.a (mon.0) 2211 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-02-13T03:32:07.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.064233+0000 mgr.y (mgr.24473) 441 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:07.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.238254+0000 mon.c (mon.2) 511 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:32:07.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: cluster 2024-02-13T03:32:06.403525+0000 mon.a (mon.0) 2212 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-02-13T03:32:07.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.407813+0000 mon.b (mon.1) 327 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:07.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.409172+0000 mon.a (mon.0) 2213 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:07.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.538532+0000 mon.c (mon.2) 512 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.539214+0000 mon.a (mon.0) 2214 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.541135+0000 mon.c (mon.2) 513 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.541811+0000 mon.a (mon.0) 2215 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.543561+0000 mon.c (mon.2) 514 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.544234+0000 mon.a (mon.0) 2216 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.545764+0000 mon.c (mon.2) 515 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.546289+0000 mon.a (mon.0) 2217 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.556100+0000 mon.a (mon.0) 2218 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:32:07.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.568499+0000 mon.c (mon.2) 516 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.569039+0000 mon.a (mon.0) 2219 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.570505+0000 mon.c (mon.2) 517 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.570921+0000 mon.a (mon.0) 2220 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.572233+0000 mon.c (mon.2) 518 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.572642+0000 mon.a (mon.0) 2221 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.574009+0000 mon.c (mon.2) 519 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.574418+0000 mon.a (mon.0) 2222 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.585871+0000 mon.a (mon.0) 2223 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:32:07.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.589658+0000 mon.c (mon.2) 520 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:32:07.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.590771+0000 mon.c (mon.2) 521 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:32:07.180 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[19769]: audit 2024-02-13T03:32:06.600312+0000 mon.a (mon.0) 2224 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:32:07.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.569039+0000 mon.a (mon.0) 2219 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.570505+0000 mon.c (mon.2) 517 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.570921+0000 mon.a (mon.0) 2220 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.572233+0000 mon.c (mon.2) 518 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.572642+0000 mon.a (mon.0) 2221 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.574009+0000 mon.c (mon.2) 519 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.574418+0000 mon.a (mon.0) 2222 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.585871+0000 mon.a (mon.0) 2223 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:32:07.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.589658+0000 mon.c (mon.2) 520 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:32:07.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.590771+0000 mon.c (mon.2) 521 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:32:07.181 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:06 smithi161 bash[27879]: audit 2024-02-13T03:32:06.600312+0000 mon.a (mon.0) 2224 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:32:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: cluster 2024-02-13T03:32:05.353464+0000 mgr.y (mgr.24473) 440 : cluster [DBG] pgmap v628: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:32:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: cluster 2024-02-13T03:32:05.401841+0000 mon.a (mon.0) 2211 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-02-13T03:32:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.064233+0000 mgr.y (mgr.24473) 441 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.238254+0000 mon.c (mon.2) 511 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:32:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: cluster 2024-02-13T03:32:06.403525+0000 mon.a (mon.0) 2212 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-02-13T03:32:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.407813+0000 mon.b (mon.1) 327 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.409172+0000 mon.a (mon.0) 2213 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.538532+0000 mon.c (mon.2) 512 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.539214+0000 mon.a (mon.0) 2214 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.541135+0000 mon.c (mon.2) 513 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.541811+0000 mon.a (mon.0) 2215 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.543561+0000 mon.c (mon.2) 514 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.544234+0000 mon.a (mon.0) 2216 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.545764+0000 mon.c (mon.2) 515 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.546289+0000 mon.a (mon.0) 2217 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.556100+0000 mon.a (mon.0) 2218 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:32:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.568499+0000 mon.c (mon.2) 516 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.569039+0000 mon.a (mon.0) 2219 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.570505+0000 mon.c (mon.2) 517 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.570921+0000 mon.a (mon.0) 2220 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.572233+0000 mon.c (mon.2) 518 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.572642+0000 mon.a (mon.0) 2221 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.574009+0000 mon.c (mon.2) 519 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.574418+0000 mon.a (mon.0) 2222 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T03:32:07.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.585871+0000 mon.a (mon.0) 2223 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:32:07.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.589658+0000 mon.c (mon.2) 520 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:32:07.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.590771+0000 mon.c (mon.2) 521 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:32:07.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:06 smithi183 bash[20824]: audit 2024-02-13T03:32:06.600312+0000 mon.a (mon.0) 2224 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:32:08.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[27879]: cephadm 2024-02-13T03:32:06.547478+0000 mgr.y (mgr.24473) 442 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 3305M 2024-02-13T03:32:08.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[27879]: cephadm 2024-02-13T03:32:06.575462+0000 mgr.y (mgr.24473) 443 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 3305M 2024-02-13T03:32:08.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[27879]: audit 2024-02-13T03:32:07.409189+0000 mon.a (mon.0) 2225 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:32:08.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[27879]: cluster 2024-02-13T03:32:07.410045+0000 mon.a (mon.0) 2226 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-02-13T03:32:08.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[27879]: audit 2024-02-13T03:32:07.423156+0000 mon.b (mon.1) 328 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:32:08.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[27879]: audit 2024-02-13T03:32:07.426623+0000 mon.a (mon.0) 2227 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:08.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[27879]: audit 2024-02-13T03:32:07.427016+0000 mon.b (mon.1) 329 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:08.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[19769]: cephadm 2024-02-13T03:32:06.547478+0000 mgr.y (mgr.24473) 442 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 3305M 2024-02-13T03:32:08.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[19769]: cephadm 2024-02-13T03:32:06.575462+0000 mgr.y (mgr.24473) 443 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 3305M 2024-02-13T03:32:08.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[19769]: audit 2024-02-13T03:32:07.409189+0000 mon.a (mon.0) 2225 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:32:08.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[19769]: cluster 2024-02-13T03:32:07.410045+0000 mon.a (mon.0) 2226 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-02-13T03:32:08.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[19769]: audit 2024-02-13T03:32:07.423156+0000 mon.b (mon.1) 328 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:32:08.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[19769]: audit 2024-02-13T03:32:07.426623+0000 mon.a (mon.0) 2227 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:08.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:07 smithi161 bash[19769]: audit 2024-02-13T03:32:07.427016+0000 mon.b (mon.1) 329 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:08.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:07 smithi183 bash[20824]: cephadm 2024-02-13T03:32:06.547478+0000 mgr.y (mgr.24473) 442 : cephadm [INF] Adjusting osd_memory_target on smithi183 to 3305M 2024-02-13T03:32:08.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:07 smithi183 bash[20824]: cephadm 2024-02-13T03:32:06.575462+0000 mgr.y (mgr.24473) 443 : cephadm [INF] Adjusting osd_memory_target on smithi161 to 3305M 2024-02-13T03:32:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:07 smithi183 bash[20824]: audit 2024-02-13T03:32:07.409189+0000 mon.a (mon.0) 2225 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:32:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:07 smithi183 bash[20824]: cluster 2024-02-13T03:32:07.410045+0000 mon.a (mon.0) 2226 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-02-13T03:32:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:07 smithi183 bash[20824]: audit 2024-02-13T03:32:07.423156+0000 mon.b (mon.1) 328 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:32:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:07 smithi183 bash[20824]: audit 2024-02-13T03:32:07.426623+0000 mon.a (mon.0) 2227 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:08.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:07 smithi183 bash[20824]: audit 2024-02-13T03:32:07.427016+0000 mon.b (mon.1) 329 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:09.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:08 smithi161 bash[27879]: cluster 2024-02-13T03:32:07.354341+0000 mgr.y (mgr.24473) 444 : cluster [DBG] pgmap v631: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:32:09.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:08 smithi161 bash[27879]: audit 2024-02-13T03:32:08.410479+0000 mon.a (mon.0) 2228 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:32:09.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:08 smithi161 bash[27879]: cluster 2024-02-13T03:32:08.410754+0000 mon.a (mon.0) 2229 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-02-13T03:32:09.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:08 smithi161 bash[19769]: cluster 2024-02-13T03:32:07.354341+0000 mgr.y (mgr.24473) 444 : cluster [DBG] pgmap v631: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:32:09.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:08 smithi161 bash[19769]: audit 2024-02-13T03:32:08.410479+0000 mon.a (mon.0) 2228 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:32:09.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:08 smithi161 bash[19769]: cluster 2024-02-13T03:32:08.410754+0000 mon.a (mon.0) 2229 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-02-13T03:32:09.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:08 smithi183 bash[20824]: cluster 2024-02-13T03:32:07.354341+0000 mgr.y (mgr.24473) 444 : cluster [DBG] pgmap v631: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:32:09.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:08 smithi183 bash[20824]: audit 2024-02-13T03:32:08.410479+0000 mon.a (mon.0) 2228 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:32:09.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:08 smithi183 bash[20824]: cluster 2024-02-13T03:32:08.410754+0000 mon.a (mon.0) 2229 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-02-13T03:32:09.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:32:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:32:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:32:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:10 smithi161 bash[27879]: cluster 2024-02-13T03:32:09.355158+0000 mgr.y (mgr.24473) 445 : cluster [DBG] pgmap v634: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:32:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:10 smithi161 bash[27879]: cluster 2024-02-13T03:32:09.412206+0000 mon.a (mon.0) 2230 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-02-13T03:32:10.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:10 smithi161 bash[19769]: cluster 2024-02-13T03:32:09.355158+0000 mgr.y (mgr.24473) 445 : cluster [DBG] pgmap v634: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:32:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:10 smithi161 bash[19769]: cluster 2024-02-13T03:32:09.412206+0000 mon.a (mon.0) 2230 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-02-13T03:32:10.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:10 smithi183 bash[20824]: cluster 2024-02-13T03:32:09.355158+0000 mgr.y (mgr.24473) 445 : cluster [DBG] pgmap v634: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:32:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:10 smithi183 bash[20824]: cluster 2024-02-13T03:32:09.412206+0000 mon.a (mon.0) 2230 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-02-13T03:32:11.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:32:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:32:11] "GET /metrics HTTP/1.1" 200 187168 "" "Prometheus/2.33.4" 2024-02-13T03:32:11.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:11 smithi183 bash[20824]: cluster 2024-02-13T03:32:10.416991+0000 mon.a (mon.0) 2231 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-02-13T03:32:11.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:11 smithi161 bash[27879]: cluster 2024-02-13T03:32:10.416991+0000 mon.a (mon.0) 2231 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-02-13T03:32:11.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:11 smithi161 bash[19769]: cluster 2024-02-13T03:32:10.416991+0000 mon.a (mon.0) 2231 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-02-13T03:32:12.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:12 smithi183 bash[20824]: cluster 2024-02-13T03:32:11.357258+0000 mgr.y (mgr.24473) 446 : cluster [DBG] pgmap v637: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-13T03:32:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:12 smithi183 bash[20824]: cluster 2024-02-13T03:32:11.437778+0000 mon.a (mon.0) 2232 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-02-13T03:32:12.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:12 smithi161 bash[27879]: cluster 2024-02-13T03:32:11.357258+0000 mgr.y (mgr.24473) 446 : cluster [DBG] pgmap v637: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-13T03:32:12.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:12 smithi161 bash[27879]: cluster 2024-02-13T03:32:11.437778+0000 mon.a (mon.0) 2232 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-02-13T03:32:12.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:12 smithi161 bash[19769]: cluster 2024-02-13T03:32:11.357258+0000 mgr.y (mgr.24473) 446 : cluster [DBG] pgmap v637: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-13T03:32:12.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:12 smithi161 bash[19769]: cluster 2024-02-13T03:32:11.437778+0000 mon.a (mon.0) 2232 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-02-13T03:32:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:13 smithi183 bash[20824]: cluster 2024-02-13T03:32:12.455512+0000 mon.a (mon.0) 2233 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-02-13T03:32:13.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:13 smithi161 bash[27879]: cluster 2024-02-13T03:32:12.455512+0000 mon.a (mon.0) 2233 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-02-13T03:32:13.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:13 smithi161 bash[19769]: cluster 2024-02-13T03:32:12.455512+0000 mon.a (mon.0) 2233 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-02-13T03:32:15.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:14 smithi161 bash[27879]: cluster 2024-02-13T03:32:13.358111+0000 mgr.y (mgr.24473) 447 : cluster [DBG] pgmap v640: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-13T03:32:15.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:14 smithi161 bash[19769]: cluster 2024-02-13T03:32:13.358111+0000 mgr.y (mgr.24473) 447 : cluster [DBG] pgmap v640: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-13T03:32:15.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:14 smithi183 bash[20824]: cluster 2024-02-13T03:32:13.358111+0000 mgr.y (mgr.24473) 447 : cluster [DBG] pgmap v640: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-13T03:32:16.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:32:16 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:32:17.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:16 smithi161 bash[27879]: cluster 2024-02-13T03:32:15.358964+0000 mgr.y (mgr.24473) 448 : cluster [DBG] pgmap v641: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-13T03:32:17.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:16 smithi161 bash[27879]: audit 2024-02-13T03:32:16.068319+0000 mgr.y (mgr.24473) 449 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:17.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:16 smithi161 bash[19769]: cluster 2024-02-13T03:32:15.358964+0000 mgr.y (mgr.24473) 448 : cluster [DBG] pgmap v641: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-13T03:32:17.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:16 smithi161 bash[19769]: audit 2024-02-13T03:32:16.068319+0000 mgr.y (mgr.24473) 449 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:17.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:16 smithi183 bash[20824]: cluster 2024-02-13T03:32:15.358964+0000 mgr.y (mgr.24473) 448 : cluster [DBG] pgmap v641: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-13T03:32:17.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:16 smithi183 bash[20824]: audit 2024-02-13T03:32:16.068319+0000 mgr.y (mgr.24473) 449 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:19.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:18 smithi161 bash[27879]: cluster 2024-02-13T03:32:17.361242+0000 mgr.y (mgr.24473) 450 : cluster [DBG] pgmap v642: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-02-13T03:32:19.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:18 smithi161 bash[19769]: cluster 2024-02-13T03:32:17.361242+0000 mgr.y (mgr.24473) 450 : cluster [DBG] pgmap v642: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-02-13T03:32:19.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:18 smithi183 bash[20824]: cluster 2024-02-13T03:32:17.361242+0000 mgr.y (mgr.24473) 450 : cluster [DBG] pgmap v642: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-02-13T03:32:19.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:32:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:32:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:32:20.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:19 smithi161 bash[27879]: cluster 2024-02-13T03:32:18.793296+0000 mon.a (mon.0) 2234 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-02-13T03:32:20.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:19 smithi161 bash[19769]: cluster 2024-02-13T03:32:18.793296+0000 mon.a (mon.0) 2234 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-02-13T03:32:20.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:19 smithi183 bash[20824]: cluster 2024-02-13T03:32:18.793296+0000 mon.a (mon.0) 2234 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-02-13T03:32:21.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:20 smithi161 bash[27879]: cluster 2024-02-13T03:32:19.362138+0000 mgr.y (mgr.24473) 451 : cluster [DBG] pgmap v644: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 258 B/s wr, 1 op/s 2024-02-13T03:32:21.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:20 smithi161 bash[19769]: cluster 2024-02-13T03:32:19.362138+0000 mgr.y (mgr.24473) 451 : cluster [DBG] pgmap v644: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 258 B/s wr, 1 op/s 2024-02-13T03:32:21.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:20 smithi183 bash[20824]: cluster 2024-02-13T03:32:19.362138+0000 mgr.y (mgr.24473) 451 : cluster [DBG] pgmap v644: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 258 B/s wr, 1 op/s 2024-02-13T03:32:21.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:32:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:32:21] "GET /metrics HTTP/1.1" 200 187290 "" "Prometheus/2.33.4" 2024-02-13T03:32:23.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:22 smithi161 bash[27879]: cluster 2024-02-13T03:32:21.364334+0000 mgr.y (mgr.24473) 452 : cluster [DBG] pgmap v645: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 229 B/s wr, 2 op/s 2024-02-13T03:32:23.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:22 smithi161 bash[19769]: cluster 2024-02-13T03:32:21.364334+0000 mgr.y (mgr.24473) 452 : cluster [DBG] pgmap v645: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 229 B/s wr, 2 op/s 2024-02-13T03:32:23.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:22 smithi183 bash[20824]: cluster 2024-02-13T03:32:21.364334+0000 mgr.y (mgr.24473) 452 : cluster [DBG] pgmap v645: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 229 B/s wr, 2 op/s 2024-02-13T03:32:24.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:23 smithi161 bash[27879]: cluster 2024-02-13T03:32:22.830465+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-02-13T03:32:24.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:23 smithi161 bash[19769]: cluster 2024-02-13T03:32:22.830465+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-02-13T03:32:24.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:23 smithi183 bash[20824]: cluster 2024-02-13T03:32:22.830465+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-02-13T03:32:25.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:24 smithi161 bash[27879]: cluster 2024-02-13T03:32:23.365182+0000 mgr.y (mgr.24473) 453 : cluster [DBG] pgmap v647: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:32:25.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:24 smithi161 bash[19769]: cluster 2024-02-13T03:32:23.365182+0000 mgr.y (mgr.24473) 453 : cluster [DBG] pgmap v647: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:32:25.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:24 smithi183 bash[20824]: cluster 2024-02-13T03:32:23.365182+0000 mgr.y (mgr.24473) 453 : cluster [DBG] pgmap v647: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:32:26.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:32:26 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:32:27.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:26 smithi161 bash[27879]: cluster 2024-02-13T03:32:25.366065+0000 mgr.y (mgr.24473) 454 : cluster [DBG] pgmap v648: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-13T03:32:27.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:26 smithi161 bash[27879]: audit 2024-02-13T03:32:26.079488+0000 mgr.y (mgr.24473) 455 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:27.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:26 smithi161 bash[19769]: cluster 2024-02-13T03:32:25.366065+0000 mgr.y (mgr.24473) 454 : cluster [DBG] pgmap v648: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-13T03:32:27.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:26 smithi161 bash[19769]: audit 2024-02-13T03:32:26.079488+0000 mgr.y (mgr.24473) 455 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:27.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:26 smithi183 bash[20824]: cluster 2024-02-13T03:32:25.366065+0000 mgr.y (mgr.24473) 454 : cluster [DBG] pgmap v648: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-13T03:32:27.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:26 smithi183 bash[20824]: audit 2024-02-13T03:32:26.079488+0000 mgr.y (mgr.24473) 455 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:29.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:28 smithi161 bash[27879]: cluster 2024-02-13T03:32:27.368259+0000 mgr.y (mgr.24473) 456 : cluster [DBG] pgmap v649: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:32:29.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:28 smithi161 bash[27879]: cluster 2024-02-13T03:32:28.799098+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-02-13T03:32:29.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:28 smithi161 bash[19769]: cluster 2024-02-13T03:32:27.368259+0000 mgr.y (mgr.24473) 456 : cluster [DBG] pgmap v649: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:32:29.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:28 smithi161 bash[19769]: cluster 2024-02-13T03:32:28.799098+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-02-13T03:32:29.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:28 smithi183 bash[20824]: cluster 2024-02-13T03:32:27.368259+0000 mgr.y (mgr.24473) 456 : cluster [DBG] pgmap v649: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:32:29.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:28 smithi183 bash[20824]: cluster 2024-02-13T03:32:28.799098+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-02-13T03:32:29.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:32:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:32:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:32:31.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:30 smithi161 bash[27879]: cluster 2024-02-13T03:32:29.369105+0000 mgr.y (mgr.24473) 457 : cluster [DBG] pgmap v651: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:32:31.176 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:32:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:32:31] "GET /metrics HTTP/1.1" 200 187290 "" "Prometheus/2.33.4" 2024-02-13T03:32:31.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:30 smithi161 bash[19769]: cluster 2024-02-13T03:32:29.369105+0000 mgr.y (mgr.24473) 457 : cluster [DBG] pgmap v651: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:32:31.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:30 smithi183 bash[20824]: cluster 2024-02-13T03:32:29.369105+0000 mgr.y (mgr.24473) 457 : cluster [DBG] pgmap v651: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:32:33.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:32 smithi161 bash[27879]: cluster 2024-02-13T03:32:31.371319+0000 mgr.y (mgr.24473) 458 : cluster [DBG] pgmap v652: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:32:33.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:32 smithi161 bash[19769]: cluster 2024-02-13T03:32:31.371319+0000 mgr.y (mgr.24473) 458 : cluster [DBG] pgmap v652: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:32:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:32 smithi183 bash[20824]: cluster 2024-02-13T03:32:31.371319+0000 mgr.y (mgr.24473) 458 : cluster [DBG] pgmap v652: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:32:34.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:33 smithi161 bash[27879]: cluster 2024-02-13T03:32:32.883544+0000 mon.a (mon.0) 2237 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-02-13T03:32:34.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:33 smithi161 bash[19769]: cluster 2024-02-13T03:32:32.883544+0000 mon.a (mon.0) 2237 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-02-13T03:32:34.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:33 smithi183 bash[20824]: cluster 2024-02-13T03:32:32.883544+0000 mon.a (mon.0) 2237 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-02-13T03:32:35.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:34 smithi161 bash[27879]: cluster 2024-02-13T03:32:33.372137+0000 mgr.y (mgr.24473) 459 : cluster [DBG] pgmap v654: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:32:35.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:34 smithi161 bash[19769]: cluster 2024-02-13T03:32:33.372137+0000 mgr.y (mgr.24473) 459 : cluster [DBG] pgmap v654: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:32:35.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:34 smithi183 bash[20824]: cluster 2024-02-13T03:32:33.372137+0000 mgr.y (mgr.24473) 459 : cluster [DBG] pgmap v654: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:32:36.487 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:32:36 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:32:37.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:36 smithi161 bash[27879]: cluster 2024-02-13T03:32:35.372933+0000 mgr.y (mgr.24473) 460 : cluster [DBG] pgmap v655: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-13T03:32:37.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:36 smithi161 bash[27879]: audit 2024-02-13T03:32:36.090765+0000 mgr.y (mgr.24473) 461 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:37.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:36 smithi161 bash[19769]: cluster 2024-02-13T03:32:35.372933+0000 mgr.y (mgr.24473) 460 : cluster [DBG] pgmap v655: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-13T03:32:37.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:36 smithi161 bash[19769]: audit 2024-02-13T03:32:36.090765+0000 mgr.y (mgr.24473) 461 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:36 smithi183 bash[20824]: cluster 2024-02-13T03:32:35.372933+0000 mgr.y (mgr.24473) 460 : cluster [DBG] pgmap v655: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-13T03:32:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:36 smithi183 bash[20824]: audit 2024-02-13T03:32:36.090765+0000 mgr.y (mgr.24473) 461 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:39.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:38 smithi161 bash[27879]: cluster 2024-02-13T03:32:37.375135+0000 mgr.y (mgr.24473) 462 : cluster [DBG] pgmap v656: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-13T03:32:39.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:38 smithi161 bash[27879]: cluster 2024-02-13T03:32:38.806063+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-02-13T03:32:39.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:38 smithi161 bash[19769]: cluster 2024-02-13T03:32:37.375135+0000 mgr.y (mgr.24473) 462 : cluster [DBG] pgmap v656: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-13T03:32:39.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:38 smithi161 bash[19769]: cluster 2024-02-13T03:32:38.806063+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-02-13T03:32:39.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:38 smithi183 bash[20824]: cluster 2024-02-13T03:32:37.375135+0000 mgr.y (mgr.24473) 462 : cluster [DBG] pgmap v656: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-13T03:32:39.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:38 smithi183 bash[20824]: cluster 2024-02-13T03:32:38.806063+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-02-13T03:32:39.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:32:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:32:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:32:41.176 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:32:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:32:41] "GET /metrics HTTP/1.1" 200 187282 "" "Prometheus/2.33.4" 2024-02-13T03:32:41.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:40 smithi161 bash[27879]: cluster 2024-02-13T03:32:39.375949+0000 mgr.y (mgr.24473) 463 : cluster [DBG] pgmap v658: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T03:32:41.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:40 smithi161 bash[19769]: cluster 2024-02-13T03:32:39.375949+0000 mgr.y (mgr.24473) 463 : cluster [DBG] pgmap v658: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T03:32:41.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:40 smithi183 bash[20824]: cluster 2024-02-13T03:32:39.375949+0000 mgr.y (mgr.24473) 463 : cluster [DBG] pgmap v658: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T03:32:43.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:42 smithi161 bash[27879]: cluster 2024-02-13T03:32:41.378154+0000 mgr.y (mgr.24473) 464 : cluster [DBG] pgmap v659: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-13T03:32:43.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:42 smithi161 bash[19769]: cluster 2024-02-13T03:32:41.378154+0000 mgr.y (mgr.24473) 464 : cluster [DBG] pgmap v659: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-13T03:32:43.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:42 smithi183 bash[20824]: cluster 2024-02-13T03:32:41.378154+0000 mgr.y (mgr.24473) 464 : cluster [DBG] pgmap v659: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-13T03:32:44.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:44 smithi161 bash[27879]: audit 2024-02-13T03:32:42.922934+0000 mon.a (mon.0) 2239 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:44.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:44 smithi161 bash[27879]: audit 2024-02-13T03:32:42.924410+0000 mon.a (mon.0) 2240 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-61"}]: dispatch 2024-02-13T03:32:44.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:44 smithi161 bash[27879]: audit 2024-02-13T03:32:42.924686+0000 mon.b (mon.1) 330 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:44.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:44 smithi161 bash[27879]: audit 2024-02-13T03:32:42.926281+0000 mon.b (mon.1) 331 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-61"}]: dispatch 2024-02-13T03:32:44.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:44 smithi161 bash[19769]: audit 2024-02-13T03:32:42.922934+0000 mon.a (mon.0) 2239 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:44.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:44 smithi161 bash[19769]: audit 2024-02-13T03:32:42.924410+0000 mon.a (mon.0) 2240 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-61"}]: dispatch 2024-02-13T03:32:44.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:44 smithi161 bash[19769]: audit 2024-02-13T03:32:42.924686+0000 mon.b (mon.1) 330 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:44.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:44 smithi161 bash[19769]: audit 2024-02-13T03:32:42.926281+0000 mon.b (mon.1) 331 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-61"}]: dispatch 2024-02-13T03:32:44.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:44 smithi183 bash[20824]: audit 2024-02-13T03:32:42.922934+0000 mon.a (mon.0) 2239 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:44.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:44 smithi183 bash[20824]: audit 2024-02-13T03:32:42.924410+0000 mon.a (mon.0) 2240 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-61"}]: dispatch 2024-02-13T03:32:44.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:44 smithi183 bash[20824]: audit 2024-02-13T03:32:42.924686+0000 mon.b (mon.1) 330 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:32:44.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:44 smithi183 bash[20824]: audit 2024-02-13T03:32:42.926281+0000 mon.b (mon.1) 331 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-61"}]: dispatch 2024-02-13T03:32:45.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:45 smithi161 bash[27879]: cluster 2024-02-13T03:32:43.378995+0000 mgr.y (mgr.24473) 465 : cluster [DBG] pgmap v660: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-13T03:32:45.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:45 smithi161 bash[27879]: cluster 2024-02-13T03:32:43.919039+0000 mon.a (mon.0) 2241 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-02-13T03:32:45.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:45 smithi161 bash[27879]: cluster 2024-02-13T03:32:44.919900+0000 mon.a (mon.0) 2242 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-02-13T03:32:45.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:45 smithi161 bash[27879]: audit 2024-02-13T03:32:44.926733+0000 mon.b (mon.1) 332 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:45.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:45 smithi161 bash[27879]: audit 2024-02-13T03:32:44.928485+0000 mon.a (mon.0) 2243 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:45.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:45 smithi161 bash[19769]: cluster 2024-02-13T03:32:43.378995+0000 mgr.y (mgr.24473) 465 : cluster [DBG] pgmap v660: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-13T03:32:45.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:45 smithi161 bash[19769]: cluster 2024-02-13T03:32:43.919039+0000 mon.a (mon.0) 2241 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-02-13T03:32:45.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:45 smithi161 bash[19769]: cluster 2024-02-13T03:32:44.919900+0000 mon.a (mon.0) 2242 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-02-13T03:32:45.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:45 smithi161 bash[19769]: audit 2024-02-13T03:32:44.926733+0000 mon.b (mon.1) 332 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:45.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:45 smithi161 bash[19769]: audit 2024-02-13T03:32:44.928485+0000 mon.a (mon.0) 2243 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:45.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:45 smithi183 bash[20824]: cluster 2024-02-13T03:32:43.378995+0000 mgr.y (mgr.24473) 465 : cluster [DBG] pgmap v660: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-13T03:32:45.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:45 smithi183 bash[20824]: cluster 2024-02-13T03:32:43.919039+0000 mon.a (mon.0) 2241 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-02-13T03:32:45.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:45 smithi183 bash[20824]: cluster 2024-02-13T03:32:44.919900+0000 mon.a (mon.0) 2242 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-02-13T03:32:45.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:45 smithi183 bash[20824]: audit 2024-02-13T03:32:44.926733+0000 mon.b (mon.1) 332 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:45.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:45 smithi183 bash[20824]: audit 2024-02-13T03:32:44.928485+0000 mon.a (mon.0) 2243 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:32:46.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:32:46 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:32:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:46 smithi183 bash[20824]: cluster 2024-02-13T03:32:45.379759+0000 mgr.y (mgr.24473) 466 : cluster [DBG] pgmap v663: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:32:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:46 smithi183 bash[20824]: audit 2024-02-13T03:32:45.925734+0000 mon.a (mon.0) 2244 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:32:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:46 smithi183 bash[20824]: cluster 2024-02-13T03:32:45.925952+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-02-13T03:32:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:46 smithi183 bash[20824]: audit 2024-02-13T03:32:45.936077+0000 mon.b (mon.1) 333 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:32:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:46 smithi183 bash[20824]: audit 2024-02-13T03:32:45.938356+0000 mon.a (mon.0) 2246 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:46 smithi183 bash[20824]: audit 2024-02-13T03:32:45.940011+0000 mon.b (mon.1) 334 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:47.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:46 smithi183 bash[20824]: audit 2024-02-13T03:32:46.096598+0000 mgr.y (mgr.24473) 467 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:47.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[27879]: cluster 2024-02-13T03:32:45.379759+0000 mgr.y (mgr.24473) 466 : cluster [DBG] pgmap v663: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:32:47.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[27879]: audit 2024-02-13T03:32:45.925734+0000 mon.a (mon.0) 2244 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:32:47.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[27879]: cluster 2024-02-13T03:32:45.925952+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-02-13T03:32:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[27879]: audit 2024-02-13T03:32:45.936077+0000 mon.b (mon.1) 333 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:32:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[27879]: audit 2024-02-13T03:32:45.938356+0000 mon.a (mon.0) 2246 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[27879]: audit 2024-02-13T03:32:45.940011+0000 mon.b (mon.1) 334 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[27879]: audit 2024-02-13T03:32:46.096598+0000 mgr.y (mgr.24473) 467 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:47.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[19769]: cluster 2024-02-13T03:32:45.379759+0000 mgr.y (mgr.24473) 466 : cluster [DBG] pgmap v663: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:32:47.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[19769]: audit 2024-02-13T03:32:45.925734+0000 mon.a (mon.0) 2244 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:32:47.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[19769]: cluster 2024-02-13T03:32:45.925952+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-02-13T03:32:47.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[19769]: audit 2024-02-13T03:32:45.936077+0000 mon.b (mon.1) 333 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:32:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[19769]: audit 2024-02-13T03:32:45.938356+0000 mon.a (mon.0) 2246 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[19769]: audit 2024-02-13T03:32:45.940011+0000 mon.b (mon.1) 334 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:32:47.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:46 smithi161 bash[19769]: audit 2024-02-13T03:32:46.096598+0000 mgr.y (mgr.24473) 467 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:48.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:47 smithi183 bash[20824]: audit 2024-02-13T03:32:46.925773+0000 mon.a (mon.0) 2247 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:32:48.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:47 smithi183 bash[20824]: cluster 2024-02-13T03:32:46.925917+0000 mon.a (mon.0) 2248 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-02-13T03:32:48.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:47 smithi161 bash[27879]: audit 2024-02-13T03:32:46.925773+0000 mon.a (mon.0) 2247 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:32:48.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:47 smithi161 bash[27879]: cluster 2024-02-13T03:32:46.925917+0000 mon.a (mon.0) 2248 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-02-13T03:32:48.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:47 smithi161 bash[19769]: audit 2024-02-13T03:32:46.925773+0000 mon.a (mon.0) 2247 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:32:48.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:47 smithi161 bash[19769]: cluster 2024-02-13T03:32:46.925917+0000 mon.a (mon.0) 2248 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-02-13T03:32:49.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:48 smithi183 bash[20824]: cluster 2024-02-13T03:32:47.380578+0000 mgr.y (mgr.24473) 468 : cluster [DBG] pgmap v666: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:32:49.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:48 smithi183 bash[20824]: cluster 2024-02-13T03:32:47.945447+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-02-13T03:32:49.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:48 smithi161 bash[27879]: cluster 2024-02-13T03:32:47.380578+0000 mgr.y (mgr.24473) 468 : cluster [DBG] pgmap v666: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:32:49.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:48 smithi161 bash[27879]: cluster 2024-02-13T03:32:47.945447+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-02-13T03:32:49.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:48 smithi161 bash[19769]: cluster 2024-02-13T03:32:47.380578+0000 mgr.y (mgr.24473) 468 : cluster [DBG] pgmap v666: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:32:49.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:48 smithi161 bash[19769]: cluster 2024-02-13T03:32:47.945447+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-02-13T03:32:49.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:32:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:32:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:32:50.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:49 smithi183 bash[20824]: cluster 2024-02-13T03:32:48.955140+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-02-13T03:32:50.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:49 smithi161 bash[27879]: cluster 2024-02-13T03:32:48.955140+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-02-13T03:32:50.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:49 smithi161 bash[19769]: cluster 2024-02-13T03:32:48.955140+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-02-13T03:32:51.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:50 smithi183 bash[20824]: cluster 2024-02-13T03:32:49.381524+0000 mgr.y (mgr.24473) 469 : cluster [DBG] pgmap v669: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:32:51.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:50 smithi183 bash[20824]: cluster 2024-02-13T03:32:49.977315+0000 mon.a (mon.0) 2251 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-02-13T03:32:51.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:50 smithi161 bash[19769]: cluster 2024-02-13T03:32:49.381524+0000 mgr.y (mgr.24473) 469 : cluster [DBG] pgmap v669: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:32:51.430 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:50 smithi161 bash[19769]: cluster 2024-02-13T03:32:49.977315+0000 mon.a (mon.0) 2251 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-02-13T03:32:51.430 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:32:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:32:51] "GET /metrics HTTP/1.1" 200 187383 "" "Prometheus/2.33.4" 2024-02-13T03:32:51.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:50 smithi161 bash[27879]: cluster 2024-02-13T03:32:49.381524+0000 mgr.y (mgr.24473) 469 : cluster [DBG] pgmap v669: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:32:51.431 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:50 smithi161 bash[27879]: cluster 2024-02-13T03:32:49.977315+0000 mon.a (mon.0) 2251 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-02-13T03:32:53.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:52 smithi183 bash[20824]: cluster 2024-02-13T03:32:51.383889+0000 mgr.y (mgr.24473) 470 : cluster [DBG] pgmap v671: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-02-13T03:32:53.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:52 smithi161 bash[27879]: cluster 2024-02-13T03:32:51.383889+0000 mgr.y (mgr.24473) 470 : cluster [DBG] pgmap v671: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-02-13T03:32:53.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:52 smithi161 bash[19769]: cluster 2024-02-13T03:32:51.383889+0000 mgr.y (mgr.24473) 470 : cluster [DBG] pgmap v671: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.4 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-02-13T03:32:55.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:54 smithi161 bash[27879]: cluster 2024-02-13T03:32:53.384722+0000 mgr.y (mgr.24473) 471 : cluster [DBG] pgmap v672: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-13T03:32:55.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:54 smithi161 bash[27879]: cluster 2024-02-13T03:32:53.807607+0000 mon.a (mon.0) 2252 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-02-13T03:32:55.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:54 smithi161 bash[19769]: cluster 2024-02-13T03:32:53.384722+0000 mgr.y (mgr.24473) 471 : cluster [DBG] pgmap v672: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-13T03:32:55.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:54 smithi161 bash[19769]: cluster 2024-02-13T03:32:53.807607+0000 mon.a (mon.0) 2252 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-02-13T03:32:55.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:54 smithi183 bash[20824]: cluster 2024-02-13T03:32:53.384722+0000 mgr.y (mgr.24473) 471 : cluster [DBG] pgmap v672: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-13T03:32:55.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:54 smithi183 bash[20824]: cluster 2024-02-13T03:32:53.807607+0000 mon.a (mon.0) 2252 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-02-13T03:32:56.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:32:56 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:32:57.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:56 smithi161 bash[27879]: cluster 2024-02-13T03:32:55.385508+0000 mgr.y (mgr.24473) 472 : cluster [DBG] pgmap v674: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-02-13T03:32:57.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:56 smithi161 bash[27879]: audit 2024-02-13T03:32:56.107929+0000 mgr.y (mgr.24473) 473 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:57.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:56 smithi161 bash[19769]: cluster 2024-02-13T03:32:55.385508+0000 mgr.y (mgr.24473) 472 : cluster [DBG] pgmap v674: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-02-13T03:32:57.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:56 smithi161 bash[19769]: audit 2024-02-13T03:32:56.107929+0000 mgr.y (mgr.24473) 473 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:57.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:56 smithi183 bash[20824]: cluster 2024-02-13T03:32:55.385508+0000 mgr.y (mgr.24473) 472 : cluster [DBG] pgmap v674: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-02-13T03:32:57.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:56 smithi183 bash[20824]: audit 2024-02-13T03:32:56.107929+0000 mgr.y (mgr.24473) 473 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:32:59.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:32:58 smithi161 bash[27879]: cluster 2024-02-13T03:32:57.387771+0000 mgr.y (mgr.24473) 474 : cluster [DBG] pgmap v675: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-13T03:32:59.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:32:58 smithi161 bash[19769]: cluster 2024-02-13T03:32:57.387771+0000 mgr.y (mgr.24473) 474 : cluster [DBG] pgmap v675: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-13T03:32:59.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:32:58 smithi183 bash[20824]: cluster 2024-02-13T03:32:57.387771+0000 mgr.y (mgr.24473) 474 : cluster [DBG] pgmap v675: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-13T03:32:59.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:32:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:32:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:33:01.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:00 smithi161 bash[19769]: cluster 2024-02-13T03:32:59.388609+0000 mgr.y (mgr.24473) 475 : cluster [DBG] pgmap v676: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-13T03:33:01.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:00 smithi161 bash[19769]: audit 2024-02-13T03:33:00.016195+0000 mon.a (mon.0) 2253 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:01.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:00 smithi161 bash[19769]: audit 2024-02-13T03:33:00.017774+0000 mon.b (mon.1) 335 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:01.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:00 smithi161 bash[19769]: audit 2024-02-13T03:33:00.017924+0000 mon.a (mon.0) 2254 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-63"}]: dispatch 2024-02-13T03:33:01.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:00 smithi161 bash[19769]: audit 2024-02-13T03:33:00.019796+0000 mon.b (mon.1) 336 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-63"}]: dispatch 2024-02-13T03:33:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:00 smithi161 bash[27879]: cluster 2024-02-13T03:32:59.388609+0000 mgr.y (mgr.24473) 475 : cluster [DBG] pgmap v676: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-13T03:33:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:00 smithi161 bash[27879]: audit 2024-02-13T03:33:00.016195+0000 mon.a (mon.0) 2253 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:00 smithi161 bash[27879]: audit 2024-02-13T03:33:00.017774+0000 mon.b (mon.1) 335 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:00 smithi161 bash[27879]: audit 2024-02-13T03:33:00.017924+0000 mon.a (mon.0) 2254 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-63"}]: dispatch 2024-02-13T03:33:01.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:00 smithi161 bash[27879]: audit 2024-02-13T03:33:00.019796+0000 mon.b (mon.1) 336 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-63"}]: dispatch 2024-02-13T03:33:01.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:00 smithi183 bash[20824]: cluster 2024-02-13T03:32:59.388609+0000 mgr.y (mgr.24473) 475 : cluster [DBG] pgmap v676: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-13T03:33:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:00 smithi183 bash[20824]: audit 2024-02-13T03:33:00.016195+0000 mon.a (mon.0) 2253 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:00 smithi183 bash[20824]: audit 2024-02-13T03:33:00.017774+0000 mon.b (mon.1) 335 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:00 smithi183 bash[20824]: audit 2024-02-13T03:33:00.017924+0000 mon.a (mon.0) 2254 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-63"}]: dispatch 2024-02-13T03:33:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:00 smithi183 bash[20824]: audit 2024-02-13T03:33:00.019796+0000 mon.b (mon.1) 336 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-63"}]: dispatch 2024-02-13T03:33:01.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:33:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:33:01] "GET /metrics HTTP/1.1" 200 187383 "" "Prometheus/2.33.4" 2024-02-13T03:33:02.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:01 smithi161 bash[27879]: cluster 2024-02-13T03:33:00.826280+0000 mon.a (mon.0) 2255 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-02-13T03:33:02.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:01 smithi161 bash[19769]: cluster 2024-02-13T03:33:00.826280+0000 mon.a (mon.0) 2255 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-02-13T03:33:02.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:01 smithi183 bash[20824]: cluster 2024-02-13T03:33:00.826280+0000 mon.a (mon.0) 2255 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-02-13T03:33:03.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:02 smithi161 bash[19769]: cluster 2024-02-13T03:33:01.389460+0000 mgr.y (mgr.24473) 476 : cluster [DBG] pgmap v678: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:03.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:02 smithi161 bash[19769]: cluster 2024-02-13T03:33:01.851520+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-02-13T03:33:03.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:02 smithi161 bash[19769]: audit 2024-02-13T03:33:01.854316+0000 mon.a (mon.0) 2257 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:03.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:02 smithi161 bash[19769]: audit 2024-02-13T03:33:01.855831+0000 mon.b (mon.1) 337 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:03.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:02 smithi161 bash[27879]: cluster 2024-02-13T03:33:01.389460+0000 mgr.y (mgr.24473) 476 : cluster [DBG] pgmap v678: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:03.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:02 smithi161 bash[27879]: cluster 2024-02-13T03:33:01.851520+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-02-13T03:33:03.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:02 smithi161 bash[27879]: audit 2024-02-13T03:33:01.854316+0000 mon.a (mon.0) 2257 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:03.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:02 smithi161 bash[27879]: audit 2024-02-13T03:33:01.855831+0000 mon.b (mon.1) 337 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:03.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:02 smithi183 bash[20824]: cluster 2024-02-13T03:33:01.389460+0000 mgr.y (mgr.24473) 476 : cluster [DBG] pgmap v678: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:03.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:02 smithi183 bash[20824]: cluster 2024-02-13T03:33:01.851520+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-02-13T03:33:03.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:02 smithi183 bash[20824]: audit 2024-02-13T03:33:01.854316+0000 mon.a (mon.0) 2257 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:03.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:02 smithi183 bash[20824]: audit 2024-02-13T03:33:01.855831+0000 mon.b (mon.1) 337 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:04.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:03 smithi161 bash[19769]: audit 2024-02-13T03:33:02.855315+0000 mon.a (mon.0) 2258 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:04.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:03 smithi161 bash[19769]: cluster 2024-02-13T03:33:02.855979+0000 mon.a (mon.0) 2259 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-02-13T03:33:04.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:03 smithi161 bash[19769]: audit 2024-02-13T03:33:02.861469+0000 mon.b (mon.1) 338 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:04.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:03 smithi161 bash[27879]: audit 2024-02-13T03:33:02.855315+0000 mon.a (mon.0) 2258 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:04.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:03 smithi161 bash[27879]: cluster 2024-02-13T03:33:02.855979+0000 mon.a (mon.0) 2259 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-02-13T03:33:04.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:03 smithi161 bash[27879]: audit 2024-02-13T03:33:02.861469+0000 mon.b (mon.1) 338 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:04.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:03 smithi183 bash[20824]: audit 2024-02-13T03:33:02.855315+0000 mon.a (mon.0) 2258 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:04.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:03 smithi183 bash[20824]: cluster 2024-02-13T03:33:02.855979+0000 mon.a (mon.0) 2259 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-02-13T03:33:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:03 smithi183 bash[20824]: audit 2024-02-13T03:33:02.861469+0000 mon.b (mon.1) 338 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:05.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:04 smithi161 bash[19769]: cluster 2024-02-13T03:33:03.390338+0000 mgr.y (mgr.24473) 477 : cluster [DBG] pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:05.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:04 smithi161 bash[19769]: cluster 2024-02-13T03:33:03.876341+0000 mon.a (mon.0) 2260 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-02-13T03:33:05.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:04 smithi161 bash[19769]: audit 2024-02-13T03:33:03.917108+0000 mon.a (mon.0) 2261 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:05.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:04 smithi161 bash[19769]: audit 2024-02-13T03:33:03.918430+0000 mon.a (mon.0) 2262 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-65"}]: dispatch 2024-02-13T03:33:05.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:04 smithi161 bash[19769]: audit 2024-02-13T03:33:03.918939+0000 mon.b (mon.1) 339 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:05.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:04 smithi161 bash[19769]: audit 2024-02-13T03:33:03.920427+0000 mon.b (mon.1) 340 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-65"}]: dispatch 2024-02-13T03:33:05.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:04 smithi161 bash[27879]: cluster 2024-02-13T03:33:03.390338+0000 mgr.y (mgr.24473) 477 : cluster [DBG] pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:05.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:04 smithi161 bash[27879]: cluster 2024-02-13T03:33:03.876341+0000 mon.a (mon.0) 2260 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-02-13T03:33:05.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:04 smithi161 bash[27879]: audit 2024-02-13T03:33:03.917108+0000 mon.a (mon.0) 2261 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:05.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:04 smithi161 bash[27879]: audit 2024-02-13T03:33:03.918430+0000 mon.a (mon.0) 2262 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-65"}]: dispatch 2024-02-13T03:33:05.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:04 smithi161 bash[27879]: audit 2024-02-13T03:33:03.918939+0000 mon.b (mon.1) 339 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:05.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:04 smithi161 bash[27879]: audit 2024-02-13T03:33:03.920427+0000 mon.b (mon.1) 340 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-65"}]: dispatch 2024-02-13T03:33:05.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:04 smithi183 bash[20824]: cluster 2024-02-13T03:33:03.390338+0000 mgr.y (mgr.24473) 477 : cluster [DBG] pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:04 smithi183 bash[20824]: cluster 2024-02-13T03:33:03.876341+0000 mon.a (mon.0) 2260 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-02-13T03:33:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:04 smithi183 bash[20824]: audit 2024-02-13T03:33:03.917108+0000 mon.a (mon.0) 2261 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:04 smithi183 bash[20824]: audit 2024-02-13T03:33:03.918430+0000 mon.a (mon.0) 2262 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-65"}]: dispatch 2024-02-13T03:33:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:04 smithi183 bash[20824]: audit 2024-02-13T03:33:03.918939+0000 mon.b (mon.1) 339 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:04 smithi183 bash[20824]: audit 2024-02-13T03:33:03.920427+0000 mon.b (mon.1) 340 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-65"}]: dispatch 2024-02-13T03:33:06.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:05 smithi161 bash[19769]: cluster 2024-02-13T03:33:04.899928+0000 mon.a (mon.0) 2263 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-02-13T03:33:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:05 smithi161 bash[27879]: cluster 2024-02-13T03:33:04.899928+0000 mon.a (mon.0) 2263 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-02-13T03:33:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:05 smithi183 bash[20824]: cluster 2024-02-13T03:33:04.899928+0000 mon.a (mon.0) 2263 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-02-13T03:33:06.237 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:33:06 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:33:07.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:06 smithi161 bash[19769]: cluster 2024-02-13T03:33:05.391061+0000 mgr.y (mgr.24473) 478 : cluster [DBG] pgmap v684: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:07.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:06 smithi161 bash[19769]: cluster 2024-02-13T03:33:05.912264+0000 mon.a (mon.0) 2264 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-02-13T03:33:07.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:06 smithi161 bash[19769]: audit 2024-02-13T03:33:05.918059+0000 mon.a (mon.0) 2265 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:07.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:06 smithi161 bash[19769]: audit 2024-02-13T03:33:05.918223+0000 mon.b (mon.1) 341 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:07.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:06 smithi161 bash[19769]: audit 2024-02-13T03:33:06.119237+0000 mgr.y (mgr.24473) 479 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:06 smithi161 bash[19769]: audit 2024-02-13T03:33:06.608878+0000 mon.c (mon.2) 522 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:33:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:06 smithi161 bash[27879]: cluster 2024-02-13T03:33:05.391061+0000 mgr.y (mgr.24473) 478 : cluster [DBG] pgmap v684: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:06 smithi161 bash[27879]: cluster 2024-02-13T03:33:05.912264+0000 mon.a (mon.0) 2264 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-02-13T03:33:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:06 smithi161 bash[27879]: audit 2024-02-13T03:33:05.918059+0000 mon.a (mon.0) 2265 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:06 smithi161 bash[27879]: audit 2024-02-13T03:33:05.918223+0000 mon.b (mon.1) 341 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:06 smithi161 bash[27879]: audit 2024-02-13T03:33:06.119237+0000 mgr.y (mgr.24473) 479 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:06 smithi161 bash[27879]: audit 2024-02-13T03:33:06.608878+0000 mon.c (mon.2) 522 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:33:07.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:06 smithi183 bash[20824]: cluster 2024-02-13T03:33:05.391061+0000 mgr.y (mgr.24473) 478 : cluster [DBG] pgmap v684: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:06 smithi183 bash[20824]: cluster 2024-02-13T03:33:05.912264+0000 mon.a (mon.0) 2264 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-02-13T03:33:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:06 smithi183 bash[20824]: audit 2024-02-13T03:33:05.918059+0000 mon.a (mon.0) 2265 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:06 smithi183 bash[20824]: audit 2024-02-13T03:33:05.918223+0000 mon.b (mon.1) 341 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:06 smithi183 bash[20824]: audit 2024-02-13T03:33:06.119237+0000 mgr.y (mgr.24473) 479 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:06 smithi183 bash[20824]: audit 2024-02-13T03:33:06.608878+0000 mon.c (mon.2) 522 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:33:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[27879]: audit 2024-02-13T03:33:06.916389+0000 mon.a (mon.0) 2266 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[27879]: cluster 2024-02-13T03:33:06.916694+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-02-13T03:33:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[27879]: audit 2024-02-13T03:33:06.926204+0000 mon.b (mon.1) 342 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[27879]: audit 2024-02-13T03:33:06.962739+0000 mon.a (mon.0) 2268 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[27879]: audit 2024-02-13T03:33:06.964037+0000 mon.a (mon.0) 2269 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-67"}]: dispatch 2024-02-13T03:33:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[27879]: audit 2024-02-13T03:33:06.964660+0000 mon.b (mon.1) 343 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[27879]: audit 2024-02-13T03:33:06.966017+0000 mon.b (mon.1) 344 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-67"}]: dispatch 2024-02-13T03:33:08.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[19769]: audit 2024-02-13T03:33:06.916389+0000 mon.a (mon.0) 2266 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:08.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[19769]: cluster 2024-02-13T03:33:06.916694+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-02-13T03:33:08.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[19769]: audit 2024-02-13T03:33:06.926204+0000 mon.b (mon.1) 342 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:08.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[19769]: audit 2024-02-13T03:33:06.962739+0000 mon.a (mon.0) 2268 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:08.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[19769]: audit 2024-02-13T03:33:06.964037+0000 mon.a (mon.0) 2269 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-67"}]: dispatch 2024-02-13T03:33:08.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[19769]: audit 2024-02-13T03:33:06.964660+0000 mon.b (mon.1) 343 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:08.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:08 smithi161 bash[19769]: audit 2024-02-13T03:33:06.966017+0000 mon.b (mon.1) 344 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-67"}]: dispatch 2024-02-13T03:33:08.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:08 smithi183 bash[20824]: audit 2024-02-13T03:33:06.916389+0000 mon.a (mon.0) 2266 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:08.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:08 smithi183 bash[20824]: cluster 2024-02-13T03:33:06.916694+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-02-13T03:33:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:08 smithi183 bash[20824]: audit 2024-02-13T03:33:06.926204+0000 mon.b (mon.1) 342 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:08 smithi183 bash[20824]: audit 2024-02-13T03:33:06.962739+0000 mon.a (mon.0) 2268 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:08 smithi183 bash[20824]: audit 2024-02-13T03:33:06.964037+0000 mon.a (mon.0) 2269 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-67"}]: dispatch 2024-02-13T03:33:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:08 smithi183 bash[20824]: audit 2024-02-13T03:33:06.964660+0000 mon.b (mon.1) 343 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:08 smithi183 bash[20824]: audit 2024-02-13T03:33:06.966017+0000 mon.b (mon.1) 344 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-67"}]: dispatch 2024-02-13T03:33:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:09 smithi161 bash[19769]: cluster 2024-02-13T03:33:07.391927+0000 mgr.y (mgr.24473) 480 : cluster [DBG] pgmap v687: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:33:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:09 smithi161 bash[19769]: cluster 2024-02-13T03:33:07.933744+0000 mon.a (mon.0) 2270 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-02-13T03:33:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:09 smithi161 bash[19769]: cluster 2024-02-13T03:33:08.936757+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-02-13T03:33:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:09 smithi161 bash[19769]: audit 2024-02-13T03:33:08.941784+0000 mon.b (mon.1) 345 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:09 smithi161 bash[19769]: audit 2024-02-13T03:33:08.943351+0000 mon.a (mon.0) 2272 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:09 smithi161 bash[27879]: cluster 2024-02-13T03:33:07.391927+0000 mgr.y (mgr.24473) 480 : cluster [DBG] pgmap v687: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:33:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:09 smithi161 bash[27879]: cluster 2024-02-13T03:33:07.933744+0000 mon.a (mon.0) 2270 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-02-13T03:33:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:09 smithi161 bash[27879]: cluster 2024-02-13T03:33:08.936757+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-02-13T03:33:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:09 smithi161 bash[27879]: audit 2024-02-13T03:33:08.941784+0000 mon.b (mon.1) 345 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:09 smithi161 bash[27879]: audit 2024-02-13T03:33:08.943351+0000 mon.a (mon.0) 2272 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:09.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:09 smithi183 bash[20824]: cluster 2024-02-13T03:33:07.391927+0000 mgr.y (mgr.24473) 480 : cluster [DBG] pgmap v687: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:33:09.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:09 smithi183 bash[20824]: cluster 2024-02-13T03:33:07.933744+0000 mon.a (mon.0) 2270 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-02-13T03:33:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:09 smithi183 bash[20824]: cluster 2024-02-13T03:33:08.936757+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-02-13T03:33:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:09 smithi183 bash[20824]: audit 2024-02-13T03:33:08.941784+0000 mon.b (mon.1) 345 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:09 smithi183 bash[20824]: audit 2024-02-13T03:33:08.943351+0000 mon.a (mon.0) 2272 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:09.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:33:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:33:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:33:11.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:10 smithi183 bash[20824]: cluster 2024-02-13T03:33:09.392741+0000 mgr.y (mgr.24473) 481 : cluster [DBG] pgmap v690: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:33:11.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:10 smithi183 bash[20824]: audit 2024-02-13T03:33:09.940459+0000 mon.a (mon.0) 2273 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:11.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:10 smithi183 bash[20824]: cluster 2024-02-13T03:33:09.941031+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-02-13T03:33:11.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:10 smithi183 bash[20824]: audit 2024-02-13T03:33:09.945298+0000 mon.b (mon.1) 346 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:11.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:10 smithi183 bash[20824]: audit 2024-02-13T03:33:09.949360+0000 mon.b (mon.1) 347 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:33:11.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:10 smithi183 bash[20824]: audit 2024-02-13T03:33:09.950307+0000 mon.a (mon.0) 2275 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:33:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:10 smithi161 bash[27879]: cluster 2024-02-13T03:33:09.392741+0000 mgr.y (mgr.24473) 481 : cluster [DBG] pgmap v690: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:33:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:10 smithi161 bash[27879]: audit 2024-02-13T03:33:09.940459+0000 mon.a (mon.0) 2273 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:10 smithi161 bash[27879]: cluster 2024-02-13T03:33:09.941031+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-02-13T03:33:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:10 smithi161 bash[27879]: audit 2024-02-13T03:33:09.945298+0000 mon.b (mon.1) 346 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:10 smithi161 bash[27879]: audit 2024-02-13T03:33:09.949360+0000 mon.b (mon.1) 347 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:33:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:10 smithi161 bash[27879]: audit 2024-02-13T03:33:09.950307+0000 mon.a (mon.0) 2275 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:33:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:10 smithi161 bash[19769]: cluster 2024-02-13T03:33:09.392741+0000 mgr.y (mgr.24473) 481 : cluster [DBG] pgmap v690: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:33:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:10 smithi161 bash[19769]: audit 2024-02-13T03:33:09.940459+0000 mon.a (mon.0) 2273 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:10 smithi161 bash[19769]: cluster 2024-02-13T03:33:09.941031+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-02-13T03:33:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:10 smithi161 bash[19769]: audit 2024-02-13T03:33:09.945298+0000 mon.b (mon.1) 346 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:10 smithi161 bash[19769]: audit 2024-02-13T03:33:09.949360+0000 mon.b (mon.1) 347 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:33:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:10 smithi161 bash[19769]: audit 2024-02-13T03:33:09.950307+0000 mon.a (mon.0) 2275 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:33:11.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:33:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:33:11] "GET /metrics HTTP/1.1" 200 187496 "" "Prometheus/2.33.4" 2024-02-13T03:33:12.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:11 smithi183 bash[20824]: audit 2024-02-13T03:33:10.942632+0000 mon.a (mon.0) 2276 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:33:12.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:11 smithi183 bash[20824]: cluster 2024-02-13T03:33:10.942734+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-02-13T03:33:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:11 smithi161 bash[19769]: audit 2024-02-13T03:33:10.942632+0000 mon.a (mon.0) 2276 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:33:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:11 smithi161 bash[19769]: cluster 2024-02-13T03:33:10.942734+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-02-13T03:33:12.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:11 smithi161 bash[27879]: audit 2024-02-13T03:33:10.942632+0000 mon.a (mon.0) 2276 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:33:12.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:11 smithi161 bash[27879]: cluster 2024-02-13T03:33:10.942734+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-02-13T03:33:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:12 smithi183 bash[20824]: cluster 2024-02-13T03:33:11.393594+0000 mgr.y (mgr.24473) 482 : cluster [DBG] pgmap v693: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-13T03:33:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:12 smithi183 bash[20824]: cluster 2024-02-13T03:33:11.963988+0000 mon.a (mon.0) 2278 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-02-13T03:33:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:12 smithi183 bash[20824]: audit 2024-02-13T03:33:12.476556+0000 mon.a (mon.0) 2279 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:33:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:12 smithi183 bash[20824]: audit 2024-02-13T03:33:12.774739+0000 mon.c (mon.2) 523 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:33:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:12 smithi183 bash[20824]: audit 2024-02-13T03:33:12.776061+0000 mon.c (mon.2) 524 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:33:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:12 smithi183 bash[20824]: audit 2024-02-13T03:33:12.784261+0000 mon.a (mon.0) 2280 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:33:13.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:12 smithi161 bash[19769]: cluster 2024-02-13T03:33:11.393594+0000 mgr.y (mgr.24473) 482 : cluster [DBG] pgmap v693: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-13T03:33:13.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:12 smithi161 bash[19769]: cluster 2024-02-13T03:33:11.963988+0000 mon.a (mon.0) 2278 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-02-13T03:33:13.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:12 smithi161 bash[19769]: audit 2024-02-13T03:33:12.476556+0000 mon.a (mon.0) 2279 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:33:13.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:12 smithi161 bash[19769]: audit 2024-02-13T03:33:12.774739+0000 mon.c (mon.2) 523 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:33:13.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:12 smithi161 bash[19769]: audit 2024-02-13T03:33:12.776061+0000 mon.c (mon.2) 524 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:33:13.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:12 smithi161 bash[19769]: audit 2024-02-13T03:33:12.784261+0000 mon.a (mon.0) 2280 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:33:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:12 smithi161 bash[27879]: cluster 2024-02-13T03:33:11.393594+0000 mgr.y (mgr.24473) 482 : cluster [DBG] pgmap v693: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-13T03:33:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:12 smithi161 bash[27879]: cluster 2024-02-13T03:33:11.963988+0000 mon.a (mon.0) 2278 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-02-13T03:33:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:12 smithi161 bash[27879]: audit 2024-02-13T03:33:12.476556+0000 mon.a (mon.0) 2279 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:33:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:12 smithi161 bash[27879]: audit 2024-02-13T03:33:12.774739+0000 mon.c (mon.2) 523 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:33:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:12 smithi161 bash[27879]: audit 2024-02-13T03:33:12.776061+0000 mon.c (mon.2) 524 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:33:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:12 smithi161 bash[27879]: audit 2024-02-13T03:33:12.784261+0000 mon.a (mon.0) 2280 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:33:15.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:14 smithi183 bash[20824]: cluster 2024-02-13T03:33:13.394458+0000 mgr.y (mgr.24473) 483 : cluster [DBG] pgmap v695: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 688 B/s wr, 2 op/s 2024-02-13T03:33:15.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:14 smithi161 bash[27879]: cluster 2024-02-13T03:33:13.394458+0000 mgr.y (mgr.24473) 483 : cluster [DBG] pgmap v695: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 688 B/s wr, 2 op/s 2024-02-13T03:33:15.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:14 smithi161 bash[19769]: cluster 2024-02-13T03:33:13.394458+0000 mgr.y (mgr.24473) 483 : cluster [DBG] pgmap v695: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 688 B/s wr, 2 op/s 2024-02-13T03:33:16.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:33:16 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:33:17.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:16 smithi183 bash[20824]: cluster 2024-02-13T03:33:15.395285+0000 mgr.y (mgr.24473) 484 : cluster [DBG] pgmap v696: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:33:17.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:16 smithi183 bash[20824]: audit 2024-02-13T03:33:16.130587+0000 mgr.y (mgr.24473) 485 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:17.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:16 smithi161 bash[19769]: cluster 2024-02-13T03:33:15.395285+0000 mgr.y (mgr.24473) 484 : cluster [DBG] pgmap v696: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:33:17.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:16 smithi161 bash[19769]: audit 2024-02-13T03:33:16.130587+0000 mgr.y (mgr.24473) 485 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:16 smithi161 bash[27879]: cluster 2024-02-13T03:33:15.395285+0000 mgr.y (mgr.24473) 484 : cluster [DBG] pgmap v696: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:33:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:16 smithi161 bash[27879]: audit 2024-02-13T03:33:16.130587+0000 mgr.y (mgr.24473) 485 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:19.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:18 smithi183 bash[20824]: cluster 2024-02-13T03:33:17.397539+0000 mgr.y (mgr.24473) 486 : cluster [DBG] pgmap v697: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-02-13T03:33:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:18 smithi161 bash[27879]: cluster 2024-02-13T03:33:17.397539+0000 mgr.y (mgr.24473) 486 : cluster [DBG] pgmap v697: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-02-13T03:33:19.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:18 smithi161 bash[19769]: cluster 2024-02-13T03:33:17.397539+0000 mgr.y (mgr.24473) 486 : cluster [DBG] pgmap v697: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-02-13T03:33:19.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:33:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:33:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:33:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:20 smithi161 bash[27879]: cluster 2024-02-13T03:33:19.398534+0000 mgr.y (mgr.24473) 487 : cluster [DBG] pgmap v698: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 726 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:33:21.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:33:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:33:21] "GET /metrics HTTP/1.1" 200 187648 "" "Prometheus/2.33.4" 2024-02-13T03:33:21.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:20 smithi161 bash[19769]: cluster 2024-02-13T03:33:19.398534+0000 mgr.y (mgr.24473) 487 : cluster [DBG] pgmap v698: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 726 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:33:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:20 smithi183 bash[20824]: cluster 2024-02-13T03:33:19.398534+0000 mgr.y (mgr.24473) 487 : cluster [DBG] pgmap v698: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 726 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:33:23.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:22 smithi161 bash[27879]: cluster 2024-02-13T03:33:21.400801+0000 mgr.y (mgr.24473) 488 : cluster [DBG] pgmap v699: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-13T03:33:23.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:22 smithi161 bash[19769]: cluster 2024-02-13T03:33:21.400801+0000 mgr.y (mgr.24473) 488 : cluster [DBG] pgmap v699: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-13T03:33:23.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:22 smithi183 bash[20824]: cluster 2024-02-13T03:33:21.400801+0000 mgr.y (mgr.24473) 488 : cluster [DBG] pgmap v699: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-13T03:33:25.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:24 smithi161 bash[27879]: cluster 2024-02-13T03:33:23.401785+0000 mgr.y (mgr.24473) 489 : cluster [DBG] pgmap v700: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 984 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-13T03:33:25.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:24 smithi161 bash[19769]: cluster 2024-02-13T03:33:23.401785+0000 mgr.y (mgr.24473) 489 : cluster [DBG] pgmap v700: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 984 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-13T03:33:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:24 smithi183 bash[20824]: cluster 2024-02-13T03:33:23.401785+0000 mgr.y (mgr.24473) 489 : cluster [DBG] pgmap v700: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 984 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-13T03:33:26.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:33:26 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:33:27.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:26 smithi161 bash[19769]: cluster 2024-02-13T03:33:25.402649+0000 mgr.y (mgr.24473) 490 : cluster [DBG] pgmap v701: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-13T03:33:27.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:26 smithi161 bash[19769]: audit 2024-02-13T03:33:26.141960+0000 mgr.y (mgr.24473) 491 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:27.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:27 smithi161 bash[27879]: cluster 2024-02-13T03:33:25.402649+0000 mgr.y (mgr.24473) 490 : cluster [DBG] pgmap v701: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-13T03:33:27.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:27 smithi161 bash[27879]: audit 2024-02-13T03:33:26.141960+0000 mgr.y (mgr.24473) 491 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:27.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:27 smithi183 bash[20824]: cluster 2024-02-13T03:33:25.402649+0000 mgr.y (mgr.24473) 490 : cluster [DBG] pgmap v701: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-13T03:33:27.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:27 smithi183 bash[20824]: audit 2024-02-13T03:33:26.141960+0000 mgr.y (mgr.24473) 491 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:29.330 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:29 smithi183 bash[20824]: cluster 2024-02-13T03:33:27.404873+0000 mgr.y (mgr.24473) 492 : cluster [DBG] pgmap v702: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-13T03:33:29.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:29 smithi161 bash[27879]: cluster 2024-02-13T03:33:27.404873+0000 mgr.y (mgr.24473) 492 : cluster [DBG] pgmap v702: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-13T03:33:29.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:29 smithi161 bash[19769]: cluster 2024-02-13T03:33:27.404873+0000 mgr.y (mgr.24473) 492 : cluster [DBG] pgmap v702: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-13T03:33:29.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:33:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:33:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:33:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:31 smithi161 bash[27879]: cluster 2024-02-13T03:33:29.405712+0000 mgr.y (mgr.24473) 493 : cluster [DBG] pgmap v703: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:33:31.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:33:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:33:31] "GET /metrics HTTP/1.1" 200 187648 "" "Prometheus/2.33.4" 2024-02-13T03:33:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:31 smithi161 bash[19769]: cluster 2024-02-13T03:33:29.405712+0000 mgr.y (mgr.24473) 493 : cluster [DBG] pgmap v703: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:33:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:31 smithi183 bash[20824]: cluster 2024-02-13T03:33:29.405712+0000 mgr.y (mgr.24473) 493 : cluster [DBG] pgmap v703: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:33:33.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:33 smithi183 bash[20824]: cluster 2024-02-13T03:33:31.407924+0000 mgr.y (mgr.24473) 494 : cluster [DBG] pgmap v704: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:33:33.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:33 smithi183 bash[20824]: audit 2024-02-13T03:33:32.035521+0000 mon.a (mon.0) 2281 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:33.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:33 smithi183 bash[20824]: audit 2024-02-13T03:33:32.037170+0000 mon.a (mon.0) 2282 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-69"}]: dispatch 2024-02-13T03:33:33.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:33 smithi183 bash[20824]: audit 2024-02-13T03:33:32.037237+0000 mon.b (mon.1) 348 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:33.739 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:33 smithi183 bash[20824]: audit 2024-02-13T03:33:32.039002+0000 mon.b (mon.1) 349 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-69"}]: dispatch 2024-02-13T03:33:33.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:33 smithi161 bash[27879]: cluster 2024-02-13T03:33:31.407924+0000 mgr.y (mgr.24473) 494 : cluster [DBG] pgmap v704: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:33:33.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:33 smithi161 bash[27879]: audit 2024-02-13T03:33:32.035521+0000 mon.a (mon.0) 2281 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:33.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:33 smithi161 bash[27879]: audit 2024-02-13T03:33:32.037170+0000 mon.a (mon.0) 2282 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-69"}]: dispatch 2024-02-13T03:33:33.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:33 smithi161 bash[27879]: audit 2024-02-13T03:33:32.037237+0000 mon.b (mon.1) 348 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:33.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:33 smithi161 bash[27879]: audit 2024-02-13T03:33:32.039002+0000 mon.b (mon.1) 349 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-69"}]: dispatch 2024-02-13T03:33:33.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:33 smithi161 bash[19769]: cluster 2024-02-13T03:33:31.407924+0000 mgr.y (mgr.24473) 494 : cluster [DBG] pgmap v704: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:33:33.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:33 smithi161 bash[19769]: audit 2024-02-13T03:33:32.035521+0000 mon.a (mon.0) 2281 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:33.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:33 smithi161 bash[19769]: audit 2024-02-13T03:33:32.037170+0000 mon.a (mon.0) 2282 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-69"}]: dispatch 2024-02-13T03:33:33.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:33 smithi161 bash[19769]: audit 2024-02-13T03:33:32.037237+0000 mon.b (mon.1) 348 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:33.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:33 smithi161 bash[19769]: audit 2024-02-13T03:33:32.039002+0000 mon.b (mon.1) 349 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-69"}]: dispatch 2024-02-13T03:33:34.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:34 smithi183 bash[20824]: cluster 2024-02-13T03:33:33.020135+0000 mon.a (mon.0) 2283 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-02-13T03:33:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:34 smithi183 bash[20824]: cluster 2024-02-13T03:33:33.408682+0000 mgr.y (mgr.24473) 495 : cluster [DBG] pgmap v706: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-02-13T03:33:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:34 smithi183 bash[20824]: cluster 2024-02-13T03:33:34.026677+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-02-13T03:33:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:34 smithi183 bash[20824]: audit 2024-02-13T03:33:34.030449+0000 mon.b (mon.1) 350 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:34 smithi183 bash[20824]: audit 2024-02-13T03:33:34.030777+0000 mon.a (mon.0) 2285 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:34 smithi161 bash[27879]: cluster 2024-02-13T03:33:33.020135+0000 mon.a (mon.0) 2283 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-02-13T03:33:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:34 smithi161 bash[27879]: cluster 2024-02-13T03:33:33.408682+0000 mgr.y (mgr.24473) 495 : cluster [DBG] pgmap v706: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-02-13T03:33:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:34 smithi161 bash[27879]: cluster 2024-02-13T03:33:34.026677+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-02-13T03:33:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:34 smithi161 bash[27879]: audit 2024-02-13T03:33:34.030449+0000 mon.b (mon.1) 350 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:34 smithi161 bash[27879]: audit 2024-02-13T03:33:34.030777+0000 mon.a (mon.0) 2285 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:34.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:34 smithi161 bash[19769]: cluster 2024-02-13T03:33:33.020135+0000 mon.a (mon.0) 2283 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-02-13T03:33:34.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:34 smithi161 bash[19769]: cluster 2024-02-13T03:33:33.408682+0000 mgr.y (mgr.24473) 495 : cluster [DBG] pgmap v706: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-02-13T03:33:34.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:34 smithi161 bash[19769]: cluster 2024-02-13T03:33:34.026677+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-02-13T03:33:34.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:34 smithi161 bash[19769]: audit 2024-02-13T03:33:34.030449+0000 mon.b (mon.1) 350 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:34.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:34 smithi161 bash[19769]: audit 2024-02-13T03:33:34.030777+0000 mon.a (mon.0) 2285 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:36.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:36 smithi161 bash[27879]: audit 2024-02-13T03:33:35.028648+0000 mon.a (mon.0) 2286 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:36.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:36 smithi161 bash[27879]: cluster 2024-02-13T03:33:35.029033+0000 mon.a (mon.0) 2287 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-02-13T03:33:36.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:36 smithi161 bash[27879]: audit 2024-02-13T03:33:35.035442+0000 mon.b (mon.1) 351 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:36.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:36 smithi161 bash[19769]: audit 2024-02-13T03:33:35.028648+0000 mon.a (mon.0) 2286 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:36.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:36 smithi161 bash[19769]: cluster 2024-02-13T03:33:35.029033+0000 mon.a (mon.0) 2287 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-02-13T03:33:36.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:36 smithi161 bash[19769]: audit 2024-02-13T03:33:35.035442+0000 mon.b (mon.1) 351 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:36.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:33:36 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:33:36.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:36 smithi183 bash[20824]: audit 2024-02-13T03:33:35.028648+0000 mon.a (mon.0) 2286 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:36.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:36 smithi183 bash[20824]: cluster 2024-02-13T03:33:35.029033+0000 mon.a (mon.0) 2287 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-02-13T03:33:36.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:36 smithi183 bash[20824]: audit 2024-02-13T03:33:35.035442+0000 mon.b (mon.1) 351 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:37.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:37 smithi161 bash[27879]: cluster 2024-02-13T03:33:35.409483+0000 mgr.y (mgr.24473) 496 : cluster [DBG] pgmap v709: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:37.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:37 smithi161 bash[27879]: cluster 2024-02-13T03:33:36.030958+0000 mon.a (mon.0) 2288 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-02-13T03:33:37.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:37 smithi161 bash[27879]: audit 2024-02-13T03:33:36.153282+0000 mgr.y (mgr.24473) 497 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:37.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:37 smithi161 bash[19769]: cluster 2024-02-13T03:33:35.409483+0000 mgr.y (mgr.24473) 496 : cluster [DBG] pgmap v709: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:37.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:37 smithi161 bash[19769]: cluster 2024-02-13T03:33:36.030958+0000 mon.a (mon.0) 2288 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-02-13T03:33:37.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:37 smithi161 bash[19769]: audit 2024-02-13T03:33:36.153282+0000 mgr.y (mgr.24473) 497 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:37 smithi183 bash[20824]: cluster 2024-02-13T03:33:35.409483+0000 mgr.y (mgr.24473) 496 : cluster [DBG] pgmap v709: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:37.550 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:37 smithi183 bash[20824]: cluster 2024-02-13T03:33:36.030958+0000 mon.a (mon.0) 2288 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-02-13T03:33:37.550 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:37 smithi183 bash[20824]: audit 2024-02-13T03:33:36.153282+0000 mgr.y (mgr.24473) 497 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:38.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:38 smithi161 bash[27879]: cluster 2024-02-13T03:33:37.043517+0000 mon.a (mon.0) 2289 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-02-13T03:33:38.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:38 smithi161 bash[19769]: cluster 2024-02-13T03:33:37.043517+0000 mon.a (mon.0) 2289 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-02-13T03:33:38.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:38 smithi183 bash[20824]: cluster 2024-02-13T03:33:37.043517+0000 mon.a (mon.0) 2289 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-02-13T03:33:39.330 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:39 smithi183 bash[20824]: cluster 2024-02-13T03:33:37.410353+0000 mgr.y (mgr.24473) 498 : cluster [DBG] pgmap v712: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:33:39.330 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:33:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:33:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:33:39.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:39 smithi161 bash[27879]: cluster 2024-02-13T03:33:37.410353+0000 mgr.y (mgr.24473) 498 : cluster [DBG] pgmap v712: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:33:39.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:39 smithi161 bash[19769]: cluster 2024-02-13T03:33:37.410353+0000 mgr.y (mgr.24473) 498 : cluster [DBG] pgmap v712: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:33:41.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:41 smithi161 bash[27879]: cluster 2024-02-13T03:33:39.411184+0000 mgr.y (mgr.24473) 499 : cluster [DBG] pgmap v713: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 950 B/s rd, 950 B/s wr, 1 op/s 2024-02-13T03:33:41.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:33:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:33:41] "GET /metrics HTTP/1.1" 200 187745 "" "Prometheus/2.33.4" 2024-02-13T03:33:41.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:41 smithi161 bash[19769]: cluster 2024-02-13T03:33:39.411184+0000 mgr.y (mgr.24473) 499 : cluster [DBG] pgmap v713: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 950 B/s rd, 950 B/s wr, 1 op/s 2024-02-13T03:33:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:41 smithi183 bash[20824]: cluster 2024-02-13T03:33:39.411184+0000 mgr.y (mgr.24473) 499 : cluster [DBG] pgmap v713: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 950 B/s rd, 950 B/s wr, 1 op/s 2024-02-13T03:33:43.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:43 smithi161 bash[27879]: cluster 2024-02-13T03:33:41.413413+0000 mgr.y (mgr.24473) 500 : cluster [DBG] pgmap v714: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-02-13T03:33:43.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:43 smithi161 bash[19769]: cluster 2024-02-13T03:33:41.413413+0000 mgr.y (mgr.24473) 500 : cluster [DBG] pgmap v714: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-02-13T03:33:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:43 smithi183 bash[20824]: cluster 2024-02-13T03:33:41.413413+0000 mgr.y (mgr.24473) 500 : cluster [DBG] pgmap v714: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-02-13T03:33:45.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:45 smithi161 bash[27879]: cluster 2024-02-13T03:33:43.414243+0000 mgr.y (mgr.24473) 501 : cluster [DBG] pgmap v715: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-13T03:33:45.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:45 smithi161 bash[19769]: cluster 2024-02-13T03:33:43.414243+0000 mgr.y (mgr.24473) 501 : cluster [DBG] pgmap v715: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-13T03:33:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:45 smithi183 bash[20824]: cluster 2024-02-13T03:33:43.414243+0000 mgr.y (mgr.24473) 501 : cluster [DBG] pgmap v715: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-13T03:33:46.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:33:46 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:33:47.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:47 smithi161 bash[27879]: cluster 2024-02-13T03:33:45.415145+0000 mgr.y (mgr.24473) 502 : cluster [DBG] pgmap v716: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 218 B/s wr, 2 op/s 2024-02-13T03:33:47.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:47 smithi161 bash[27879]: audit 2024-02-13T03:33:46.164571+0000 mgr.y (mgr.24473) 503 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:47.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:47 smithi161 bash[19769]: cluster 2024-02-13T03:33:45.415145+0000 mgr.y (mgr.24473) 502 : cluster [DBG] pgmap v716: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 218 B/s wr, 2 op/s 2024-02-13T03:33:47.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:47 smithi161 bash[19769]: audit 2024-02-13T03:33:46.164571+0000 mgr.y (mgr.24473) 503 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:47 smithi183 bash[20824]: cluster 2024-02-13T03:33:45.415145+0000 mgr.y (mgr.24473) 502 : cluster [DBG] pgmap v716: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 218 B/s wr, 2 op/s 2024-02-13T03:33:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:47 smithi183 bash[20824]: audit 2024-02-13T03:33:46.164571+0000 mgr.y (mgr.24473) 503 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:48.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:48 smithi161 bash[27879]: audit 2024-02-13T03:33:47.104814+0000 mon.a (mon.0) 2290 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:48.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:48 smithi161 bash[27879]: audit 2024-02-13T03:33:47.106416+0000 mon.a (mon.0) 2291 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-71"}]: dispatch 2024-02-13T03:33:48.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:48 smithi161 bash[27879]: audit 2024-02-13T03:33:47.106524+0000 mon.b (mon.1) 352 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:48.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:48 smithi161 bash[27879]: audit 2024-02-13T03:33:47.108221+0000 mon.b (mon.1) 353 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-71"}]: dispatch 2024-02-13T03:33:48.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:48 smithi161 bash[19769]: audit 2024-02-13T03:33:47.104814+0000 mon.a (mon.0) 2290 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:48 smithi161 bash[19769]: audit 2024-02-13T03:33:47.106416+0000 mon.a (mon.0) 2291 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-71"}]: dispatch 2024-02-13T03:33:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:48 smithi161 bash[19769]: audit 2024-02-13T03:33:47.106524+0000 mon.b (mon.1) 352 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:48.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:48 smithi161 bash[19769]: audit 2024-02-13T03:33:47.108221+0000 mon.b (mon.1) 353 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-71"}]: dispatch 2024-02-13T03:33:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:48 smithi183 bash[20824]: audit 2024-02-13T03:33:47.104814+0000 mon.a (mon.0) 2290 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:48 smithi183 bash[20824]: audit 2024-02-13T03:33:47.106416+0000 mon.a (mon.0) 2291 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-71"}]: dispatch 2024-02-13T03:33:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:48 smithi183 bash[20824]: audit 2024-02-13T03:33:47.106524+0000 mon.b (mon.1) 352 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:48 smithi183 bash[20824]: audit 2024-02-13T03:33:47.108221+0000 mon.b (mon.1) 353 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-71"}]: dispatch 2024-02-13T03:33:49.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:49 smithi161 bash[27879]: cluster 2024-02-13T03:33:47.417392+0000 mgr.y (mgr.24473) 504 : cluster [DBG] pgmap v717: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 197 B/s wr, 2 op/s 2024-02-13T03:33:49.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:49 smithi161 bash[27879]: cluster 2024-02-13T03:33:48.098983+0000 mon.a (mon.0) 2292 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-02-13T03:33:49.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:49 smithi161 bash[19769]: cluster 2024-02-13T03:33:47.417392+0000 mgr.y (mgr.24473) 504 : cluster [DBG] pgmap v717: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 197 B/s wr, 2 op/s 2024-02-13T03:33:49.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:49 smithi161 bash[19769]: cluster 2024-02-13T03:33:48.098983+0000 mon.a (mon.0) 2292 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-02-13T03:33:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:49 smithi183 bash[20824]: cluster 2024-02-13T03:33:47.417392+0000 mgr.y (mgr.24473) 504 : cluster [DBG] pgmap v717: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 197 B/s wr, 2 op/s 2024-02-13T03:33:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:49 smithi183 bash[20824]: cluster 2024-02-13T03:33:48.098983+0000 mon.a (mon.0) 2292 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-02-13T03:33:49.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:33:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:33:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:33:50.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:50 smithi161 bash[19769]: cluster 2024-02-13T03:33:49.107280+0000 mon.a (mon.0) 2293 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-02-13T03:33:50.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:50 smithi161 bash[19769]: audit 2024-02-13T03:33:49.112330+0000 mon.a (mon.0) 2294 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:50.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:50 smithi161 bash[19769]: audit 2024-02-13T03:33:49.114156+0000 mon.b (mon.1) 354 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:50.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:50 smithi161 bash[27879]: cluster 2024-02-13T03:33:49.107280+0000 mon.a (mon.0) 2293 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-02-13T03:33:50.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:50 smithi161 bash[27879]: audit 2024-02-13T03:33:49.112330+0000 mon.a (mon.0) 2294 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:50.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:50 smithi161 bash[27879]: audit 2024-02-13T03:33:49.114156+0000 mon.b (mon.1) 354 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:50.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:50 smithi183 bash[20824]: cluster 2024-02-13T03:33:49.107280+0000 mon.a (mon.0) 2293 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-02-13T03:33:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:50 smithi183 bash[20824]: audit 2024-02-13T03:33:49.112330+0000 mon.a (mon.0) 2294 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:50 smithi183 bash[20824]: audit 2024-02-13T03:33:49.114156+0000 mon.b (mon.1) 354 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:51 smithi161 bash[27879]: cluster 2024-02-13T03:33:49.418242+0000 mgr.y (mgr.24473) 505 : cluster [DBG] pgmap v720: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:51 smithi161 bash[27879]: audit 2024-02-13T03:33:50.107592+0000 mon.a (mon.0) 2295 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:51 smithi161 bash[27879]: cluster 2024-02-13T03:33:50.107767+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-02-13T03:33:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:51 smithi161 bash[27879]: audit 2024-02-13T03:33:50.121948+0000 mon.b (mon.1) 355 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:33:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:33:51] "GET /metrics HTTP/1.1" 200 187845 "" "Prometheus/2.33.4" 2024-02-13T03:33:51.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:51 smithi161 bash[19769]: cluster 2024-02-13T03:33:49.418242+0000 mgr.y (mgr.24473) 505 : cluster [DBG] pgmap v720: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:51 smithi161 bash[19769]: audit 2024-02-13T03:33:50.107592+0000 mon.a (mon.0) 2295 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:51 smithi161 bash[19769]: cluster 2024-02-13T03:33:50.107767+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-02-13T03:33:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:51 smithi161 bash[19769]: audit 2024-02-13T03:33:50.121948+0000 mon.b (mon.1) 355 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:51 smithi183 bash[20824]: cluster 2024-02-13T03:33:49.418242+0000 mgr.y (mgr.24473) 505 : cluster [DBG] pgmap v720: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:51 smithi183 bash[20824]: audit 2024-02-13T03:33:50.107592+0000 mon.a (mon.0) 2295 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:51 smithi183 bash[20824]: cluster 2024-02-13T03:33:50.107767+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-02-13T03:33:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:51 smithi183 bash[20824]: audit 2024-02-13T03:33:50.121948+0000 mon.b (mon.1) 355 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:52.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:52 smithi161 bash[27879]: cluster 2024-02-13T03:33:51.129121+0000 mon.a (mon.0) 2297 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-02-13T03:33:52.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:52 smithi161 bash[19769]: cluster 2024-02-13T03:33:51.129121+0000 mon.a (mon.0) 2297 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-02-13T03:33:52.427 INFO:journalctl@ceph.osd.0.smithi161.stdout:Feb 13 03:33:52 smithi161 bash[31189]: debug 2024-02-13T03:33:52.164+0000 7fcf7fda9700 -1 snap_mapper.add_oid found existing snaps mapped on 62:96b2390c:test-rados-api-smithi161-61906-74::foo:24, removing 2024-02-13T03:33:52.427 INFO:journalctl@ceph.osd.1.smithi161.stdout:Feb 13 03:33:52 smithi161 bash[36121]: debug 2024-02-13T03:33:52.164+0000 7fa28a880700 -1 snap_mapper.add_oid found existing snaps mapped on 62:96b2390c:test-rados-api-smithi161-61906-74::foo:24, removing 2024-02-13T03:33:52.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:52 smithi183 bash[20824]: cluster 2024-02-13T03:33:51.129121+0000 mon.a (mon.0) 2297 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-02-13T03:33:52.487 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:33:52 smithi183 bash[24826]: debug 2024-02-13T03:33:52.165+0000 7f27d0e58700 -1 snap_mapper.add_oid found existing snaps mapped on 62:96b2390c:test-rados-api-smithi161-61906-74::foo:24, removing 2024-02-13T03:33:53.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:53 smithi161 bash[27879]: cluster 2024-02-13T03:33:51.419048+0000 mgr.y (mgr.24473) 506 : cluster [DBG] pgmap v723: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:53.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:53 smithi161 bash[27879]: cluster 2024-02-13T03:33:52.129702+0000 mon.a (mon.0) 2298 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-02-13T03:33:53.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:53 smithi161 bash[27879]: audit 2024-02-13T03:33:52.188527+0000 mon.a (mon.0) 2299 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:53.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:53 smithi161 bash[27879]: audit 2024-02-13T03:33:52.189714+0000 mon.a (mon.0) 2300 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-73"}]: dispatch 2024-02-13T03:33:53.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:53 smithi161 bash[27879]: audit 2024-02-13T03:33:52.190350+0000 mon.b (mon.1) 356 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:53.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:53 smithi161 bash[27879]: audit 2024-02-13T03:33:52.191643+0000 mon.b (mon.1) 357 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-73"}]: dispatch 2024-02-13T03:33:53.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:53 smithi161 bash[19769]: cluster 2024-02-13T03:33:51.419048+0000 mgr.y (mgr.24473) 506 : cluster [DBG] pgmap v723: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:53.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:53 smithi161 bash[19769]: cluster 2024-02-13T03:33:52.129702+0000 mon.a (mon.0) 2298 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-02-13T03:33:53.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:53 smithi161 bash[19769]: audit 2024-02-13T03:33:52.188527+0000 mon.a (mon.0) 2299 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:53.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:53 smithi161 bash[19769]: audit 2024-02-13T03:33:52.189714+0000 mon.a (mon.0) 2300 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-73"}]: dispatch 2024-02-13T03:33:53.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:53 smithi161 bash[19769]: audit 2024-02-13T03:33:52.190350+0000 mon.b (mon.1) 356 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:53.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:53 smithi161 bash[19769]: audit 2024-02-13T03:33:52.191643+0000 mon.b (mon.1) 357 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-73"}]: dispatch 2024-02-13T03:33:53.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:53 smithi183 bash[20824]: cluster 2024-02-13T03:33:51.419048+0000 mgr.y (mgr.24473) 506 : cluster [DBG] pgmap v723: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:53.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:53 smithi183 bash[20824]: cluster 2024-02-13T03:33:52.129702+0000 mon.a (mon.0) 2298 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-02-13T03:33:53.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:53 smithi183 bash[20824]: audit 2024-02-13T03:33:52.188527+0000 mon.a (mon.0) 2299 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:53.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:53 smithi183 bash[20824]: audit 2024-02-13T03:33:52.189714+0000 mon.a (mon.0) 2300 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-73"}]: dispatch 2024-02-13T03:33:53.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:53 smithi183 bash[20824]: audit 2024-02-13T03:33:52.190350+0000 mon.b (mon.1) 356 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:33:53.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:53 smithi183 bash[20824]: audit 2024-02-13T03:33:52.191643+0000 mon.b (mon.1) 357 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-73"}]: dispatch 2024-02-13T03:33:54.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:54 smithi161 bash[27879]: cluster 2024-02-13T03:33:53.152225+0000 mon.a (mon.0) 2301 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-02-13T03:33:54.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:54 smithi161 bash[19769]: cluster 2024-02-13T03:33:53.152225+0000 mon.a (mon.0) 2301 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-02-13T03:33:54.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:54 smithi183 bash[20824]: cluster 2024-02-13T03:33:53.152225+0000 mon.a (mon.0) 2301 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-02-13T03:33:55.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:55 smithi161 bash[27879]: cluster 2024-02-13T03:33:53.419830+0000 mgr.y (mgr.24473) 507 : cluster [DBG] pgmap v726: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:33:55.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:55 smithi161 bash[27879]: cluster 2024-02-13T03:33:54.162925+0000 mon.a (mon.0) 2302 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-02-13T03:33:55.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:55 smithi161 bash[27879]: audit 2024-02-13T03:33:54.168591+0000 mon.a (mon.0) 2303 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:55.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:55 smithi161 bash[27879]: audit 2024-02-13T03:33:54.170253+0000 mon.b (mon.1) 358 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:55.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:55 smithi161 bash[19769]: cluster 2024-02-13T03:33:53.419830+0000 mgr.y (mgr.24473) 507 : cluster [DBG] pgmap v726: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:33:55.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:55 smithi161 bash[19769]: cluster 2024-02-13T03:33:54.162925+0000 mon.a (mon.0) 2302 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-02-13T03:33:55.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:55 smithi161 bash[19769]: audit 2024-02-13T03:33:54.168591+0000 mon.a (mon.0) 2303 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:55.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:55 smithi161 bash[19769]: audit 2024-02-13T03:33:54.170253+0000 mon.b (mon.1) 358 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:55 smithi183 bash[20824]: cluster 2024-02-13T03:33:53.419830+0000 mgr.y (mgr.24473) 507 : cluster [DBG] pgmap v726: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:33:55.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:55 smithi183 bash[20824]: cluster 2024-02-13T03:33:54.162925+0000 mon.a (mon.0) 2302 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-02-13T03:33:55.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:55 smithi183 bash[20824]: audit 2024-02-13T03:33:54.168591+0000 mon.a (mon.0) 2303 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:55.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:55 smithi183 bash[20824]: audit 2024-02-13T03:33:54.170253+0000 mon.b (mon.1) 358 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:33:56.487 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:33:56 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:33:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:56 smithi183 bash[20824]: audit 2024-02-13T03:33:55.174528+0000 mon.a (mon.0) 2304 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:56 smithi183 bash[20824]: cluster 2024-02-13T03:33:55.174761+0000 mon.a (mon.0) 2305 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-02-13T03:33:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:56 smithi183 bash[20824]: audit 2024-02-13T03:33:55.182612+0000 mon.b (mon.1) 359 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:56 smithi161 bash[27879]: audit 2024-02-13T03:33:55.174528+0000 mon.a (mon.0) 2304 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:56 smithi161 bash[27879]: cluster 2024-02-13T03:33:55.174761+0000 mon.a (mon.0) 2305 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-02-13T03:33:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:56 smithi161 bash[27879]: audit 2024-02-13T03:33:55.182612+0000 mon.b (mon.1) 359 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:56.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:56 smithi161 bash[19769]: audit 2024-02-13T03:33:55.174528+0000 mon.a (mon.0) 2304 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:33:56.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:56 smithi161 bash[19769]: cluster 2024-02-13T03:33:55.174761+0000 mon.a (mon.0) 2305 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-02-13T03:33:56.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:56 smithi161 bash[19769]: audit 2024-02-13T03:33:55.182612+0000 mon.b (mon.1) 359 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:33:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:57 smithi183 bash[20824]: cluster 2024-02-13T03:33:55.420679+0000 mgr.y (mgr.24473) 508 : cluster [DBG] pgmap v729: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:57 smithi183 bash[20824]: audit 2024-02-13T03:33:56.171726+0000 mgr.y (mgr.24473) 509 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:57 smithi183 bash[20824]: cluster 2024-02-13T03:33:56.177621+0000 mon.a (mon.0) 2306 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-02-13T03:33:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:57 smithi161 bash[27879]: cluster 2024-02-13T03:33:55.420679+0000 mgr.y (mgr.24473) 508 : cluster [DBG] pgmap v729: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:57 smithi161 bash[27879]: audit 2024-02-13T03:33:56.171726+0000 mgr.y (mgr.24473) 509 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:57 smithi161 bash[27879]: cluster 2024-02-13T03:33:56.177621+0000 mon.a (mon.0) 2306 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-02-13T03:33:57.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:57 smithi161 bash[19769]: cluster 2024-02-13T03:33:55.420679+0000 mgr.y (mgr.24473) 508 : cluster [DBG] pgmap v729: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:33:57.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:57 smithi161 bash[19769]: audit 2024-02-13T03:33:56.171726+0000 mgr.y (mgr.24473) 509 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:33:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:57 smithi161 bash[19769]: cluster 2024-02-13T03:33:56.177621+0000 mon.a (mon.0) 2306 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-02-13T03:33:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:58 smithi183 bash[20824]: cluster 2024-02-13T03:33:57.191964+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-02-13T03:33:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:58 smithi183 bash[20824]: audit 2024-02-13T03:33:57.209787+0000 mgr.y (mgr.24473) 510 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.3"}]: dispatch 2024-02-13T03:33:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:58 smithi183 bash[20824]: audit 2024-02-13T03:33:57.211854+0000 mon.b (mon.1) 360 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.3"}]: dispatch 2024-02-13T03:33:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:58 smithi183 bash[20824]: cluster 2024-02-13T03:33:57.943581+0000 osd.0 (osd.0) 21 : cluster [DBG] 219.3 deep-scrub starts 2024-02-13T03:33:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:58 smithi183 bash[20824]: cluster 2024-02-13T03:33:57.948232+0000 osd.0 (osd.0) 22 : cluster [DBG] 219.3 deep-scrub ok 2024-02-13T03:33:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:58 smithi161 bash[27879]: cluster 2024-02-13T03:33:57.191964+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-02-13T03:33:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:58 smithi161 bash[27879]: audit 2024-02-13T03:33:57.209787+0000 mgr.y (mgr.24473) 510 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.3"}]: dispatch 2024-02-13T03:33:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:58 smithi161 bash[27879]: audit 2024-02-13T03:33:57.211854+0000 mon.b (mon.1) 360 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.3"}]: dispatch 2024-02-13T03:33:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:58 smithi161 bash[27879]: cluster 2024-02-13T03:33:57.943581+0000 osd.0 (osd.0) 21 : cluster [DBG] 219.3 deep-scrub starts 2024-02-13T03:33:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:58 smithi161 bash[27879]: cluster 2024-02-13T03:33:57.948232+0000 osd.0 (osd.0) 22 : cluster [DBG] 219.3 deep-scrub ok 2024-02-13T03:33:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:58 smithi161 bash[19769]: cluster 2024-02-13T03:33:57.191964+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-02-13T03:33:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:58 smithi161 bash[19769]: audit 2024-02-13T03:33:57.209787+0000 mgr.y (mgr.24473) 510 : audit [DBG] from='mon.? v1:172.21.15.183:0/2722870393' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.3"}]: dispatch 2024-02-13T03:33:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:58 smithi161 bash[19769]: audit 2024-02-13T03:33:57.211854+0000 mon.b (mon.1) 360 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.3"}]: dispatch 2024-02-13T03:33:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:58 smithi161 bash[19769]: cluster 2024-02-13T03:33:57.943581+0000 osd.0 (osd.0) 21 : cluster [DBG] 219.3 deep-scrub starts 2024-02-13T03:33:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:58 smithi161 bash[19769]: cluster 2024-02-13T03:33:57.948232+0000 osd.0 (osd.0) 22 : cluster [DBG] 219.3 deep-scrub ok 2024-02-13T03:33:59.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:33:59 smithi183 bash[20824]: cluster 2024-02-13T03:33:57.421538+0000 mgr.y (mgr.24473) 511 : cluster [DBG] pgmap v732: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-02-13T03:33:59.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:33:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:33:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:33:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:33:59 smithi161 bash[27879]: cluster 2024-02-13T03:33:57.421538+0000 mgr.y (mgr.24473) 511 : cluster [DBG] pgmap v732: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-02-13T03:33:59.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:33:59 smithi161 bash[19769]: cluster 2024-02-13T03:33:57.421538+0000 mgr.y (mgr.24473) 511 : cluster [DBG] pgmap v732: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-02-13T03:34:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:01 smithi161 bash[27879]: cluster 2024-02-13T03:33:59.422414+0000 mgr.y (mgr.24473) 512 : cluster [DBG] pgmap v733: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 972 B/s rd, 1.9 KiB/s wr, 3 op/s 2024-02-13T03:34:01.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:34:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:34:01] "GET /metrics HTTP/1.1" 200 187845 "" "Prometheus/2.33.4" 2024-02-13T03:34:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:01 smithi161 bash[19769]: cluster 2024-02-13T03:33:59.422414+0000 mgr.y (mgr.24473) 512 : cluster [DBG] pgmap v733: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 972 B/s rd, 1.9 KiB/s wr, 3 op/s 2024-02-13T03:34:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:01 smithi183 bash[20824]: cluster 2024-02-13T03:33:59.422414+0000 mgr.y (mgr.24473) 512 : cluster [DBG] pgmap v733: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 972 B/s rd, 1.9 KiB/s wr, 3 op/s 2024-02-13T03:34:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:03 smithi183 bash[20824]: cluster 2024-02-13T03:34:01.424661+0000 mgr.y (mgr.24473) 513 : cluster [DBG] pgmap v734: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-13T03:34:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:03 smithi161 bash[27879]: cluster 2024-02-13T03:34:01.424661+0000 mgr.y (mgr.24473) 513 : cluster [DBG] pgmap v734: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-13T03:34:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:03 smithi161 bash[19769]: cluster 2024-02-13T03:34:01.424661+0000 mgr.y (mgr.24473) 513 : cluster [DBG] pgmap v734: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-13T03:34:05.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:05 smithi183 bash[20824]: cluster 2024-02-13T03:34:03.425474+0000 mgr.y (mgr.24473) 514 : cluster [DBG] pgmap v735: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:34:05.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:05 smithi161 bash[27879]: cluster 2024-02-13T03:34:03.425474+0000 mgr.y (mgr.24473) 514 : cluster [DBG] pgmap v735: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:34:05.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:05 smithi161 bash[19769]: cluster 2024-02-13T03:34:03.425474+0000 mgr.y (mgr.24473) 514 : cluster [DBG] pgmap v735: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:34:06.487 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:34:06 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:34:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:07 smithi183 bash[20824]: cluster 2024-02-13T03:34:05.426293+0000 mgr.y (mgr.24473) 515 : cluster [DBG] pgmap v736: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 553 B/s rd, 221 B/s wr, 1 op/s 2024-02-13T03:34:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:07 smithi183 bash[20824]: audit 2024-02-13T03:34:06.173660+0000 mgr.y (mgr.24473) 516 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:07 smithi161 bash[27879]: cluster 2024-02-13T03:34:05.426293+0000 mgr.y (mgr.24473) 515 : cluster [DBG] pgmap v736: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 553 B/s rd, 221 B/s wr, 1 op/s 2024-02-13T03:34:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:07 smithi161 bash[27879]: audit 2024-02-13T03:34:06.173660+0000 mgr.y (mgr.24473) 516 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:07 smithi161 bash[19769]: cluster 2024-02-13T03:34:05.426293+0000 mgr.y (mgr.24473) 515 : cluster [DBG] pgmap v736: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 553 B/s rd, 221 B/s wr, 1 op/s 2024-02-13T03:34:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:07 smithi161 bash[19769]: audit 2024-02-13T03:34:06.173660+0000 mgr.y (mgr.24473) 516 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:09 smithi161 bash[27879]: cluster 2024-02-13T03:34:07.428507+0000 mgr.y (mgr.24473) 517 : cluster [DBG] pgmap v737: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1000 B/s rd, 200 B/s wr, 1 op/s 2024-02-13T03:34:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:09 smithi161 bash[19769]: cluster 2024-02-13T03:34:07.428507+0000 mgr.y (mgr.24473) 517 : cluster [DBG] pgmap v737: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1000 B/s rd, 200 B/s wr, 1 op/s 2024-02-13T03:34:09.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:09 smithi183 bash[20824]: cluster 2024-02-13T03:34:07.428507+0000 mgr.y (mgr.24473) 517 : cluster [DBG] pgmap v737: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1000 B/s rd, 200 B/s wr, 1 op/s 2024-02-13T03:34:09.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:34:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:34:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:34:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:11 smithi161 bash[27879]: cluster 2024-02-13T03:34:09.429403+0000 mgr.y (mgr.24473) 518 : cluster [DBG] pgmap v738: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 1 op/s 2024-02-13T03:34:11.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:34:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:34:11] "GET /metrics HTTP/1.1" 200 188006 "" "Prometheus/2.33.4" 2024-02-13T03:34:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:11 smithi161 bash[19769]: cluster 2024-02-13T03:34:09.429403+0000 mgr.y (mgr.24473) 518 : cluster [DBG] pgmap v738: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 1 op/s 2024-02-13T03:34:11.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:11 smithi183 bash[20824]: cluster 2024-02-13T03:34:09.429403+0000 mgr.y (mgr.24473) 518 : cluster [DBG] pgmap v738: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 1 op/s 2024-02-13T03:34:13.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:13 smithi161 bash[19769]: cluster 2024-02-13T03:34:11.431649+0000 mgr.y (mgr.24473) 519 : cluster [DBG] pgmap v739: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-02-13T03:34:13.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:13 smithi161 bash[19769]: audit 2024-02-13T03:34:12.793262+0000 mon.c (mon.2) 525 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:34:13.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:13 smithi161 bash[19769]: audit 2024-02-13T03:34:13.103416+0000 mon.a (mon.0) 2308 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:34:13.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:13 smithi161 bash[27879]: cluster 2024-02-13T03:34:11.431649+0000 mgr.y (mgr.24473) 519 : cluster [DBG] pgmap v739: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-02-13T03:34:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:13 smithi161 bash[27879]: audit 2024-02-13T03:34:12.793262+0000 mon.c (mon.2) 525 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:34:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:13 smithi161 bash[27879]: audit 2024-02-13T03:34:13.103416+0000 mon.a (mon.0) 2308 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:34:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:13 smithi183 bash[20824]: cluster 2024-02-13T03:34:11.431649+0000 mgr.y (mgr.24473) 519 : cluster [DBG] pgmap v739: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-02-13T03:34:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:13 smithi183 bash[20824]: audit 2024-02-13T03:34:12.793262+0000 mon.c (mon.2) 525 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:34:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:13 smithi183 bash[20824]: audit 2024-02-13T03:34:13.103416+0000 mon.a (mon.0) 2308 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:34:15.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:15 smithi161 bash[27879]: cluster 2024-02-13T03:34:13.432531+0000 mgr.y (mgr.24473) 520 : cluster [DBG] pgmap v740: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:34:15.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:15 smithi161 bash[19769]: cluster 2024-02-13T03:34:13.432531+0000 mgr.y (mgr.24473) 520 : cluster [DBG] pgmap v740: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:34:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:15 smithi183 bash[20824]: cluster 2024-02-13T03:34:13.432531+0000 mgr.y (mgr.24473) 520 : cluster [DBG] pgmap v740: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:34:16.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:34:16 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:34:17.676 INFO:journalctl@ceph.osd.0.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[31189]: debug 2024-02-13T03:34:17.235+0000 7fcf82daf700 -1 snap_mapper.add_oid found existing snaps mapped on 219:c60dfb62:test-rados-api-smithi161-61906-76::foo:2, removing 2024-02-13T03:34:17.676 INFO:journalctl@ceph.osd.1.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[36121]: debug 2024-02-13T03:34:17.235+0000 7fa28d886700 -1 snap_mapper.add_oid found existing snaps mapped on 219:c60dfb62:test-rados-api-smithi161-61906-76::foo:2, removing 2024-02-13T03:34:17.677 INFO:journalctl@ceph.osd.2.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[41241]: debug 2024-02-13T03:34:17.235+0000 7f6d118f6700 -1 snap_mapper.add_oid found existing snaps mapped on 219:c60dfb62:test-rados-api-smithi161-61906-76::foo:2, removing 2024-02-13T03:34:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[27879]: cluster 2024-02-13T03:34:15.433317+0000 mgr.y (mgr.24473) 521 : cluster [DBG] pgmap v741: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:34:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[27879]: audit 2024-02-13T03:34:16.184968+0000 mgr.y (mgr.24473) 522 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[27879]: audit 2024-02-13T03:34:17.242290+0000 mon.a (mon.0) 2309 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[27879]: audit 2024-02-13T03:34:17.243859+0000 mon.b (mon.1) 361 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[27879]: audit 2024-02-13T03:34:17.243976+0000 mon.a (mon.0) 2310 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-75"}]: dispatch 2024-02-13T03:34:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[27879]: audit 2024-02-13T03:34:17.245797+0000 mon.b (mon.1) 362 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-75"}]: dispatch 2024-02-13T03:34:17.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[19769]: cluster 2024-02-13T03:34:15.433317+0000 mgr.y (mgr.24473) 521 : cluster [DBG] pgmap v741: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:34:17.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[19769]: audit 2024-02-13T03:34:16.184968+0000 mgr.y (mgr.24473) 522 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:17.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[19769]: audit 2024-02-13T03:34:17.242290+0000 mon.a (mon.0) 2309 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:17.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[19769]: audit 2024-02-13T03:34:17.243859+0000 mon.b (mon.1) 361 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:17.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[19769]: audit 2024-02-13T03:34:17.243976+0000 mon.a (mon.0) 2310 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-75"}]: dispatch 2024-02-13T03:34:17.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:17 smithi161 bash[19769]: audit 2024-02-13T03:34:17.245797+0000 mon.b (mon.1) 362 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-75"}]: dispatch 2024-02-13T03:34:17.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:17 smithi183 bash[20824]: cluster 2024-02-13T03:34:15.433317+0000 mgr.y (mgr.24473) 521 : cluster [DBG] pgmap v741: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:34:17.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:17 smithi183 bash[20824]: audit 2024-02-13T03:34:16.184968+0000 mgr.y (mgr.24473) 522 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:17.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:17 smithi183 bash[20824]: audit 2024-02-13T03:34:17.242290+0000 mon.a (mon.0) 2309 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:17.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:17 smithi183 bash[20824]: audit 2024-02-13T03:34:17.243859+0000 mon.b (mon.1) 361 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:17.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:17 smithi183 bash[20824]: audit 2024-02-13T03:34:17.243976+0000 mon.a (mon.0) 2310 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-75"}]: dispatch 2024-02-13T03:34:17.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:17 smithi183 bash[20824]: audit 2024-02-13T03:34:17.245797+0000 mon.b (mon.1) 362 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-75"}]: dispatch 2024-02-13T03:34:19.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:19 smithi161 bash[27879]: cluster 2024-02-13T03:34:17.435692+0000 mgr.y (mgr.24473) 523 : cluster [DBG] pgmap v742: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:34:19.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:19 smithi161 bash[27879]: cluster 2024-02-13T03:34:18.269511+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-02-13T03:34:19.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:19 smithi161 bash[27879]: audit 2024-02-13T03:34:19.070270+0000 mon.a (mon.0) 2312 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:34:19.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:19 smithi161 bash[19769]: cluster 2024-02-13T03:34:17.435692+0000 mgr.y (mgr.24473) 523 : cluster [DBG] pgmap v742: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:34:19.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:19 smithi161 bash[19769]: cluster 2024-02-13T03:34:18.269511+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-02-13T03:34:19.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:19 smithi161 bash[19769]: audit 2024-02-13T03:34:19.070270+0000 mon.a (mon.0) 2312 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:34:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:19 smithi183 bash[20824]: cluster 2024-02-13T03:34:17.435692+0000 mgr.y (mgr.24473) 523 : cluster [DBG] pgmap v742: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:34:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:19 smithi183 bash[20824]: cluster 2024-02-13T03:34:18.269511+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-02-13T03:34:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:19 smithi183 bash[20824]: audit 2024-02-13T03:34:19.070270+0000 mon.a (mon.0) 2312 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:34:19.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:34:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:34:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:34:20.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:20 smithi161 bash[27879]: cluster 2024-02-13T03:34:19.275620+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-02-13T03:34:20.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:20 smithi161 bash[27879]: audit 2024-02-13T03:34:19.280878+0000 mon.b (mon.1) 363 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:20.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:20 smithi161 bash[27879]: audit 2024-02-13T03:34:19.286508+0000 mon.a (mon.0) 2314 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:20.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:20 smithi161 bash[27879]: audit 2024-02-13T03:34:19.359700+0000 mon.c (mon.2) 526 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:34:20.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:20 smithi161 bash[27879]: audit 2024-02-13T03:34:19.361468+0000 mon.c (mon.2) 527 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:34:20.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:20 smithi161 bash[27879]: audit 2024-02-13T03:34:19.372200+0000 mon.a (mon.0) 2315 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:34:20.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:20 smithi161 bash[19769]: cluster 2024-02-13T03:34:19.275620+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-02-13T03:34:20.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:20 smithi161 bash[19769]: audit 2024-02-13T03:34:19.280878+0000 mon.b (mon.1) 363 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:20.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:20 smithi161 bash[19769]: audit 2024-02-13T03:34:19.286508+0000 mon.a (mon.0) 2314 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:20.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:20 smithi161 bash[19769]: audit 2024-02-13T03:34:19.359700+0000 mon.c (mon.2) 526 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:34:20.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:20 smithi161 bash[19769]: audit 2024-02-13T03:34:19.361468+0000 mon.c (mon.2) 527 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:34:20.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:20 smithi161 bash[19769]: audit 2024-02-13T03:34:19.372200+0000 mon.a (mon.0) 2315 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:34:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:20 smithi183 bash[20824]: cluster 2024-02-13T03:34:19.275620+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-02-13T03:34:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:20 smithi183 bash[20824]: audit 2024-02-13T03:34:19.280878+0000 mon.b (mon.1) 363 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:20 smithi183 bash[20824]: audit 2024-02-13T03:34:19.286508+0000 mon.a (mon.0) 2314 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:20 smithi183 bash[20824]: audit 2024-02-13T03:34:19.359700+0000 mon.c (mon.2) 526 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:34:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:20 smithi183 bash[20824]: audit 2024-02-13T03:34:19.361468+0000 mon.c (mon.2) 527 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:34:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:20 smithi183 bash[20824]: audit 2024-02-13T03:34:19.372200+0000 mon.a (mon.0) 2315 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:34:21.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:34:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:34:21] "GET /metrics HTTP/1.1" 200 188095 "" "Prometheus/2.33.4" 2024-02-13T03:34:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:21 smithi161 bash[27879]: cluster 2024-02-13T03:34:19.436557+0000 mgr.y (mgr.24473) 524 : cluster [DBG] pgmap v745: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-13T03:34:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:21 smithi161 bash[27879]: audit 2024-02-13T03:34:20.288745+0000 mon.a (mon.0) 2316 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:34:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:21 smithi161 bash[27879]: cluster 2024-02-13T03:34:20.289135+0000 mon.a (mon.0) 2317 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-02-13T03:34:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:21 smithi161 bash[27879]: audit 2024-02-13T03:34:20.294675+0000 mon.b (mon.1) 364 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:34:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:21 smithi161 bash[27879]: audit 2024-02-13T03:34:20.305023+0000 mon.a (mon.0) 2318 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:21.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:21 smithi161 bash[27879]: audit 2024-02-13T03:34:20.306203+0000 mon.b (mon.1) 365 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:21 smithi161 bash[19769]: cluster 2024-02-13T03:34:19.436557+0000 mgr.y (mgr.24473) 524 : cluster [DBG] pgmap v745: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-13T03:34:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:21 smithi161 bash[19769]: audit 2024-02-13T03:34:20.288745+0000 mon.a (mon.0) 2316 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:34:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:21 smithi161 bash[19769]: cluster 2024-02-13T03:34:20.289135+0000 mon.a (mon.0) 2317 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-02-13T03:34:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:21 smithi161 bash[19769]: audit 2024-02-13T03:34:20.294675+0000 mon.b (mon.1) 364 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:34:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:21 smithi161 bash[19769]: audit 2024-02-13T03:34:20.305023+0000 mon.a (mon.0) 2318 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:21 smithi161 bash[19769]: audit 2024-02-13T03:34:20.306203+0000 mon.b (mon.1) 365 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:21 smithi183 bash[20824]: cluster 2024-02-13T03:34:19.436557+0000 mgr.y (mgr.24473) 524 : cluster [DBG] pgmap v745: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-13T03:34:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:21 smithi183 bash[20824]: audit 2024-02-13T03:34:20.288745+0000 mon.a (mon.0) 2316 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:34:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:21 smithi183 bash[20824]: cluster 2024-02-13T03:34:20.289135+0000 mon.a (mon.0) 2317 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-02-13T03:34:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:21 smithi183 bash[20824]: audit 2024-02-13T03:34:20.294675+0000 mon.b (mon.1) 364 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:34:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:21 smithi183 bash[20824]: audit 2024-02-13T03:34:20.305023+0000 mon.a (mon.0) 2318 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:21 smithi183 bash[20824]: audit 2024-02-13T03:34:20.306203+0000 mon.b (mon.1) 365 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:22 smithi161 bash[27879]: audit 2024-02-13T03:34:21.315254+0000 mon.a (mon.0) 2319 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:34:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:22 smithi161 bash[27879]: cluster 2024-02-13T03:34:21.315352+0000 mon.a (mon.0) 2320 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-02-13T03:34:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:22 smithi161 bash[27879]: audit 2024-02-13T03:34:21.318911+0000 mon.a (mon.0) 2321 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:22 smithi161 bash[27879]: audit 2024-02-13T03:34:21.320130+0000 mon.b (mon.1) 366 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:22.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:22 smithi161 bash[19769]: audit 2024-02-13T03:34:21.315254+0000 mon.a (mon.0) 2319 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:34:22.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:22 smithi161 bash[19769]: cluster 2024-02-13T03:34:21.315352+0000 mon.a (mon.0) 2320 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-02-13T03:34:22.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:22 smithi161 bash[19769]: audit 2024-02-13T03:34:21.318911+0000 mon.a (mon.0) 2321 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:22.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:22 smithi161 bash[19769]: audit 2024-02-13T03:34:21.320130+0000 mon.b (mon.1) 366 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:22 smithi183 bash[20824]: audit 2024-02-13T03:34:21.315254+0000 mon.a (mon.0) 2319 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:34:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:22 smithi183 bash[20824]: cluster 2024-02-13T03:34:21.315352+0000 mon.a (mon.0) 2320 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-02-13T03:34:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:22 smithi183 bash[20824]: audit 2024-02-13T03:34:21.318911+0000 mon.a (mon.0) 2321 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:22 smithi183 bash[20824]: audit 2024-02-13T03:34:21.320130+0000 mon.b (mon.1) 366 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[27879]: cluster 2024-02-13T03:34:21.437423+0000 mgr.y (mgr.24473) 525 : cluster [DBG] pgmap v748: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-13T03:34:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[27879]: audit 2024-02-13T03:34:22.321182+0000 mon.a (mon.0) 2322 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:34:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[27879]: cluster 2024-02-13T03:34:22.321478+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-02-13T03:34:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[27879]: audit 2024-02-13T03:34:22.323427+0000 mon.a (mon.0) 2324 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[27879]: audit 2024-02-13T03:34:22.324655+0000 mon.b (mon.1) 367 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[27879]: audit 2024-02-13T03:34:23.322926+0000 mon.a (mon.0) 2325 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:34:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[27879]: cluster 2024-02-13T03:34:23.323154+0000 mon.a (mon.0) 2326 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-02-13T03:34:23.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[19769]: cluster 2024-02-13T03:34:21.437423+0000 mgr.y (mgr.24473) 525 : cluster [DBG] pgmap v748: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-13T03:34:23.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[19769]: audit 2024-02-13T03:34:22.321182+0000 mon.a (mon.0) 2322 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:34:23.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[19769]: cluster 2024-02-13T03:34:22.321478+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-02-13T03:34:23.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[19769]: audit 2024-02-13T03:34:22.323427+0000 mon.a (mon.0) 2324 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:23.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[19769]: audit 2024-02-13T03:34:22.324655+0000 mon.b (mon.1) 367 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:23.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[19769]: audit 2024-02-13T03:34:23.322926+0000 mon.a (mon.0) 2325 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:34:23.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:23 smithi161 bash[19769]: cluster 2024-02-13T03:34:23.323154+0000 mon.a (mon.0) 2326 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-02-13T03:34:23.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:23 smithi183 bash[20824]: cluster 2024-02-13T03:34:21.437423+0000 mgr.y (mgr.24473) 525 : cluster [DBG] pgmap v748: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-13T03:34:23.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:23 smithi183 bash[20824]: audit 2024-02-13T03:34:22.321182+0000 mon.a (mon.0) 2322 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:34:23.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:23 smithi183 bash[20824]: cluster 2024-02-13T03:34:22.321478+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-02-13T03:34:23.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:23 smithi183 bash[20824]: audit 2024-02-13T03:34:22.323427+0000 mon.a (mon.0) 2324 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:23.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:23 smithi183 bash[20824]: audit 2024-02-13T03:34:22.324655+0000 mon.b (mon.1) 367 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:23.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:23 smithi183 bash[20824]: audit 2024-02-13T03:34:23.322926+0000 mon.a (mon.0) 2325 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:34:23.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:23 smithi183 bash[20824]: cluster 2024-02-13T03:34:23.323154+0000 mon.a (mon.0) 2326 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-02-13T03:34:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:24 smithi161 bash[27879]: audit 2024-02-13T03:34:23.326291+0000 mon.a (mon.0) 2327 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:24 smithi161 bash[27879]: audit 2024-02-13T03:34:23.326471+0000 mon.b (mon.1) 368 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:24 smithi161 bash[27879]: audit 2024-02-13T03:34:24.325347+0000 mon.a (mon.0) 2328 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:24 smithi161 bash[27879]: cluster 2024-02-13T03:34:24.325628+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-02-13T03:34:24.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:24 smithi161 bash[19769]: audit 2024-02-13T03:34:23.326291+0000 mon.a (mon.0) 2327 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:24.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:24 smithi161 bash[19769]: audit 2024-02-13T03:34:23.326471+0000 mon.b (mon.1) 368 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:24 smithi161 bash[19769]: audit 2024-02-13T03:34:24.325347+0000 mon.a (mon.0) 2328 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:24 smithi161 bash[19769]: cluster 2024-02-13T03:34:24.325628+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-02-13T03:34:24.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:24 smithi183 bash[20824]: audit 2024-02-13T03:34:23.326291+0000 mon.a (mon.0) 2327 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:24.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:24 smithi183 bash[20824]: audit 2024-02-13T03:34:23.326471+0000 mon.b (mon.1) 368 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:24.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:24 smithi183 bash[20824]: audit 2024-02-13T03:34:24.325347+0000 mon.a (mon.0) 2328 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:24.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:24 smithi183 bash[20824]: cluster 2024-02-13T03:34:24.325628+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-02-13T03:34:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:25 smithi161 bash[27879]: cluster 2024-02-13T03:34:23.438232+0000 mgr.y (mgr.24473) 526 : cluster [DBG] pgmap v751: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-13T03:34:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:25 smithi161 bash[27879]: audit 2024-02-13T03:34:24.350433+0000 mon.a (mon.0) 2330 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-13T03:34:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:25 smithi161 bash[27879]: audit 2024-02-13T03:34:24.352229+0000 mon.b (mon.1) 369 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-13T03:34:25.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:25 smithi161 bash[19769]: cluster 2024-02-13T03:34:23.438232+0000 mgr.y (mgr.24473) 526 : cluster [DBG] pgmap v751: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-13T03:34:25.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:25 smithi161 bash[19769]: audit 2024-02-13T03:34:24.350433+0000 mon.a (mon.0) 2330 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-13T03:34:25.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:25 smithi161 bash[19769]: audit 2024-02-13T03:34:24.352229+0000 mon.b (mon.1) 369 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-13T03:34:25.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:25 smithi183 bash[20824]: cluster 2024-02-13T03:34:23.438232+0000 mgr.y (mgr.24473) 526 : cluster [DBG] pgmap v751: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-13T03:34:25.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:25 smithi183 bash[20824]: audit 2024-02-13T03:34:24.350433+0000 mon.a (mon.0) 2330 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-13T03:34:25.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:25 smithi183 bash[20824]: audit 2024-02-13T03:34:24.352229+0000 mon.b (mon.1) 369 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-13T03:34:26.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:34:26 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:34:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:26 smithi183 bash[20824]: audit 2024-02-13T03:34:25.351880+0000 mon.a (mon.0) 2331 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-13T03:34:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:26 smithi183 bash[20824]: cluster 2024-02-13T03:34:25.351978+0000 mon.a (mon.0) 2332 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-02-13T03:34:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:26 smithi183 bash[20824]: audit 2024-02-13T03:34:25.369172+0000 mon.a (mon.0) 2333 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-13T03:34:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:26 smithi183 bash[20824]: audit 2024-02-13T03:34:25.370674+0000 mon.b (mon.1) 370 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-13T03:34:26.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:26 smithi161 bash[27879]: audit 2024-02-13T03:34:25.351880+0000 mon.a (mon.0) 2331 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-13T03:34:26.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:26 smithi161 bash[27879]: cluster 2024-02-13T03:34:25.351978+0000 mon.a (mon.0) 2332 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-02-13T03:34:26.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:26 smithi161 bash[27879]: audit 2024-02-13T03:34:25.369172+0000 mon.a (mon.0) 2333 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-13T03:34:26.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:26 smithi161 bash[27879]: audit 2024-02-13T03:34:25.370674+0000 mon.b (mon.1) 370 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-13T03:34:26.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:26 smithi161 bash[19769]: audit 2024-02-13T03:34:25.351880+0000 mon.a (mon.0) 2331 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-13T03:34:26.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:26 smithi161 bash[19769]: cluster 2024-02-13T03:34:25.351978+0000 mon.a (mon.0) 2332 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-02-13T03:34:26.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:26 smithi161 bash[19769]: audit 2024-02-13T03:34:25.369172+0000 mon.a (mon.0) 2333 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-13T03:34:26.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:26 smithi161 bash[19769]: audit 2024-02-13T03:34:25.370674+0000 mon.b (mon.1) 370 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-13T03:34:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:27 smithi161 bash[27879]: cluster 2024-02-13T03:34:25.439119+0000 mgr.y (mgr.24473) 527 : cluster [DBG] pgmap v754: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:34:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:27 smithi161 bash[27879]: audit 2024-02-13T03:34:26.188825+0000 mgr.y (mgr.24473) 528 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:27 smithi161 bash[27879]: audit 2024-02-13T03:34:26.369051+0000 mon.a (mon.0) 2334 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-13T03:34:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:27 smithi161 bash[27879]: cluster 2024-02-13T03:34:26.369239+0000 mon.a (mon.0) 2335 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-02-13T03:34:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:27 smithi161 bash[27879]: audit 2024-02-13T03:34:26.384036+0000 mon.a (mon.0) 2336 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:27 smithi161 bash[27879]: audit 2024-02-13T03:34:26.385683+0000 mon.b (mon.1) 371 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:27.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:27 smithi161 bash[19769]: cluster 2024-02-13T03:34:25.439119+0000 mgr.y (mgr.24473) 527 : cluster [DBG] pgmap v754: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:34:27.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:27 smithi161 bash[19769]: audit 2024-02-13T03:34:26.188825+0000 mgr.y (mgr.24473) 528 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:27.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:27 smithi161 bash[19769]: audit 2024-02-13T03:34:26.369051+0000 mon.a (mon.0) 2334 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-13T03:34:27.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:27 smithi161 bash[19769]: cluster 2024-02-13T03:34:26.369239+0000 mon.a (mon.0) 2335 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-02-13T03:34:27.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:27 smithi161 bash[19769]: audit 2024-02-13T03:34:26.384036+0000 mon.a (mon.0) 2336 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:27.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:27 smithi161 bash[19769]: audit 2024-02-13T03:34:26.385683+0000 mon.b (mon.1) 371 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:27 smithi183 bash[20824]: cluster 2024-02-13T03:34:25.439119+0000 mgr.y (mgr.24473) 527 : cluster [DBG] pgmap v754: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:34:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:27 smithi183 bash[20824]: audit 2024-02-13T03:34:26.188825+0000 mgr.y (mgr.24473) 528 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:27 smithi183 bash[20824]: audit 2024-02-13T03:34:26.369051+0000 mon.a (mon.0) 2334 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-13T03:34:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:27 smithi183 bash[20824]: cluster 2024-02-13T03:34:26.369239+0000 mon.a (mon.0) 2335 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-02-13T03:34:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:27 smithi183 bash[20824]: audit 2024-02-13T03:34:26.384036+0000 mon.a (mon.0) 2336 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:27 smithi183 bash[20824]: audit 2024-02-13T03:34:26.385683+0000 mon.b (mon.1) 371 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:28 smithi161 bash[27879]: audit 2024-02-13T03:34:27.377845+0000 mon.a (mon.0) 2337 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:28.753 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:28 smithi161 bash[27879]: cluster 2024-02-13T03:34:27.377933+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-02-13T03:34:28.753 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:28 smithi161 bash[27879]: audit 2024-02-13T03:34:27.418231+0000 mon.a (mon.0) 2339 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:28.753 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:28 smithi161 bash[27879]: audit 2024-02-13T03:34:27.419433+0000 mon.a (mon.0) 2340 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-77"}]: dispatch 2024-02-13T03:34:28.753 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:28 smithi161 bash[27879]: audit 2024-02-13T03:34:27.420000+0000 mon.b (mon.1) 372 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:28.753 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:28 smithi161 bash[27879]: audit 2024-02-13T03:34:27.421359+0000 mon.b (mon.1) 373 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-77"}]: dispatch 2024-02-13T03:34:28.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:28 smithi161 bash[19769]: audit 2024-02-13T03:34:27.377845+0000 mon.a (mon.0) 2337 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:28.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:28 smithi161 bash[19769]: cluster 2024-02-13T03:34:27.377933+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-02-13T03:34:28.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:28 smithi161 bash[19769]: audit 2024-02-13T03:34:27.418231+0000 mon.a (mon.0) 2339 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:28.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:28 smithi161 bash[19769]: audit 2024-02-13T03:34:27.419433+0000 mon.a (mon.0) 2340 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-77"}]: dispatch 2024-02-13T03:34:28.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:28 smithi161 bash[19769]: audit 2024-02-13T03:34:27.420000+0000 mon.b (mon.1) 372 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:28.754 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:28 smithi161 bash[19769]: audit 2024-02-13T03:34:27.421359+0000 mon.b (mon.1) 373 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-77"}]: dispatch 2024-02-13T03:34:28.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:28 smithi183 bash[20824]: audit 2024-02-13T03:34:27.377845+0000 mon.a (mon.0) 2337 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:28.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:28 smithi183 bash[20824]: cluster 2024-02-13T03:34:27.377933+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-02-13T03:34:28.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:28 smithi183 bash[20824]: audit 2024-02-13T03:34:27.418231+0000 mon.a (mon.0) 2339 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:28.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:28 smithi183 bash[20824]: audit 2024-02-13T03:34:27.419433+0000 mon.a (mon.0) 2340 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-77"}]: dispatch 2024-02-13T03:34:28.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:28 smithi183 bash[20824]: audit 2024-02-13T03:34:27.420000+0000 mon.b (mon.1) 372 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:28.755 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:28 smithi183 bash[20824]: audit 2024-02-13T03:34:27.421359+0000 mon.b (mon.1) 373 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-77"}]: dispatch 2024-02-13T03:34:29.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:29 smithi161 bash[27879]: cluster 2024-02-13T03:34:27.439939+0000 mgr.y (mgr.24473) 529 : cluster [DBG] pgmap v757: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-02-13T03:34:29.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:29 smithi161 bash[27879]: cluster 2024-02-13T03:34:28.388908+0000 mon.a (mon.0) 2341 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-02-13T03:34:29.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:29 smithi161 bash[19769]: cluster 2024-02-13T03:34:27.439939+0000 mgr.y (mgr.24473) 529 : cluster [DBG] pgmap v757: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-02-13T03:34:29.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:29 smithi161 bash[19769]: cluster 2024-02-13T03:34:28.388908+0000 mon.a (mon.0) 2341 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-02-13T03:34:29.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:29 smithi183 bash[20824]: cluster 2024-02-13T03:34:27.439939+0000 mgr.y (mgr.24473) 529 : cluster [DBG] pgmap v757: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-02-13T03:34:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:29 smithi183 bash[20824]: cluster 2024-02-13T03:34:28.388908+0000 mon.a (mon.0) 2341 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-02-13T03:34:29.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:34:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:34:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:34:30.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:30 smithi161 bash[27879]: cluster 2024-02-13T03:34:29.393375+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-02-13T03:34:30.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:30 smithi161 bash[27879]: audit 2024-02-13T03:34:29.398397+0000 mon.a (mon.0) 2343 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:30.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:30 smithi161 bash[27879]: audit 2024-02-13T03:34:29.399602+0000 mon.b (mon.1) 374 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:30.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:30 smithi161 bash[27879]: cluster 2024-02-13T03:34:29.440793+0000 mgr.y (mgr.24473) 530 : cluster [DBG] pgmap v760: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 7.0 KiB/s wr, 21 op/s 2024-02-13T03:34:30.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:30 smithi161 bash[19769]: cluster 2024-02-13T03:34:29.393375+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-02-13T03:34:30.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:30 smithi161 bash[19769]: audit 2024-02-13T03:34:29.398397+0000 mon.a (mon.0) 2343 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:30.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:30 smithi161 bash[19769]: audit 2024-02-13T03:34:29.399602+0000 mon.b (mon.1) 374 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:30.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:30 smithi161 bash[19769]: cluster 2024-02-13T03:34:29.440793+0000 mgr.y (mgr.24473) 530 : cluster [DBG] pgmap v760: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 7.0 KiB/s wr, 21 op/s 2024-02-13T03:34:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:30 smithi183 bash[20824]: cluster 2024-02-13T03:34:29.393375+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-02-13T03:34:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:30 smithi183 bash[20824]: audit 2024-02-13T03:34:29.398397+0000 mon.a (mon.0) 2343 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:30 smithi183 bash[20824]: audit 2024-02-13T03:34:29.399602+0000 mon.b (mon.1) 374 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:30 smithi183 bash[20824]: cluster 2024-02-13T03:34:29.440793+0000 mgr.y (mgr.24473) 530 : cluster [DBG] pgmap v760: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 7.0 KiB/s wr, 21 op/s 2024-02-13T03:34:31.406 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:34:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:34:31] "GET /metrics HTTP/1.1" 200 188095 "" "Prometheus/2.33.4" 2024-02-13T03:34:31.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:31 smithi161 bash[27879]: audit 2024-02-13T03:34:30.397664+0000 mon.a (mon.0) 2344 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:34:31.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:31 smithi161 bash[27879]: cluster 2024-02-13T03:34:30.397870+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-02-13T03:34:31.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:31 smithi161 bash[27879]: audit 2024-02-13T03:34:30.413024+0000 mon.b (mon.1) 375 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:34:31.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:31 smithi161 bash[27879]: audit 2024-02-13T03:34:30.415814+0000 mon.a (mon.0) 2346 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:31.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:31 smithi161 bash[27879]: audit 2024-02-13T03:34:30.416176+0000 mon.b (mon.1) 376 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:31.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:31 smithi161 bash[19769]: audit 2024-02-13T03:34:30.397664+0000 mon.a (mon.0) 2344 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:34:31.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:31 smithi161 bash[19769]: cluster 2024-02-13T03:34:30.397870+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-02-13T03:34:31.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:31 smithi161 bash[19769]: audit 2024-02-13T03:34:30.413024+0000 mon.b (mon.1) 375 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:34:31.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:31 smithi161 bash[19769]: audit 2024-02-13T03:34:30.415814+0000 mon.a (mon.0) 2346 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:31.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:31 smithi161 bash[19769]: audit 2024-02-13T03:34:30.416176+0000 mon.b (mon.1) 376 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:31 smithi183 bash[20824]: audit 2024-02-13T03:34:30.397664+0000 mon.a (mon.0) 2344 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:34:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:31 smithi183 bash[20824]: cluster 2024-02-13T03:34:30.397870+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-02-13T03:34:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:31 smithi183 bash[20824]: audit 2024-02-13T03:34:30.413024+0000 mon.b (mon.1) 375 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:34:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:31 smithi183 bash[20824]: audit 2024-02-13T03:34:30.415814+0000 mon.a (mon.0) 2346 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:31 smithi183 bash[20824]: audit 2024-02-13T03:34:30.416176+0000 mon.b (mon.1) 376 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:32 smithi183 bash[20824]: audit 2024-02-13T03:34:31.418996+0000 mon.a (mon.0) 2347 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:34:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:32 smithi183 bash[20824]: cluster 2024-02-13T03:34:31.419081+0000 mon.a (mon.0) 2348 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-02-13T03:34:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:32 smithi183 bash[20824]: audit 2024-02-13T03:34:31.424173+0000 mon.a (mon.0) 2349 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:32 smithi183 bash[20824]: audit 2024-02-13T03:34:31.425829+0000 mon.b (mon.1) 377 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:32 smithi183 bash[20824]: cluster 2024-02-13T03:34:31.441332+0000 mgr.y (mgr.24473) 531 : cluster [DBG] pgmap v763: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-02-13T03:34:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:32 smithi161 bash[27879]: audit 2024-02-13T03:34:31.418996+0000 mon.a (mon.0) 2347 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:34:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:32 smithi161 bash[27879]: cluster 2024-02-13T03:34:31.419081+0000 mon.a (mon.0) 2348 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-02-13T03:34:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:32 smithi161 bash[27879]: audit 2024-02-13T03:34:31.424173+0000 mon.a (mon.0) 2349 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:32 smithi161 bash[27879]: audit 2024-02-13T03:34:31.425829+0000 mon.b (mon.1) 377 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:32 smithi161 bash[27879]: cluster 2024-02-13T03:34:31.441332+0000 mgr.y (mgr.24473) 531 : cluster [DBG] pgmap v763: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-02-13T03:34:32.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:32 smithi161 bash[19769]: audit 2024-02-13T03:34:31.418996+0000 mon.a (mon.0) 2347 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:34:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:32 smithi161 bash[19769]: cluster 2024-02-13T03:34:31.419081+0000 mon.a (mon.0) 2348 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-02-13T03:34:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:32 smithi161 bash[19769]: audit 2024-02-13T03:34:31.424173+0000 mon.a (mon.0) 2349 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:32 smithi161 bash[19769]: audit 2024-02-13T03:34:31.425829+0000 mon.b (mon.1) 377 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:32 smithi161 bash[19769]: cluster 2024-02-13T03:34:31.441332+0000 mgr.y (mgr.24473) 531 : cluster [DBG] pgmap v763: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-02-13T03:34:33.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:33 smithi183 bash[20824]: audit 2024-02-13T03:34:32.430924+0000 mon.a (mon.0) 2350 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:34:33.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:33 smithi183 bash[20824]: cluster 2024-02-13T03:34:32.431076+0000 mon.a (mon.0) 2351 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-02-13T03:34:33.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:33 smithi183 bash[20824]: audit 2024-02-13T03:34:32.436787+0000 mon.a (mon.0) 2352 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:33.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:33 smithi183 bash[20824]: audit 2024-02-13T03:34:32.438652+0000 mon.b (mon.1) 378 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:33.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:33 smithi161 bash[27879]: audit 2024-02-13T03:34:32.430924+0000 mon.a (mon.0) 2350 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:34:33.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:33 smithi161 bash[27879]: cluster 2024-02-13T03:34:32.431076+0000 mon.a (mon.0) 2351 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-02-13T03:34:33.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:33 smithi161 bash[27879]: audit 2024-02-13T03:34:32.436787+0000 mon.a (mon.0) 2352 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:33.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:33 smithi161 bash[27879]: audit 2024-02-13T03:34:32.438652+0000 mon.b (mon.1) 378 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:33.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:33 smithi161 bash[19769]: audit 2024-02-13T03:34:32.430924+0000 mon.a (mon.0) 2350 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:34:33.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:33 smithi161 bash[19769]: cluster 2024-02-13T03:34:32.431076+0000 mon.a (mon.0) 2351 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-02-13T03:34:33.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:33 smithi161 bash[19769]: audit 2024-02-13T03:34:32.436787+0000 mon.a (mon.0) 2352 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:33.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:33 smithi161 bash[19769]: audit 2024-02-13T03:34:32.438652+0000 mon.b (mon.1) 378 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:34 smithi183 bash[20824]: audit 2024-02-13T03:34:33.432818+0000 mon.a (mon.0) 2353 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:34:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:34 smithi183 bash[20824]: cluster 2024-02-13T03:34:33.432959+0000 mon.a (mon.0) 2354 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-02-13T03:34:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:34 smithi183 bash[20824]: audit 2024-02-13T03:34:33.439070+0000 mon.b (mon.1) 379 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:34 smithi183 bash[20824]: audit 2024-02-13T03:34:33.441505+0000 mon.a (mon.0) 2355 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:34 smithi183 bash[20824]: cluster 2024-02-13T03:34:33.441944+0000 mgr.y (mgr.24473) 532 : cluster [DBG] pgmap v766: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-02-13T03:34:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:34 smithi161 bash[27879]: audit 2024-02-13T03:34:33.432818+0000 mon.a (mon.0) 2353 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:34:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:34 smithi161 bash[27879]: cluster 2024-02-13T03:34:33.432959+0000 mon.a (mon.0) 2354 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-02-13T03:34:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:34 smithi161 bash[27879]: audit 2024-02-13T03:34:33.439070+0000 mon.b (mon.1) 379 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:34 smithi161 bash[27879]: audit 2024-02-13T03:34:33.441505+0000 mon.a (mon.0) 2355 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:34 smithi161 bash[27879]: cluster 2024-02-13T03:34:33.441944+0000 mgr.y (mgr.24473) 532 : cluster [DBG] pgmap v766: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-02-13T03:34:34.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:34 smithi161 bash[19769]: audit 2024-02-13T03:34:33.432818+0000 mon.a (mon.0) 2353 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:34:34.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:34 smithi161 bash[19769]: cluster 2024-02-13T03:34:33.432959+0000 mon.a (mon.0) 2354 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-02-13T03:34:34.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:34 smithi161 bash[19769]: audit 2024-02-13T03:34:33.439070+0000 mon.b (mon.1) 379 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:34.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:34 smithi161 bash[19769]: audit 2024-02-13T03:34:33.441505+0000 mon.a (mon.0) 2355 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:34.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:34 smithi161 bash[19769]: cluster 2024-02-13T03:34:33.441944+0000 mgr.y (mgr.24473) 532 : cluster [DBG] pgmap v766: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-02-13T03:34:35.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:35 smithi183 bash[20824]: audit 2024-02-13T03:34:34.451834+0000 mon.a (mon.0) 2356 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:35 smithi183 bash[20824]: cluster 2024-02-13T03:34:34.451945+0000 mon.a (mon.0) 2357 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-02-13T03:34:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:35 smithi161 bash[27879]: audit 2024-02-13T03:34:34.451834+0000 mon.a (mon.0) 2356 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:35 smithi161 bash[27879]: cluster 2024-02-13T03:34:34.451945+0000 mon.a (mon.0) 2357 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-02-13T03:34:35.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:35 smithi161 bash[19769]: audit 2024-02-13T03:34:34.451834+0000 mon.a (mon.0) 2356 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:35.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:35 smithi161 bash[19769]: cluster 2024-02-13T03:34:34.451945+0000 mon.a (mon.0) 2357 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-02-13T03:34:36.465 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:34:36 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:34:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:36 smithi183 bash[20824]: cluster 2024-02-13T03:34:35.442724+0000 mgr.y (mgr.24473) 533 : cluster [DBG] pgmap v768: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:34:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:36 smithi183 bash[20824]: cluster 2024-02-13T03:34:35.454026+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-02-13T03:34:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:36 smithi183 bash[20824]: audit 2024-02-13T03:34:36.167872+0000 mon.c (mon.2) 528 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.11", "id": [0, 1]}]: dispatch 2024-02-13T03:34:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:36 smithi183 bash[20824]: audit 2024-02-13T03:34:36.168399+0000 mon.c (mon.2) 529 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1e", "id": [3, 1]}]: dispatch 2024-02-13T03:34:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:36 smithi183 bash[20824]: audit 2024-02-13T03:34:36.168744+0000 mon.a (mon.0) 2359 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.11", "id": [0, 1]}]: dispatch 2024-02-13T03:34:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:36 smithi183 bash[20824]: audit 2024-02-13T03:34:36.169577+0000 mon.a (mon.0) 2360 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1e", "id": [3, 1]}]: dispatch 2024-02-13T03:34:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:36 smithi183 bash[20824]: audit 2024-02-13T03:34:36.199899+0000 mgr.y (mgr.24473) 534 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[27879]: cluster 2024-02-13T03:34:35.442724+0000 mgr.y (mgr.24473) 533 : cluster [DBG] pgmap v768: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:34:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[27879]: cluster 2024-02-13T03:34:35.454026+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-02-13T03:34:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[27879]: audit 2024-02-13T03:34:36.167872+0000 mon.c (mon.2) 528 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.11", "id": [0, 1]}]: dispatch 2024-02-13T03:34:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[27879]: audit 2024-02-13T03:34:36.168399+0000 mon.c (mon.2) 529 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1e", "id": [3, 1]}]: dispatch 2024-02-13T03:34:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[27879]: audit 2024-02-13T03:34:36.168744+0000 mon.a (mon.0) 2359 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.11", "id": [0, 1]}]: dispatch 2024-02-13T03:34:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[27879]: audit 2024-02-13T03:34:36.169577+0000 mon.a (mon.0) 2360 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1e", "id": [3, 1]}]: dispatch 2024-02-13T03:34:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[27879]: audit 2024-02-13T03:34:36.199899+0000 mgr.y (mgr.24473) 534 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[19769]: cluster 2024-02-13T03:34:35.442724+0000 mgr.y (mgr.24473) 533 : cluster [DBG] pgmap v768: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:34:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[19769]: cluster 2024-02-13T03:34:35.454026+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-02-13T03:34:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[19769]: audit 2024-02-13T03:34:36.167872+0000 mon.c (mon.2) 528 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.11", "id": [0, 1]}]: dispatch 2024-02-13T03:34:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[19769]: audit 2024-02-13T03:34:36.168399+0000 mon.c (mon.2) 529 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1e", "id": [3, 1]}]: dispatch 2024-02-13T03:34:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[19769]: audit 2024-02-13T03:34:36.168744+0000 mon.a (mon.0) 2359 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.11", "id": [0, 1]}]: dispatch 2024-02-13T03:34:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[19769]: audit 2024-02-13T03:34:36.169577+0000 mon.a (mon.0) 2360 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1e", "id": [3, 1]}]: dispatch 2024-02-13T03:34:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:36 smithi161 bash[19769]: audit 2024-02-13T03:34:36.199899+0000 mgr.y (mgr.24473) 534 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:37 smithi161 bash[27879]: audit 2024-02-13T03:34:36.479717+0000 mon.a (mon.0) 2361 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.11", "id": [0, 1]}]': finished 2024-02-13T03:34:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:37 smithi161 bash[27879]: audit 2024-02-13T03:34:36.479876+0000 mon.a (mon.0) 2362 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1e", "id": [3, 1]}]': finished 2024-02-13T03:34:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:37 smithi161 bash[27879]: cluster 2024-02-13T03:34:36.479929+0000 mon.a (mon.0) 2363 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-02-13T03:34:37.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:37 smithi161 bash[19769]: audit 2024-02-13T03:34:36.479717+0000 mon.a (mon.0) 2361 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.11", "id": [0, 1]}]': finished 2024-02-13T03:34:37.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:37 smithi161 bash[19769]: audit 2024-02-13T03:34:36.479876+0000 mon.a (mon.0) 2362 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1e", "id": [3, 1]}]': finished 2024-02-13T03:34:37.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:37 smithi161 bash[19769]: cluster 2024-02-13T03:34:36.479929+0000 mon.a (mon.0) 2363 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-02-13T03:34:37.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:37 smithi183 bash[20824]: audit 2024-02-13T03:34:36.479717+0000 mon.a (mon.0) 2361 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.11", "id": [0, 1]}]': finished 2024-02-13T03:34:37.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:37 smithi183 bash[20824]: audit 2024-02-13T03:34:36.479876+0000 mon.a (mon.0) 2362 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1e", "id": [3, 1]}]': finished 2024-02-13T03:34:37.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:37 smithi183 bash[20824]: cluster 2024-02-13T03:34:36.479929+0000 mon.a (mon.0) 2363 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-02-13T03:34:38.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:38 smithi161 bash[27879]: cluster 2024-02-13T03:34:37.443537+0000 mgr.y (mgr.24473) 535 : cluster [DBG] pgmap v771: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-13T03:34:38.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:38 smithi161 bash[27879]: cluster 2024-02-13T03:34:37.482125+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-02-13T03:34:38.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:38 smithi161 bash[27879]: audit 2024-02-13T03:34:37.808292+0000 mon.a (mon.0) 2365 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:38.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:38 smithi161 bash[27879]: audit 2024-02-13T03:34:37.809476+0000 mon.a (mon.0) 2366 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-79"}]: dispatch 2024-02-13T03:34:38.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:38 smithi161 bash[27879]: audit 2024-02-13T03:34:37.810088+0000 mon.b (mon.1) 380 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:38.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:38 smithi161 bash[27879]: audit 2024-02-13T03:34:37.811306+0000 mon.b (mon.1) 381 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-79"}]: dispatch 2024-02-13T03:34:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:38 smithi161 bash[19769]: cluster 2024-02-13T03:34:37.443537+0000 mgr.y (mgr.24473) 535 : cluster [DBG] pgmap v771: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-13T03:34:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:38 smithi161 bash[19769]: cluster 2024-02-13T03:34:37.482125+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-02-13T03:34:38.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:38 smithi161 bash[19769]: audit 2024-02-13T03:34:37.808292+0000 mon.a (mon.0) 2365 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:38.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:38 smithi161 bash[19769]: audit 2024-02-13T03:34:37.809476+0000 mon.a (mon.0) 2366 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-79"}]: dispatch 2024-02-13T03:34:38.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:38 smithi161 bash[19769]: audit 2024-02-13T03:34:37.810088+0000 mon.b (mon.1) 380 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:38.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:38 smithi161 bash[19769]: audit 2024-02-13T03:34:37.811306+0000 mon.b (mon.1) 381 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-79"}]: dispatch 2024-02-13T03:34:38.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:38 smithi183 bash[20824]: cluster 2024-02-13T03:34:37.443537+0000 mgr.y (mgr.24473) 535 : cluster [DBG] pgmap v771: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-13T03:34:38.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:38 smithi183 bash[20824]: cluster 2024-02-13T03:34:37.482125+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-02-13T03:34:38.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:38 smithi183 bash[20824]: audit 2024-02-13T03:34:37.808292+0000 mon.a (mon.0) 2365 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:38.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:38 smithi183 bash[20824]: audit 2024-02-13T03:34:37.809476+0000 mon.a (mon.0) 2366 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-79"}]: dispatch 2024-02-13T03:34:38.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:38 smithi183 bash[20824]: audit 2024-02-13T03:34:37.810088+0000 mon.b (mon.1) 380 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:38.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:38 smithi183 bash[20824]: audit 2024-02-13T03:34:37.811306+0000 mon.b (mon.1) 381 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-79"}]: dispatch 2024-02-13T03:34:39.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:39 smithi183 bash[20824]: cluster 2024-02-13T03:34:38.501007+0000 mon.a (mon.0) 2367 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-02-13T03:34:39.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:34:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:34:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:34:39.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:39 smithi161 bash[27879]: cluster 2024-02-13T03:34:38.501007+0000 mon.a (mon.0) 2367 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-02-13T03:34:39.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:39 smithi161 bash[19769]: cluster 2024-02-13T03:34:38.501007+0000 mon.a (mon.0) 2367 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-02-13T03:34:40.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:40 smithi161 bash[27879]: cluster 2024-02-13T03:34:39.445424+0000 mgr.y (mgr.24473) 536 : cluster [DBG] pgmap v774: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:34:40.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:40 smithi161 bash[27879]: cluster 2024-02-13T03:34:39.512530+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-02-13T03:34:40.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:40 smithi161 bash[27879]: audit 2024-02-13T03:34:39.515994+0000 mon.a (mon.0) 2369 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:40.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:40 smithi161 bash[27879]: audit 2024-02-13T03:34:39.516761+0000 mon.b (mon.1) 382 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:40.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:40 smithi161 bash[19769]: cluster 2024-02-13T03:34:39.445424+0000 mgr.y (mgr.24473) 536 : cluster [DBG] pgmap v774: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:34:40.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:40 smithi161 bash[19769]: cluster 2024-02-13T03:34:39.512530+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-02-13T03:34:40.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:40 smithi161 bash[19769]: audit 2024-02-13T03:34:39.515994+0000 mon.a (mon.0) 2369 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:40.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:40 smithi161 bash[19769]: audit 2024-02-13T03:34:39.516761+0000 mon.b (mon.1) 382 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:40.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:40 smithi183 bash[20824]: cluster 2024-02-13T03:34:39.445424+0000 mgr.y (mgr.24473) 536 : cluster [DBG] pgmap v774: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:34:40.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:40 smithi183 bash[20824]: cluster 2024-02-13T03:34:39.512530+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-02-13T03:34:40.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:40 smithi183 bash[20824]: audit 2024-02-13T03:34:39.515994+0000 mon.a (mon.0) 2369 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:40.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:40 smithi183 bash[20824]: audit 2024-02-13T03:34:39.516761+0000 mon.b (mon.1) 382 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:41.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:34:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:34:41] "GET /metrics HTTP/1.1" 200 188237 "" "Prometheus/2.33.4" 2024-02-13T03:34:41.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:41 smithi161 bash[27879]: audit 2024-02-13T03:34:40.516370+0000 mon.a (mon.0) 2370 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:34:41.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:41 smithi161 bash[27879]: cluster 2024-02-13T03:34:40.516773+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-02-13T03:34:41.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:41 smithi161 bash[27879]: audit 2024-02-13T03:34:40.526346+0000 mon.b (mon.1) 383 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:34:41.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:41 smithi161 bash[27879]: audit 2024-02-13T03:34:40.540224+0000 mon.a (mon.0) 2372 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:41 smithi161 bash[27879]: audit 2024-02-13T03:34:40.541667+0000 mon.b (mon.1) 384 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:41.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:41 smithi161 bash[19769]: audit 2024-02-13T03:34:40.516370+0000 mon.a (mon.0) 2370 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:34:41.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:41 smithi161 bash[19769]: cluster 2024-02-13T03:34:40.516773+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-02-13T03:34:41.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:41 smithi161 bash[19769]: audit 2024-02-13T03:34:40.526346+0000 mon.b (mon.1) 383 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:34:41.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:41 smithi161 bash[19769]: audit 2024-02-13T03:34:40.540224+0000 mon.a (mon.0) 2372 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:41.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:41 smithi161 bash[19769]: audit 2024-02-13T03:34:40.541667+0000 mon.b (mon.1) 384 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:41.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:41 smithi183 bash[20824]: audit 2024-02-13T03:34:40.516370+0000 mon.a (mon.0) 2370 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:34:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:41 smithi183 bash[20824]: cluster 2024-02-13T03:34:40.516773+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-02-13T03:34:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:41 smithi183 bash[20824]: audit 2024-02-13T03:34:40.526346+0000 mon.b (mon.1) 383 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:34:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:41 smithi183 bash[20824]: audit 2024-02-13T03:34:40.540224+0000 mon.a (mon.0) 2372 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:41.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:41 smithi183 bash[20824]: audit 2024-02-13T03:34:40.541667+0000 mon.b (mon.1) 384 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:42.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:42 smithi161 bash[27879]: cluster 2024-02-13T03:34:41.447578+0000 mgr.y (mgr.24473) 537 : cluster [DBG] pgmap v777: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-13T03:34:42.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:42 smithi161 bash[27879]: audit 2024-02-13T03:34:41.539292+0000 mon.a (mon.0) 2373 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:34:42.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:42 smithi161 bash[27879]: cluster 2024-02-13T03:34:41.539499+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-02-13T03:34:42.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:42 smithi161 bash[27879]: audit 2024-02-13T03:34:41.544743+0000 mon.b (mon.1) 385 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:42.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:42 smithi161 bash[27879]: audit 2024-02-13T03:34:41.548776+0000 mon.a (mon.0) 2375 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:42.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:42 smithi161 bash[19769]: cluster 2024-02-13T03:34:41.447578+0000 mgr.y (mgr.24473) 537 : cluster [DBG] pgmap v777: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-13T03:34:42.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:42 smithi161 bash[19769]: audit 2024-02-13T03:34:41.539292+0000 mon.a (mon.0) 2373 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:34:42.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:42 smithi161 bash[19769]: cluster 2024-02-13T03:34:41.539499+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-02-13T03:34:42.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:42 smithi161 bash[19769]: audit 2024-02-13T03:34:41.544743+0000 mon.b (mon.1) 385 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:42.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:42 smithi161 bash[19769]: audit 2024-02-13T03:34:41.548776+0000 mon.a (mon.0) 2375 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:42 smithi183 bash[20824]: cluster 2024-02-13T03:34:41.447578+0000 mgr.y (mgr.24473) 537 : cluster [DBG] pgmap v777: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-02-13T03:34:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:42 smithi183 bash[20824]: audit 2024-02-13T03:34:41.539292+0000 mon.a (mon.0) 2373 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:34:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:42 smithi183 bash[20824]: cluster 2024-02-13T03:34:41.539499+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-02-13T03:34:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:42 smithi183 bash[20824]: audit 2024-02-13T03:34:41.544743+0000 mon.b (mon.1) 385 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:42.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:42 smithi183 bash[20824]: audit 2024-02-13T03:34:41.548776+0000 mon.a (mon.0) 2375 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:43.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:43 smithi161 bash[27879]: audit 2024-02-13T03:34:42.556479+0000 mon.a (mon.0) 2376 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:34:43.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:43 smithi161 bash[27879]: cluster 2024-02-13T03:34:42.556589+0000 mon.a (mon.0) 2377 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-02-13T03:34:43.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:43 smithi161 bash[27879]: audit 2024-02-13T03:34:42.563294+0000 mon.a (mon.0) 2378 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:43.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:43 smithi161 bash[27879]: audit 2024-02-13T03:34:42.565119+0000 mon.b (mon.1) 386 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:43.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:43 smithi161 bash[19769]: audit 2024-02-13T03:34:42.556479+0000 mon.a (mon.0) 2376 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:34:43.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:43 smithi161 bash[19769]: cluster 2024-02-13T03:34:42.556589+0000 mon.a (mon.0) 2377 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-02-13T03:34:43.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:43 smithi161 bash[19769]: audit 2024-02-13T03:34:42.563294+0000 mon.a (mon.0) 2378 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:43.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:43 smithi161 bash[19769]: audit 2024-02-13T03:34:42.565119+0000 mon.b (mon.1) 386 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:43.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:43 smithi183 bash[20824]: audit 2024-02-13T03:34:42.556479+0000 mon.a (mon.0) 2376 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_tier","val": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:34:43.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:43 smithi183 bash[20824]: cluster 2024-02-13T03:34:42.556589+0000 mon.a (mon.0) 2377 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-02-13T03:34:43.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:43 smithi183 bash[20824]: audit 2024-02-13T03:34:42.563294+0000 mon.a (mon.0) 2378 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:43.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:43 smithi183 bash[20824]: audit 2024-02-13T03:34:42.565119+0000 mon.b (mon.1) 386 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:44.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:44 smithi161 bash[27879]: cluster 2024-02-13T03:34:43.448438+0000 mgr.y (mgr.24473) 538 : cluster [DBG] pgmap v780: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-13T03:34:44.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:44 smithi161 bash[27879]: audit 2024-02-13T03:34:43.562873+0000 mon.a (mon.0) 2379 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:34:44.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:44 smithi161 bash[27879]: cluster 2024-02-13T03:34:43.563028+0000 mon.a (mon.0) 2380 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-02-13T03:34:44.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:44 smithi161 bash[27879]: audit 2024-02-13T03:34:43.568432+0000 mon.b (mon.1) 387 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:44.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:44 smithi161 bash[27879]: audit 2024-02-13T03:34:43.571804+0000 mon.a (mon.0) 2381 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:44.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:44 smithi161 bash[19769]: cluster 2024-02-13T03:34:43.448438+0000 mgr.y (mgr.24473) 538 : cluster [DBG] pgmap v780: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-13T03:34:44.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:44 smithi161 bash[19769]: audit 2024-02-13T03:34:43.562873+0000 mon.a (mon.0) 2379 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:34:44.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:44 smithi161 bash[19769]: cluster 2024-02-13T03:34:43.563028+0000 mon.a (mon.0) 2380 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-02-13T03:34:44.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:44 smithi161 bash[19769]: audit 2024-02-13T03:34:43.568432+0000 mon.b (mon.1) 387 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:44.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:44 smithi161 bash[19769]: audit 2024-02-13T03:34:43.571804+0000 mon.a (mon.0) 2381 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:44.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:44 smithi183 bash[20824]: cluster 2024-02-13T03:34:43.448438+0000 mgr.y (mgr.24473) 538 : cluster [DBG] pgmap v780: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-13T03:34:44.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:44 smithi183 bash[20824]: audit 2024-02-13T03:34:43.562873+0000 mon.a (mon.0) 2379 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:34:44.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:44 smithi183 bash[20824]: cluster 2024-02-13T03:34:43.563028+0000 mon.a (mon.0) 2380 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-02-13T03:34:44.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:44 smithi183 bash[20824]: audit 2024-02-13T03:34:43.568432+0000 mon.b (mon.1) 387 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:44.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:44 smithi183 bash[20824]: audit 2024-02-13T03:34:43.571804+0000 mon.a (mon.0) 2381 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:45.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:45 smithi161 bash[27879]: audit 2024-02-13T03:34:44.577248+0000 mon.a (mon.0) 2382 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:45.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:45 smithi161 bash[27879]: cluster 2024-02-13T03:34:44.577506+0000 mon.a (mon.0) 2383 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-02-13T03:34:45.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:45 smithi161 bash[19769]: audit 2024-02-13T03:34:44.577248+0000 mon.a (mon.0) 2382 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:45.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:45 smithi161 bash[19769]: cluster 2024-02-13T03:34:44.577506+0000 mon.a (mon.0) 2383 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-02-13T03:34:45.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:45 smithi183 bash[20824]: audit 2024-02-13T03:34:44.577248+0000 mon.a (mon.0) 2382 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:45.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:45 smithi183 bash[20824]: cluster 2024-02-13T03:34:44.577506+0000 mon.a (mon.0) 2383 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-02-13T03:34:46.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:34:46 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:34:46.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:46 smithi161 bash[19769]: cluster 2024-02-13T03:34:45.449238+0000 mgr.y (mgr.24473) 539 : cluster [DBG] pgmap v783: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:34:46.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:46 smithi161 bash[19769]: cluster 2024-02-13T03:34:45.594751+0000 mon.a (mon.0) 2384 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-02-13T03:34:46.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:46 smithi161 bash[19769]: audit 2024-02-13T03:34:46.211260+0000 mgr.y (mgr.24473) 540 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:46.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:46 smithi161 bash[27879]: cluster 2024-02-13T03:34:45.449238+0000 mgr.y (mgr.24473) 539 : cluster [DBG] pgmap v783: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:34:46.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:46 smithi161 bash[27879]: cluster 2024-02-13T03:34:45.594751+0000 mon.a (mon.0) 2384 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-02-13T03:34:46.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:46 smithi161 bash[27879]: audit 2024-02-13T03:34:46.211260+0000 mgr.y (mgr.24473) 540 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:46.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:46 smithi183 bash[20824]: cluster 2024-02-13T03:34:45.449238+0000 mgr.y (mgr.24473) 539 : cluster [DBG] pgmap v783: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:34:46.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:46 smithi183 bash[20824]: cluster 2024-02-13T03:34:45.594751+0000 mon.a (mon.0) 2384 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-02-13T03:34:46.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:46 smithi183 bash[20824]: audit 2024-02-13T03:34:46.211260+0000 mgr.y (mgr.24473) 540 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:47.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:47 smithi161 bash[27879]: cluster 2024-02-13T03:34:46.599364+0000 mon.a (mon.0) 2385 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-02-13T03:34:47.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:47 smithi161 bash[27879]: audit 2024-02-13T03:34:46.647335+0000 mon.a (mon.0) 2386 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:47.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:47 smithi161 bash[27879]: audit 2024-02-13T03:34:46.648566+0000 mon.a (mon.0) 2387 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-81"}]: dispatch 2024-02-13T03:34:47.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:47 smithi161 bash[27879]: audit 2024-02-13T03:34:46.649062+0000 mon.b (mon.1) 388 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:47.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:47 smithi161 bash[27879]: audit 2024-02-13T03:34:46.650496+0000 mon.b (mon.1) 389 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-81"}]: dispatch 2024-02-13T03:34:47.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:47 smithi161 bash[19769]: cluster 2024-02-13T03:34:46.599364+0000 mon.a (mon.0) 2385 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-02-13T03:34:47.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:47 smithi161 bash[19769]: audit 2024-02-13T03:34:46.647335+0000 mon.a (mon.0) 2386 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:47.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:47 smithi161 bash[19769]: audit 2024-02-13T03:34:46.648566+0000 mon.a (mon.0) 2387 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-81"}]: dispatch 2024-02-13T03:34:47.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:47 smithi161 bash[19769]: audit 2024-02-13T03:34:46.649062+0000 mon.b (mon.1) 388 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:47.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:47 smithi161 bash[19769]: audit 2024-02-13T03:34:46.650496+0000 mon.b (mon.1) 389 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-81"}]: dispatch 2024-02-13T03:34:47.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:47 smithi183 bash[20824]: cluster 2024-02-13T03:34:46.599364+0000 mon.a (mon.0) 2385 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-02-13T03:34:47.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:47 smithi183 bash[20824]: audit 2024-02-13T03:34:46.647335+0000 mon.a (mon.0) 2386 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:47.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:47 smithi183 bash[20824]: audit 2024-02-13T03:34:46.648566+0000 mon.a (mon.0) 2387 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-81"}]: dispatch 2024-02-13T03:34:47.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:47 smithi183 bash[20824]: audit 2024-02-13T03:34:46.649062+0000 mon.b (mon.1) 388 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:47.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:47 smithi183 bash[20824]: audit 2024-02-13T03:34:46.650496+0000 mon.b (mon.1) 389 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-81"}]: dispatch 2024-02-13T03:34:48.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:48 smithi161 bash[19769]: cluster 2024-02-13T03:34:47.450143+0000 mgr.y (mgr.24473) 541 : cluster [DBG] pgmap v786: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-13T03:34:48.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:48 smithi161 bash[19769]: cluster 2024-02-13T03:34:47.626327+0000 mon.a (mon.0) 2388 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-02-13T03:34:48.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:48 smithi161 bash[27879]: cluster 2024-02-13T03:34:47.450143+0000 mgr.y (mgr.24473) 541 : cluster [DBG] pgmap v786: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-13T03:34:48.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:48 smithi161 bash[27879]: cluster 2024-02-13T03:34:47.626327+0000 mon.a (mon.0) 2388 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-02-13T03:34:48.945 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:48 smithi183 bash[20824]: cluster 2024-02-13T03:34:47.450143+0000 mgr.y (mgr.24473) 541 : cluster [DBG] pgmap v786: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-13T03:34:48.946 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:48 smithi183 bash[20824]: cluster 2024-02-13T03:34:47.626327+0000 mon.a (mon.0) 2388 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-02-13T03:34:49.634 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:34:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:34:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:34:49.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:49 smithi161 bash[19769]: cluster 2024-02-13T03:34:48.635168+0000 mon.a (mon.0) 2389 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-02-13T03:34:49.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:49 smithi161 bash[19769]: audit 2024-02-13T03:34:48.642306+0000 mon.a (mon.0) 2390 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:49.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:49 smithi161 bash[19769]: audit 2024-02-13T03:34:48.642790+0000 mon.b (mon.1) 390 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:49.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:49 smithi161 bash[27879]: cluster 2024-02-13T03:34:48.635168+0000 mon.a (mon.0) 2389 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-02-13T03:34:49.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:49 smithi161 bash[27879]: audit 2024-02-13T03:34:48.642306+0000 mon.a (mon.0) 2390 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:49.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:49 smithi161 bash[27879]: audit 2024-02-13T03:34:48.642790+0000 mon.b (mon.1) 390 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:49.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:49 smithi183 bash[20824]: cluster 2024-02-13T03:34:48.635168+0000 mon.a (mon.0) 2389 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-02-13T03:34:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:49 smithi183 bash[20824]: audit 2024-02-13T03:34:48.642306+0000 mon.a (mon.0) 2390 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:49.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:49 smithi183 bash[20824]: audit 2024-02-13T03:34:48.642790+0000 mon.b (mon.1) 390 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:50.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:50 smithi161 bash[19769]: cluster 2024-02-13T03:34:49.451013+0000 mgr.y (mgr.24473) 542 : cluster [DBG] pgmap v789: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:34:50.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:50 smithi161 bash[19769]: audit 2024-02-13T03:34:49.645019+0000 mon.a (mon.0) 2391 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:34:50.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:50 smithi161 bash[19769]: cluster 2024-02-13T03:34:49.645376+0000 mon.a (mon.0) 2392 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-02-13T03:34:50.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:50 smithi161 bash[19769]: audit 2024-02-13T03:34:49.652300+0000 mon.b (mon.1) 391 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:34:50.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:50 smithi161 bash[27879]: cluster 2024-02-13T03:34:49.451013+0000 mgr.y (mgr.24473) 542 : cluster [DBG] pgmap v789: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:34:50.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:50 smithi161 bash[27879]: audit 2024-02-13T03:34:49.645019+0000 mon.a (mon.0) 2391 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:34:50.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:50 smithi161 bash[27879]: cluster 2024-02-13T03:34:49.645376+0000 mon.a (mon.0) 2392 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-02-13T03:34:50.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:50 smithi161 bash[27879]: audit 2024-02-13T03:34:49.652300+0000 mon.b (mon.1) 391 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:34:50.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:50 smithi183 bash[20824]: cluster 2024-02-13T03:34:49.451013+0000 mgr.y (mgr.24473) 542 : cluster [DBG] pgmap v789: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:34:50.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:50 smithi183 bash[20824]: audit 2024-02-13T03:34:49.645019+0000 mon.a (mon.0) 2391 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:34:50.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:50 smithi183 bash[20824]: cluster 2024-02-13T03:34:49.645376+0000 mon.a (mon.0) 2392 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-02-13T03:34:50.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:50 smithi183 bash[20824]: audit 2024-02-13T03:34:49.652300+0000 mon.b (mon.1) 391 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:34:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:34:51] "GET /metrics HTTP/1.1" 200 188358 "" "Prometheus/2.33.4" 2024-02-13T03:34:51.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[27879]: cluster 2024-02-13T03:34:50.646547+0000 mon.a (mon.0) 2393 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-02-13T03:34:51.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[27879]: audit 2024-02-13T03:34:50.660779+0000 mon.b (mon.1) 392 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:51.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[27879]: audit 2024-02-13T03:34:50.668247+0000 mon.a (mon.0) 2394 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:51.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[27879]: audit 2024-02-13T03:34:51.654207+0000 mon.a (mon.0) 2395 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:34:51.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[27879]: cluster 2024-02-13T03:34:51.654322+0000 mon.a (mon.0) 2396 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-02-13T03:34:51.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[27879]: audit 2024-02-13T03:34:51.658088+0000 mon.a (mon.0) 2397 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_tier","val": "test-rados-api-smithi161-61906-85-test-flush"}]: dispatch 2024-02-13T03:34:51.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[27879]: audit 2024-02-13T03:34:51.658686+0000 mon.b (mon.1) 393 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_tier","val": "test-rados-api-smithi161-61906-85-test-flush"}]: dispatch 2024-02-13T03:34:51.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[19769]: cluster 2024-02-13T03:34:50.646547+0000 mon.a (mon.0) 2393 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-02-13T03:34:51.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[19769]: audit 2024-02-13T03:34:50.660779+0000 mon.b (mon.1) 392 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:51.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[19769]: audit 2024-02-13T03:34:50.668247+0000 mon.a (mon.0) 2394 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:51.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[19769]: audit 2024-02-13T03:34:51.654207+0000 mon.a (mon.0) 2395 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:34:51.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[19769]: cluster 2024-02-13T03:34:51.654322+0000 mon.a (mon.0) 2396 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-02-13T03:34:51.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[19769]: audit 2024-02-13T03:34:51.658088+0000 mon.a (mon.0) 2397 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_tier","val": "test-rados-api-smithi161-61906-85-test-flush"}]: dispatch 2024-02-13T03:34:51.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:51 smithi161 bash[19769]: audit 2024-02-13T03:34:51.658686+0000 mon.b (mon.1) 393 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_tier","val": "test-rados-api-smithi161-61906-85-test-flush"}]: dispatch 2024-02-13T03:34:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:51 smithi183 bash[20824]: cluster 2024-02-13T03:34:50.646547+0000 mon.a (mon.0) 2393 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-02-13T03:34:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:51 smithi183 bash[20824]: audit 2024-02-13T03:34:50.660779+0000 mon.b (mon.1) 392 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:51 smithi183 bash[20824]: audit 2024-02-13T03:34:50.668247+0000 mon.a (mon.0) 2394 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:34:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:51 smithi183 bash[20824]: audit 2024-02-13T03:34:51.654207+0000 mon.a (mon.0) 2395 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:34:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:51 smithi183 bash[20824]: cluster 2024-02-13T03:34:51.654322+0000 mon.a (mon.0) 2396 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-02-13T03:34:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:51 smithi183 bash[20824]: audit 2024-02-13T03:34:51.658088+0000 mon.a (mon.0) 2397 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_tier","val": "test-rados-api-smithi161-61906-85-test-flush"}]: dispatch 2024-02-13T03:34:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:51 smithi183 bash[20824]: audit 2024-02-13T03:34:51.658686+0000 mon.b (mon.1) 393 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_tier","val": "test-rados-api-smithi161-61906-85-test-flush"}]: dispatch 2024-02-13T03:34:52.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:52 smithi161 bash[27879]: cluster 2024-02-13T03:34:51.453339+0000 mgr.y (mgr.24473) 543 : cluster [DBG] pgmap v792: 260 pgs: 30 creating+peering, 2 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-02-13T03:34:52.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:52 smithi161 bash[27879]: audit 2024-02-13T03:34:52.659587+0000 mon.a (mon.0) 2398 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_tier","val": "test-rados-api-smithi161-61906-85-test-flush"}]': finished 2024-02-13T03:34:52.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:52 smithi161 bash[27879]: cluster 2024-02-13T03:34:52.659775+0000 mon.a (mon.0) 2399 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-02-13T03:34:52.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:52 smithi161 bash[27879]: audit 2024-02-13T03:34:52.661898+0000 mon.a (mon.0) 2400 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:52.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:52 smithi161 bash[27879]: audit 2024-02-13T03:34:52.663405+0000 mon.b (mon.1) 394 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:52.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:52 smithi161 bash[19769]: cluster 2024-02-13T03:34:51.453339+0000 mgr.y (mgr.24473) 543 : cluster [DBG] pgmap v792: 260 pgs: 30 creating+peering, 2 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-02-13T03:34:52.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:52 smithi161 bash[19769]: audit 2024-02-13T03:34:52.659587+0000 mon.a (mon.0) 2398 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_tier","val": "test-rados-api-smithi161-61906-85-test-flush"}]': finished 2024-02-13T03:34:52.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:52 smithi161 bash[19769]: cluster 2024-02-13T03:34:52.659775+0000 mon.a (mon.0) 2399 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-02-13T03:34:52.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:52 smithi161 bash[19769]: audit 2024-02-13T03:34:52.661898+0000 mon.a (mon.0) 2400 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:52.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:52 smithi161 bash[19769]: audit 2024-02-13T03:34:52.663405+0000 mon.b (mon.1) 394 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:52.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:52 smithi183 bash[20824]: cluster 2024-02-13T03:34:51.453339+0000 mgr.y (mgr.24473) 543 : cluster [DBG] pgmap v792: 260 pgs: 30 creating+peering, 2 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-02-13T03:34:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:52 smithi183 bash[20824]: audit 2024-02-13T03:34:52.659587+0000 mon.a (mon.0) 2398 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_tier","val": "test-rados-api-smithi161-61906-85-test-flush"}]': finished 2024-02-13T03:34:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:52 smithi183 bash[20824]: cluster 2024-02-13T03:34:52.659775+0000 mon.a (mon.0) 2399 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-02-13T03:34:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:52 smithi183 bash[20824]: audit 2024-02-13T03:34:52.661898+0000 mon.a (mon.0) 2400 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:52 smithi183 bash[20824]: audit 2024-02-13T03:34:52.663405+0000 mon.b (mon.1) 394 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:34:54.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:54 smithi161 bash[19769]: cluster 2024-02-13T03:34:53.454270+0000 mgr.y (mgr.24473) 544 : cluster [DBG] pgmap v795: 260 pgs: 30 creating+peering, 2 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-13T03:34:54.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:54 smithi161 bash[19769]: audit 2024-02-13T03:34:53.656794+0000 mon.a (mon.0) 2401 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:34:54.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:54 smithi161 bash[19769]: cluster 2024-02-13T03:34:53.656964+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-02-13T03:34:54.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:54 smithi161 bash[19769]: audit 2024-02-13T03:34:53.659585+0000 mon.a (mon.0) 2403 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:54.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:54 smithi161 bash[19769]: audit 2024-02-13T03:34:53.661079+0000 mon.b (mon.1) 395 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:54 smithi161 bash[27879]: cluster 2024-02-13T03:34:53.454270+0000 mgr.y (mgr.24473) 544 : cluster [DBG] pgmap v795: 260 pgs: 30 creating+peering, 2 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-13T03:34:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:54 smithi161 bash[27879]: audit 2024-02-13T03:34:53.656794+0000 mon.a (mon.0) 2401 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:34:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:54 smithi161 bash[27879]: cluster 2024-02-13T03:34:53.656964+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-02-13T03:34:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:54 smithi161 bash[27879]: audit 2024-02-13T03:34:53.659585+0000 mon.a (mon.0) 2403 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:54 smithi161 bash[27879]: audit 2024-02-13T03:34:53.661079+0000 mon.b (mon.1) 395 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:54 smithi183 bash[20824]: cluster 2024-02-13T03:34:53.454270+0000 mgr.y (mgr.24473) 544 : cluster [DBG] pgmap v795: 260 pgs: 30 creating+peering, 2 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-13T03:34:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:54 smithi183 bash[20824]: audit 2024-02-13T03:34:53.656794+0000 mon.a (mon.0) 2401 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:34:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:54 smithi183 bash[20824]: cluster 2024-02-13T03:34:53.656964+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-02-13T03:34:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:54 smithi183 bash[20824]: audit 2024-02-13T03:34:53.659585+0000 mon.a (mon.0) 2403 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:54 smithi183 bash[20824]: audit 2024-02-13T03:34:53.661079+0000 mon.b (mon.1) 395 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:34:55.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:55 smithi183 bash[20824]: audit 2024-02-13T03:34:54.666508+0000 mon.a (mon.0) 2404 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:55.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:55 smithi183 bash[20824]: cluster 2024-02-13T03:34:54.666593+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-02-13T03:34:56.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:55 smithi161 bash[27879]: audit 2024-02-13T03:34:54.666508+0000 mon.a (mon.0) 2404 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:56.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:55 smithi161 bash[27879]: cluster 2024-02-13T03:34:54.666593+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-02-13T03:34:56.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:55 smithi161 bash[19769]: audit 2024-02-13T03:34:54.666508+0000 mon.a (mon.0) 2404 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:34:56.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:55 smithi161 bash[19769]: cluster 2024-02-13T03:34:54.666593+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-02-13T03:34:56.487 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:34:56 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:34:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:56 smithi183 bash[20824]: cluster 2024-02-13T03:34:55.455123+0000 mgr.y (mgr.24473) 545 : cluster [DBG] pgmap v798: 260 pgs: 30 creating+peering, 2 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:34:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:56 smithi183 bash[20824]: cluster 2024-02-13T03:34:55.670972+0000 mon.a (mon.0) 2406 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-02-13T03:34:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:56 smithi183 bash[20824]: audit 2024-02-13T03:34:55.713192+0000 mon.a (mon.0) 2407 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:56 smithi183 bash[20824]: audit 2024-02-13T03:34:55.714448+0000 mon.a (mon.0) 2408 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-83"}]: dispatch 2024-02-13T03:34:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:56 smithi183 bash[20824]: audit 2024-02-13T03:34:55.715053+0000 mon.b (mon.1) 396 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:56 smithi183 bash[20824]: audit 2024-02-13T03:34:55.716395+0000 mon.b (mon.1) 397 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-83"}]: dispatch 2024-02-13T03:34:56.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:56 smithi183 bash[20824]: audit 2024-02-13T03:34:56.222597+0000 mgr.y (mgr.24473) 546 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:57.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[19769]: cluster 2024-02-13T03:34:55.455123+0000 mgr.y (mgr.24473) 545 : cluster [DBG] pgmap v798: 260 pgs: 30 creating+peering, 2 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:34:57.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[19769]: cluster 2024-02-13T03:34:55.670972+0000 mon.a (mon.0) 2406 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-02-13T03:34:57.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[19769]: audit 2024-02-13T03:34:55.713192+0000 mon.a (mon.0) 2407 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:57.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[19769]: audit 2024-02-13T03:34:55.714448+0000 mon.a (mon.0) 2408 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-83"}]: dispatch 2024-02-13T03:34:57.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[19769]: audit 2024-02-13T03:34:55.715053+0000 mon.b (mon.1) 396 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:57.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[19769]: audit 2024-02-13T03:34:55.716395+0000 mon.b (mon.1) 397 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-83"}]: dispatch 2024-02-13T03:34:57.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[19769]: audit 2024-02-13T03:34:56.222597+0000 mgr.y (mgr.24473) 546 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:57.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[27879]: cluster 2024-02-13T03:34:55.455123+0000 mgr.y (mgr.24473) 545 : cluster [DBG] pgmap v798: 260 pgs: 30 creating+peering, 2 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:34:57.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[27879]: cluster 2024-02-13T03:34:55.670972+0000 mon.a (mon.0) 2406 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-02-13T03:34:57.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[27879]: audit 2024-02-13T03:34:55.713192+0000 mon.a (mon.0) 2407 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:57.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[27879]: audit 2024-02-13T03:34:55.714448+0000 mon.a (mon.0) 2408 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-83"}]: dispatch 2024-02-13T03:34:57.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[27879]: audit 2024-02-13T03:34:55.715053+0000 mon.b (mon.1) 396 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:34:57.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[27879]: audit 2024-02-13T03:34:55.716395+0000 mon.b (mon.1) 397 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-83"}]: dispatch 2024-02-13T03:34:57.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:56 smithi161 bash[27879]: audit 2024-02-13T03:34:56.222597+0000 mgr.y (mgr.24473) 546 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:34:57.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:57 smithi183 bash[20824]: cluster 2024-02-13T03:34:56.698092+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-02-13T03:34:58.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:57 smithi161 bash[27879]: cluster 2024-02-13T03:34:56.698092+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-02-13T03:34:58.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:57 smithi161 bash[19769]: cluster 2024-02-13T03:34:56.698092+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-02-13T03:34:58.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:58 smithi183 bash[20824]: cluster 2024-02-13T03:34:57.455929+0000 mgr.y (mgr.24473) 547 : cluster [DBG] pgmap v801: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:34:58.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:58 smithi183 bash[20824]: cluster 2024-02-13T03:34:57.715867+0000 mon.a (mon.0) 2410 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-02-13T03:34:58.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:58 smithi183 bash[20824]: audit 2024-02-13T03:34:57.720950+0000 mon.a (mon.0) 2411 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:58.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:58 smithi183 bash[20824]: audit 2024-02-13T03:34:57.722375+0000 mon.b (mon.1) 398 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:59.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:58 smithi161 bash[27879]: cluster 2024-02-13T03:34:57.455929+0000 mgr.y (mgr.24473) 547 : cluster [DBG] pgmap v801: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:34:59.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:58 smithi161 bash[27879]: cluster 2024-02-13T03:34:57.715867+0000 mon.a (mon.0) 2410 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-02-13T03:34:59.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:58 smithi161 bash[27879]: audit 2024-02-13T03:34:57.720950+0000 mon.a (mon.0) 2411 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:59.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:58 smithi161 bash[27879]: audit 2024-02-13T03:34:57.722375+0000 mon.b (mon.1) 398 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:59.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:58 smithi161 bash[19769]: cluster 2024-02-13T03:34:57.455929+0000 mgr.y (mgr.24473) 547 : cluster [DBG] pgmap v801: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:34:59.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:58 smithi161 bash[19769]: cluster 2024-02-13T03:34:57.715867+0000 mon.a (mon.0) 2410 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-02-13T03:34:59.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:58 smithi161 bash[19769]: audit 2024-02-13T03:34:57.720950+0000 mon.a (mon.0) 2411 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:59.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:58 smithi161 bash[19769]: audit 2024-02-13T03:34:57.722375+0000 mon.b (mon.1) 398 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:34:59.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:34:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:34:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:35:00.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:59 smithi161 bash[27879]: audit 2024-02-13T03:34:58.721787+0000 mon.a (mon.0) 2412 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:00.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:59 smithi161 bash[27879]: cluster 2024-02-13T03:34:58.722243+0000 mon.a (mon.0) 2413 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-02-13T03:35:00.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:59 smithi161 bash[27879]: audit 2024-02-13T03:34:58.742286+0000 mon.b (mon.1) 399 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:35:00.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:59 smithi161 bash[27879]: audit 2024-02-13T03:34:58.743939+0000 mon.a (mon.0) 2414 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:35:00.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:34:59 smithi161 bash[27879]: audit 2024-02-13T03:34:58.745871+0000 mon.b (mon.1) 400 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:35:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:59 smithi161 bash[19769]: audit 2024-02-13T03:34:58.721787+0000 mon.a (mon.0) 2412 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:59 smithi161 bash[19769]: cluster 2024-02-13T03:34:58.722243+0000 mon.a (mon.0) 2413 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-02-13T03:35:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:59 smithi161 bash[19769]: audit 2024-02-13T03:34:58.742286+0000 mon.b (mon.1) 399 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:35:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:59 smithi161 bash[19769]: audit 2024-02-13T03:34:58.743939+0000 mon.a (mon.0) 2414 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:35:00.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:34:59 smithi161 bash[19769]: audit 2024-02-13T03:34:58.745871+0000 mon.b (mon.1) 400 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:35:00.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:59 smithi183 bash[20824]: audit 2024-02-13T03:34:58.721787+0000 mon.a (mon.0) 2412 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:00.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:59 smithi183 bash[20824]: cluster 2024-02-13T03:34:58.722243+0000 mon.a (mon.0) 2413 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-02-13T03:35:00.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:59 smithi183 bash[20824]: audit 2024-02-13T03:34:58.742286+0000 mon.b (mon.1) 399 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:35:00.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:59 smithi183 bash[20824]: audit 2024-02-13T03:34:58.743939+0000 mon.a (mon.0) 2414 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:35:00.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:34:59 smithi183 bash[20824]: audit 2024-02-13T03:34:58.745871+0000 mon.b (mon.1) 400 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:35:01.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:00 smithi161 bash[27879]: cluster 2024-02-13T03:34:59.456791+0000 mgr.y (mgr.24473) 548 : cluster [DBG] pgmap v804: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:35:01.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:00 smithi161 bash[27879]: audit 2024-02-13T03:34:59.737282+0000 mon.a (mon.0) 2415 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:35:01.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:00 smithi161 bash[27879]: cluster 2024-02-13T03:34:59.737383+0000 mon.a (mon.0) 2416 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-02-13T03:35:01.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:00 smithi161 bash[19769]: cluster 2024-02-13T03:34:59.456791+0000 mgr.y (mgr.24473) 548 : cluster [DBG] pgmap v804: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:35:01.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:00 smithi161 bash[19769]: audit 2024-02-13T03:34:59.737282+0000 mon.a (mon.0) 2415 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:35:01.099 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:00 smithi161 bash[19769]: cluster 2024-02-13T03:34:59.737383+0000 mon.a (mon.0) 2416 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-02-13T03:35:01.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:00 smithi183 bash[20824]: cluster 2024-02-13T03:34:59.456791+0000 mgr.y (mgr.24473) 548 : cluster [DBG] pgmap v804: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:35:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:00 smithi183 bash[20824]: audit 2024-02-13T03:34:59.737282+0000 mon.a (mon.0) 2415 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:35:01.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:00 smithi183 bash[20824]: cluster 2024-02-13T03:34:59.737383+0000 mon.a (mon.0) 2416 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-02-13T03:35:01.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:35:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:35:01] "GET /metrics HTTP/1.1" 200 188358 "" "Prometheus/2.33.4" 2024-02-13T03:35:01.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:01 smithi183 bash[20824]: cluster 2024-02-13T03:35:00.750305+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-02-13T03:35:02.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:01 smithi161 bash[27879]: cluster 2024-02-13T03:35:00.750305+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-02-13T03:35:02.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:01 smithi161 bash[19769]: cluster 2024-02-13T03:35:00.750305+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-02-13T03:35:02.786 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22183 ms) 2024-02-13T03:35:02.787 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-02-13T03:35:02.787 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3038 ms) 2024-02-13T03:35:02.787 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-02-13T03:35:02.787 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.787 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3024 ms) 2024-02-13T03:35:02.787 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-02-13T03:35:02.787 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3030 ms) 2024-02-13T03:35:02.787 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-02-13T03:35:02.787 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.787 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3015 ms) 2024-02-13T03:35:02.787 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-02-13T03:35:02.787 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.788 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4047 ms) 2024-02-13T03:35:02.788 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-02-13T03:35:02.788 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.788 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38518 ms) 2024-02-13T03:35:02.788 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-02-13T03:35:02.788 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.788 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16908 ms) 2024-02-13T03:35:02.788 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-02-13T03:35:02.788 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.788 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4074 ms) 2024-02-13T03:35:02.788 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-02-13T03:35:02.788 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.788 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3032 ms) 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25087 ms) 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15078 ms) 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5055 ms) 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: waiting for scrubs... 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: done waiting 2024-02-13T03:35:02.789 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25117 ms) 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10121 ms) 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (10111 ms) 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9123 ms) 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9072 ms) 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:35:02.790 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6086 ms) 2024-02-13T03:35:03.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:02 smithi161 bash[27879]: cluster 2024-02-13T03:35:01.457688+0000 mgr.y (mgr.24473) 549 : cluster [DBG] pgmap v807: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-13T03:35:03.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:02 smithi161 bash[27879]: cluster 2024-02-13T03:35:01.764005+0000 mon.a (mon.0) 2418 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-02-13T03:35:03.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:02 smithi161 bash[27879]: audit 2024-02-13T03:35:01.813255+0000 mon.a (mon.0) 2419 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:03.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:02 smithi161 bash[27879]: audit 2024-02-13T03:35:01.814821+0000 mon.a (mon.0) 2420 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-86"}]: dispatch 2024-02-13T03:35:03.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:02 smithi161 bash[27879]: audit 2024-02-13T03:35:01.814829+0000 mon.b (mon.1) 401 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:03.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:02 smithi161 bash[27879]: audit 2024-02-13T03:35:01.816576+0000 mon.b (mon.1) 402 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-86"}]: dispatch 2024-02-13T03:35:03.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:02 smithi161 bash[19769]: cluster 2024-02-13T03:35:01.457688+0000 mgr.y (mgr.24473) 549 : cluster [DBG] pgmap v807: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-13T03:35:03.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:02 smithi161 bash[19769]: cluster 2024-02-13T03:35:01.764005+0000 mon.a (mon.0) 2418 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-02-13T03:35:03.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:02 smithi161 bash[19769]: audit 2024-02-13T03:35:01.813255+0000 mon.a (mon.0) 2419 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:03.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:02 smithi161 bash[19769]: audit 2024-02-13T03:35:01.814821+0000 mon.a (mon.0) 2420 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-86"}]: dispatch 2024-02-13T03:35:03.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:02 smithi161 bash[19769]: audit 2024-02-13T03:35:01.814829+0000 mon.b (mon.1) 401 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:03.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:02 smithi161 bash[19769]: audit 2024-02-13T03:35:01.816576+0000 mon.b (mon.1) 402 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-86"}]: dispatch 2024-02-13T03:35:03.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:02 smithi183 bash[20824]: cluster 2024-02-13T03:35:01.457688+0000 mgr.y (mgr.24473) 549 : cluster [DBG] pgmap v807: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-13T03:35:03.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:02 smithi183 bash[20824]: cluster 2024-02-13T03:35:01.764005+0000 mon.a (mon.0) 2418 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-02-13T03:35:03.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:02 smithi183 bash[20824]: audit 2024-02-13T03:35:01.813255+0000 mon.a (mon.0) 2419 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:03.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:02 smithi183 bash[20824]: audit 2024-02-13T03:35:01.814821+0000 mon.a (mon.0) 2420 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-86"}]: dispatch 2024-02-13T03:35:03.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:02 smithi183 bash[20824]: audit 2024-02-13T03:35:01.814829+0000 mon.b (mon.1) 401 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:03.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:02 smithi183 bash[20824]: audit 2024-02-13T03:35:01.816576+0000 mon.b (mon.1) 402 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-86"}]: dispatch 2024-02-13T03:35:04.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:03 smithi161 bash[27879]: cluster 2024-02-13T03:35:02.784365+0000 mon.a (mon.0) 2421 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-02-13T03:35:04.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:03 smithi161 bash[19769]: cluster 2024-02-13T03:35:02.784365+0000 mon.a (mon.0) 2421 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-02-13T03:35:04.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:03 smithi183 bash[20824]: cluster 2024-02-13T03:35:02.784365+0000 mon.a (mon.0) 2421 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-02-13T03:35:05.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:04 smithi161 bash[27879]: cluster 2024-02-13T03:35:03.458650+0000 mgr.y (mgr.24473) 550 : cluster [DBG] pgmap v810: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:35:05.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:04 smithi161 bash[27879]: cluster 2024-02-13T03:35:03.796129+0000 mon.a (mon.0) 2422 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-02-13T03:35:05.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:04 smithi161 bash[27879]: audit 2024-02-13T03:35:03.806064+0000 mon.a (mon.0) 2423 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:05.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:04 smithi161 bash[27879]: audit 2024-02-13T03:35:03.807500+0000 mon.b (mon.1) 403 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:05.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:04 smithi161 bash[19769]: cluster 2024-02-13T03:35:03.458650+0000 mgr.y (mgr.24473) 550 : cluster [DBG] pgmap v810: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:35:05.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:04 smithi161 bash[19769]: cluster 2024-02-13T03:35:03.796129+0000 mon.a (mon.0) 2422 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-02-13T03:35:05.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:04 smithi161 bash[19769]: audit 2024-02-13T03:35:03.806064+0000 mon.a (mon.0) 2423 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:05.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:04 smithi161 bash[19769]: audit 2024-02-13T03:35:03.807500+0000 mon.b (mon.1) 403 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:04 smithi183 bash[20824]: cluster 2024-02-13T03:35:03.458650+0000 mgr.y (mgr.24473) 550 : cluster [DBG] pgmap v810: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:35:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:04 smithi183 bash[20824]: cluster 2024-02-13T03:35:03.796129+0000 mon.a (mon.0) 2422 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-02-13T03:35:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:04 smithi183 bash[20824]: audit 2024-02-13T03:35:03.806064+0000 mon.a (mon.0) 2423 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:05.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:04 smithi183 bash[20824]: audit 2024-02-13T03:35:03.807500+0000 mon.b (mon.1) 403 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:05 smithi161 bash[27879]: audit 2024-02-13T03:35:04.800240+0000 mon.a (mon.0) 2424 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:05 smithi161 bash[27879]: cluster 2024-02-13T03:35:04.800630+0000 mon.a (mon.0) 2425 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-02-13T03:35:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:05 smithi161 bash[27879]: audit 2024-02-13T03:35:04.826543+0000 mon.a (mon.0) 2426 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:35:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:05 smithi161 bash[27879]: audit 2024-02-13T03:35:04.828418+0000 mon.b (mon.1) 404 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:35:06.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:05 smithi161 bash[19769]: audit 2024-02-13T03:35:04.800240+0000 mon.a (mon.0) 2424 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:06.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:05 smithi161 bash[19769]: cluster 2024-02-13T03:35:04.800630+0000 mon.a (mon.0) 2425 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-02-13T03:35:06.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:05 smithi161 bash[19769]: audit 2024-02-13T03:35:04.826543+0000 mon.a (mon.0) 2426 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:35:06.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:05 smithi161 bash[19769]: audit 2024-02-13T03:35:04.828418+0000 mon.b (mon.1) 404 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:35:06.234 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:05 smithi183 bash[20824]: audit 2024-02-13T03:35:04.800240+0000 mon.a (mon.0) 2424 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:06.234 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:05 smithi183 bash[20824]: cluster 2024-02-13T03:35:04.800630+0000 mon.a (mon.0) 2425 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-02-13T03:35:06.235 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:05 smithi183 bash[20824]: audit 2024-02-13T03:35:04.826543+0000 mon.a (mon.0) 2426 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:35:06.235 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:05 smithi183 bash[20824]: audit 2024-02-13T03:35:04.828418+0000 mon.b (mon.1) 404 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:35:06.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:35:06 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:35:07.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[27879]: cluster 2024-02-13T03:35:05.459455+0000 mgr.y (mgr.24473) 551 : cluster [DBG] pgmap v813: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:07.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[27879]: audit 2024-02-13T03:35:05.816305+0000 mon.a (mon.0) 2427 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:35:07.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[27879]: cluster 2024-02-13T03:35:05.816519+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-02-13T03:35:07.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[27879]: audit 2024-02-13T03:35:05.818630+0000 mon.a (mon.0) 2429 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-88", "mode": "writeback"}]: dispatch 2024-02-13T03:35:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[27879]: audit 2024-02-13T03:35:05.820387+0000 mon.b (mon.1) 405 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-88", "mode": "writeback"}]: dispatch 2024-02-13T03:35:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[27879]: audit 2024-02-13T03:35:06.232834+0000 mgr.y (mgr.24473) 552 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[27879]: cluster 2024-02-13T03:35:06.810069+0000 mon.a (mon.0) 2430 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:35:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[27879]: audit 2024-02-13T03:35:06.819024+0000 mon.a (mon.0) 2431 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-88", "mode": "writeback"}]': finished 2024-02-13T03:35:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[27879]: cluster 2024-02-13T03:35:06.819226+0000 mon.a (mon.0) 2432 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-02-13T03:35:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[27879]: audit 2024-02-13T03:35:06.821971+0000 mon.a (mon.0) 2433 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-88"}]: dispatch 2024-02-13T03:35:07.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[27879]: audit 2024-02-13T03:35:06.823774+0000 mon.b (mon.1) 406 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-88"}]: dispatch 2024-02-13T03:35:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[19769]: cluster 2024-02-13T03:35:05.459455+0000 mgr.y (mgr.24473) 551 : cluster [DBG] pgmap v813: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[19769]: audit 2024-02-13T03:35:05.816305+0000 mon.a (mon.0) 2427 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:35:07.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[19769]: cluster 2024-02-13T03:35:05.816519+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-02-13T03:35:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[19769]: audit 2024-02-13T03:35:05.818630+0000 mon.a (mon.0) 2429 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-88", "mode": "writeback"}]: dispatch 2024-02-13T03:35:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[19769]: audit 2024-02-13T03:35:05.820387+0000 mon.b (mon.1) 405 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-88", "mode": "writeback"}]: dispatch 2024-02-13T03:35:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[19769]: audit 2024-02-13T03:35:06.232834+0000 mgr.y (mgr.24473) 552 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[19769]: cluster 2024-02-13T03:35:06.810069+0000 mon.a (mon.0) 2430 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:35:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[19769]: audit 2024-02-13T03:35:06.819024+0000 mon.a (mon.0) 2431 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-88", "mode": "writeback"}]': finished 2024-02-13T03:35:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[19769]: cluster 2024-02-13T03:35:06.819226+0000 mon.a (mon.0) 2432 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-02-13T03:35:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[19769]: audit 2024-02-13T03:35:06.821971+0000 mon.a (mon.0) 2433 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-88"}]: dispatch 2024-02-13T03:35:07.178 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:06 smithi161 bash[19769]: audit 2024-02-13T03:35:06.823774+0000 mon.b (mon.1) 406 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-88"}]: dispatch 2024-02-13T03:35:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:06 smithi183 bash[20824]: cluster 2024-02-13T03:35:05.459455+0000 mgr.y (mgr.24473) 551 : cluster [DBG] pgmap v813: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:06 smithi183 bash[20824]: audit 2024-02-13T03:35:05.816305+0000 mon.a (mon.0) 2427 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:35:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:06 smithi183 bash[20824]: cluster 2024-02-13T03:35:05.816519+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-02-13T03:35:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:06 smithi183 bash[20824]: audit 2024-02-13T03:35:05.818630+0000 mon.a (mon.0) 2429 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-88", "mode": "writeback"}]: dispatch 2024-02-13T03:35:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:06 smithi183 bash[20824]: audit 2024-02-13T03:35:05.820387+0000 mon.b (mon.1) 405 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-88", "mode": "writeback"}]: dispatch 2024-02-13T03:35:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:06 smithi183 bash[20824]: audit 2024-02-13T03:35:06.232834+0000 mgr.y (mgr.24473) 552 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:06 smithi183 bash[20824]: cluster 2024-02-13T03:35:06.810069+0000 mon.a (mon.0) 2430 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:35:07.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:06 smithi183 bash[20824]: audit 2024-02-13T03:35:06.819024+0000 mon.a (mon.0) 2431 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-88", "mode": "writeback"}]': finished 2024-02-13T03:35:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:06 smithi183 bash[20824]: cluster 2024-02-13T03:35:06.819226+0000 mon.a (mon.0) 2432 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-02-13T03:35:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:06 smithi183 bash[20824]: audit 2024-02-13T03:35:06.821971+0000 mon.a (mon.0) 2433 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-88"}]: dispatch 2024-02-13T03:35:07.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:06 smithi183 bash[20824]: audit 2024-02-13T03:35:06.823774+0000 mon.b (mon.1) 406 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-88"}]: dispatch 2024-02-13T03:35:09.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:08 smithi161 bash[27879]: cluster 2024-02-13T03:35:07.460295+0000 mgr.y (mgr.24473) 553 : cluster [DBG] pgmap v816: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-13T03:35:09.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:08 smithi161 bash[27879]: audit 2024-02-13T03:35:07.822784+0000 mon.a (mon.0) 2434 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-88"}]': finished 2024-02-13T03:35:09.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:08 smithi161 bash[27879]: cluster 2024-02-13T03:35:07.823155+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-02-13T03:35:09.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:08 smithi161 bash[27879]: audit 2024-02-13T03:35:07.825082+0000 mon.a (mon.0) 2436 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:35:09.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:08 smithi161 bash[27879]: audit 2024-02-13T03:35:07.826775+0000 mon.b (mon.1) 407 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:35:09.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:08 smithi161 bash[19769]: cluster 2024-02-13T03:35:07.460295+0000 mgr.y (mgr.24473) 553 : cluster [DBG] pgmap v816: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-13T03:35:09.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:08 smithi161 bash[19769]: audit 2024-02-13T03:35:07.822784+0000 mon.a (mon.0) 2434 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-88"}]': finished 2024-02-13T03:35:09.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:08 smithi161 bash[19769]: cluster 2024-02-13T03:35:07.823155+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-02-13T03:35:09.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:08 smithi161 bash[19769]: audit 2024-02-13T03:35:07.825082+0000 mon.a (mon.0) 2436 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:35:09.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:08 smithi161 bash[19769]: audit 2024-02-13T03:35:07.826775+0000 mon.b (mon.1) 407 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:35:09.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:08 smithi183 bash[20824]: cluster 2024-02-13T03:35:07.460295+0000 mgr.y (mgr.24473) 553 : cluster [DBG] pgmap v816: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-13T03:35:09.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:08 smithi183 bash[20824]: audit 2024-02-13T03:35:07.822784+0000 mon.a (mon.0) 2434 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-88"}]': finished 2024-02-13T03:35:09.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:08 smithi183 bash[20824]: cluster 2024-02-13T03:35:07.823155+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-02-13T03:35:09.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:08 smithi183 bash[20824]: audit 2024-02-13T03:35:07.825082+0000 mon.a (mon.0) 2436 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:35:09.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:08 smithi183 bash[20824]: audit 2024-02-13T03:35:07.826775+0000 mon.b (mon.1) 407 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:35:09.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:35:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:35:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:35:10.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:09 smithi161 bash[19769]: cluster 2024-02-13T03:35:08.815189+0000 mon.a (mon.0) 2437 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:35:10.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:09 smithi161 bash[19769]: audit 2024-02-13T03:35:08.823162+0000 mon.a (mon.0) 2438 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:35:10.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:09 smithi161 bash[19769]: cluster 2024-02-13T03:35:08.823262+0000 mon.a (mon.0) 2439 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-02-13T03:35:10.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:09 smithi161 bash[19769]: audit 2024-02-13T03:35:08.827935+0000 mon.b (mon.1) 408 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-13T03:35:10.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:09 smithi161 bash[19769]: audit 2024-02-13T03:35:08.830794+0000 mon.a (mon.0) 2440 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-13T03:35:10.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:09 smithi161 bash[27879]: cluster 2024-02-13T03:35:08.815189+0000 mon.a (mon.0) 2437 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:35:10.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:09 smithi161 bash[27879]: audit 2024-02-13T03:35:08.823162+0000 mon.a (mon.0) 2438 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:35:10.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:09 smithi161 bash[27879]: cluster 2024-02-13T03:35:08.823262+0000 mon.a (mon.0) 2439 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-02-13T03:35:10.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:09 smithi161 bash[27879]: audit 2024-02-13T03:35:08.827935+0000 mon.b (mon.1) 408 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-13T03:35:10.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:09 smithi161 bash[27879]: audit 2024-02-13T03:35:08.830794+0000 mon.a (mon.0) 2440 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-13T03:35:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:09 smithi183 bash[20824]: cluster 2024-02-13T03:35:08.815189+0000 mon.a (mon.0) 2437 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:35:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:09 smithi183 bash[20824]: audit 2024-02-13T03:35:08.823162+0000 mon.a (mon.0) 2438 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:35:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:09 smithi183 bash[20824]: cluster 2024-02-13T03:35:08.823262+0000 mon.a (mon.0) 2439 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-02-13T03:35:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:09 smithi183 bash[20824]: audit 2024-02-13T03:35:08.827935+0000 mon.b (mon.1) 408 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-13T03:35:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:09 smithi183 bash[20824]: audit 2024-02-13T03:35:08.830794+0000 mon.a (mon.0) 2440 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-13T03:35:11.176 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:35:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:35:11] "GET /metrics HTTP/1.1" 200 188485 "" "Prometheus/2.33.4" 2024-02-13T03:35:11.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:10 smithi161 bash[27879]: cluster 2024-02-13T03:35:09.461321+0000 mgr.y (mgr.24473) 554 : cluster [DBG] pgmap v819: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-13T03:35:11.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:10 smithi161 bash[27879]: audit 2024-02-13T03:35:09.834956+0000 mon.a (mon.0) 2441 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_count","val": "1"}]': finished 2024-02-13T03:35:11.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:10 smithi161 bash[27879]: cluster 2024-02-13T03:35:09.835182+0000 mon.a (mon.0) 2442 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-02-13T03:35:11.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:10 smithi161 bash[27879]: audit 2024-02-13T03:35:09.840225+0000 mon.b (mon.1) 409 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:11.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:10 smithi161 bash[27879]: audit 2024-02-13T03:35:09.847137+0000 mon.a (mon.0) 2443 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:11.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:10 smithi161 bash[19769]: cluster 2024-02-13T03:35:09.461321+0000 mgr.y (mgr.24473) 554 : cluster [DBG] pgmap v819: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-13T03:35:11.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:10 smithi161 bash[19769]: audit 2024-02-13T03:35:09.834956+0000 mon.a (mon.0) 2441 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_count","val": "1"}]': finished 2024-02-13T03:35:11.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:10 smithi161 bash[19769]: cluster 2024-02-13T03:35:09.835182+0000 mon.a (mon.0) 2442 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-02-13T03:35:11.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:10 smithi161 bash[19769]: audit 2024-02-13T03:35:09.840225+0000 mon.b (mon.1) 409 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:11.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:10 smithi161 bash[19769]: audit 2024-02-13T03:35:09.847137+0000 mon.a (mon.0) 2443 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:11.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:10 smithi183 bash[20824]: cluster 2024-02-13T03:35:09.461321+0000 mgr.y (mgr.24473) 554 : cluster [DBG] pgmap v819: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-13T03:35:11.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:10 smithi183 bash[20824]: audit 2024-02-13T03:35:09.834956+0000 mon.a (mon.0) 2441 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_count","val": "1"}]': finished 2024-02-13T03:35:11.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:10 smithi183 bash[20824]: cluster 2024-02-13T03:35:09.835182+0000 mon.a (mon.0) 2442 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-02-13T03:35:11.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:10 smithi183 bash[20824]: audit 2024-02-13T03:35:09.840225+0000 mon.b (mon.1) 409 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:11.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:10 smithi183 bash[20824]: audit 2024-02-13T03:35:09.847137+0000 mon.a (mon.0) 2443 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:11 smithi161 bash[27879]: audit 2024-02-13T03:35:10.858479+0000 mon.a (mon.0) 2444 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:35:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:11 smithi161 bash[27879]: cluster 2024-02-13T03:35:10.858675+0000 mon.a (mon.0) 2445 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-02-13T03:35:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:11 smithi161 bash[27879]: audit 2024-02-13T03:35:10.860340+0000 mon.a (mon.0) 2446 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-13T03:35:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:11 smithi161 bash[27879]: audit 2024-02-13T03:35:10.862089+0000 mon.b (mon.1) 410 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-13T03:35:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:11 smithi161 bash[27879]: audit 2024-02-13T03:35:11.857334+0000 mon.a (mon.0) 2447 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "target_max_objects","val": "250"}]': finished 2024-02-13T03:35:12.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:11 smithi161 bash[27879]: cluster 2024-02-13T03:35:11.857456+0000 mon.a (mon.0) 2448 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-02-13T03:35:12.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:11 smithi161 bash[19769]: audit 2024-02-13T03:35:10.858479+0000 mon.a (mon.0) 2444 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:35:12.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:11 smithi161 bash[19769]: cluster 2024-02-13T03:35:10.858675+0000 mon.a (mon.0) 2445 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-02-13T03:35:12.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:11 smithi161 bash[19769]: audit 2024-02-13T03:35:10.860340+0000 mon.a (mon.0) 2446 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-13T03:35:12.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:11 smithi161 bash[19769]: audit 2024-02-13T03:35:10.862089+0000 mon.b (mon.1) 410 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-13T03:35:12.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:11 smithi161 bash[19769]: audit 2024-02-13T03:35:11.857334+0000 mon.a (mon.0) 2447 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "target_max_objects","val": "250"}]': finished 2024-02-13T03:35:12.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:11 smithi161 bash[19769]: cluster 2024-02-13T03:35:11.857456+0000 mon.a (mon.0) 2448 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-02-13T03:35:12.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:11 smithi183 bash[20824]: audit 2024-02-13T03:35:10.858479+0000 mon.a (mon.0) 2444 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:35:12.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:11 smithi183 bash[20824]: cluster 2024-02-13T03:35:10.858675+0000 mon.a (mon.0) 2445 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-02-13T03:35:12.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:11 smithi183 bash[20824]: audit 2024-02-13T03:35:10.860340+0000 mon.a (mon.0) 2446 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-13T03:35:12.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:11 smithi183 bash[20824]: audit 2024-02-13T03:35:10.862089+0000 mon.b (mon.1) 410 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-13T03:35:12.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:11 smithi183 bash[20824]: audit 2024-02-13T03:35:11.857334+0000 mon.a (mon.0) 2447 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-88","var": "target_max_objects","val": "250"}]': finished 2024-02-13T03:35:12.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:11 smithi183 bash[20824]: cluster 2024-02-13T03:35:11.857456+0000 mon.a (mon.0) 2448 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-02-13T03:35:13.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:12 smithi161 bash[19769]: cluster 2024-02-13T03:35:11.462102+0000 mgr.y (mgr.24473) 555 : cluster [DBG] pgmap v822: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:35:13.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:12 smithi161 bash[19769]: audit 2024-02-13T03:35:11.897197+0000 mon.a (mon.0) 2449 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:13.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:12 smithi161 bash[19769]: audit 2024-02-13T03:35:11.898929+0000 mon.b (mon.1) 411 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:13.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:12 smithi161 bash[27879]: cluster 2024-02-13T03:35:11.462102+0000 mgr.y (mgr.24473) 555 : cluster [DBG] pgmap v822: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:35:13.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:12 smithi161 bash[27879]: audit 2024-02-13T03:35:11.897197+0000 mon.a (mon.0) 2449 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:13.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:12 smithi161 bash[27879]: audit 2024-02-13T03:35:11.898929+0000 mon.b (mon.1) 411 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:12 smithi183 bash[20824]: cluster 2024-02-13T03:35:11.462102+0000 mgr.y (mgr.24473) 555 : cluster [DBG] pgmap v822: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:35:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:12 smithi183 bash[20824]: audit 2024-02-13T03:35:11.897197+0000 mon.a (mon.0) 2449 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:12 smithi183 bash[20824]: audit 2024-02-13T03:35:11.898929+0000 mon.b (mon.1) 411 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:14.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:13 smithi161 bash[27879]: audit 2024-02-13T03:35:12.885004+0000 mon.a (mon.0) 2450 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:35:14.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:13 smithi161 bash[27879]: cluster 2024-02-13T03:35:12.885143+0000 mon.a (mon.0) 2451 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-02-13T03:35:14.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:13 smithi161 bash[27879]: audit 2024-02-13T03:35:12.887220+0000 mon.a (mon.0) 2452 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88"}]: dispatch 2024-02-13T03:35:14.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:13 smithi161 bash[27879]: audit 2024-02-13T03:35:12.888463+0000 mon.b (mon.1) 412 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88"}]: dispatch 2024-02-13T03:35:14.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:13 smithi161 bash[19769]: audit 2024-02-13T03:35:12.885004+0000 mon.a (mon.0) 2450 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:35:14.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:13 smithi161 bash[19769]: cluster 2024-02-13T03:35:12.885143+0000 mon.a (mon.0) 2451 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-02-13T03:35:14.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:13 smithi161 bash[19769]: audit 2024-02-13T03:35:12.887220+0000 mon.a (mon.0) 2452 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88"}]: dispatch 2024-02-13T03:35:14.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:13 smithi161 bash[19769]: audit 2024-02-13T03:35:12.888463+0000 mon.b (mon.1) 412 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88"}]: dispatch 2024-02-13T03:35:14.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:13 smithi183 bash[20824]: audit 2024-02-13T03:35:12.885004+0000 mon.a (mon.0) 2450 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:35:14.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:13 smithi183 bash[20824]: cluster 2024-02-13T03:35:12.885143+0000 mon.a (mon.0) 2451 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-02-13T03:35:14.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:13 smithi183 bash[20824]: audit 2024-02-13T03:35:12.887220+0000 mon.a (mon.0) 2452 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88"}]: dispatch 2024-02-13T03:35:14.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:13 smithi183 bash[20824]: audit 2024-02-13T03:35:12.888463+0000 mon.b (mon.1) 412 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88"}]: dispatch 2024-02-13T03:35:15.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:14 smithi161 bash[27879]: cluster 2024-02-13T03:35:13.462892+0000 mgr.y (mgr.24473) 556 : cluster [DBG] pgmap v825: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:35:15.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:14 smithi161 bash[27879]: audit 2024-02-13T03:35:13.888883+0000 mon.a (mon.0) 2453 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88"}]': finished 2024-02-13T03:35:15.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:14 smithi161 bash[27879]: cluster 2024-02-13T03:35:13.889146+0000 mon.a (mon.0) 2454 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-02-13T03:35:15.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:14 smithi161 bash[19769]: cluster 2024-02-13T03:35:13.462892+0000 mgr.y (mgr.24473) 556 : cluster [DBG] pgmap v825: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:35:15.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:14 smithi161 bash[19769]: audit 2024-02-13T03:35:13.888883+0000 mon.a (mon.0) 2453 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88"}]': finished 2024-02-13T03:35:15.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:14 smithi161 bash[19769]: cluster 2024-02-13T03:35:13.889146+0000 mon.a (mon.0) 2454 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-02-13T03:35:15.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:14 smithi183 bash[20824]: cluster 2024-02-13T03:35:13.462892+0000 mgr.y (mgr.24473) 556 : cluster [DBG] pgmap v825: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:35:15.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:14 smithi183 bash[20824]: audit 2024-02-13T03:35:13.888883+0000 mon.a (mon.0) 2453 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-88"}]': finished 2024-02-13T03:35:15.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:14 smithi183 bash[20824]: cluster 2024-02-13T03:35:13.889146+0000 mon.a (mon.0) 2454 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-02-13T03:35:16.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:15 smithi161 bash[27879]: cluster 2024-02-13T03:35:14.913047+0000 mon.a (mon.0) 2455 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-02-13T03:35:16.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:15 smithi161 bash[19769]: cluster 2024-02-13T03:35:14.913047+0000 mon.a (mon.0) 2455 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-02-13T03:35:16.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:15 smithi183 bash[20824]: cluster 2024-02-13T03:35:14.913047+0000 mon.a (mon.0) 2455 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-02-13T03:35:16.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:35:16 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:35:17.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:16 smithi183 bash[20824]: cluster 2024-02-13T03:35:15.463602+0000 mgr.y (mgr.24473) 557 : cluster [DBG] pgmap v828: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:17.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:16 smithi183 bash[20824]: cluster 2024-02-13T03:35:15.926519+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-02-13T03:35:17.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:16 smithi183 bash[20824]: audit 2024-02-13T03:35:15.933377+0000 mon.a (mon.0) 2457 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:17.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:16 smithi183 bash[20824]: audit 2024-02-13T03:35:15.934977+0000 mon.b (mon.1) 413 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:17.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:16 smithi183 bash[20824]: audit 2024-02-13T03:35:16.239884+0000 mgr.y (mgr.24473) 558 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:16 smithi161 bash[27879]: cluster 2024-02-13T03:35:15.463602+0000 mgr.y (mgr.24473) 557 : cluster [DBG] pgmap v828: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:16 smithi161 bash[27879]: cluster 2024-02-13T03:35:15.926519+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-02-13T03:35:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:16 smithi161 bash[27879]: audit 2024-02-13T03:35:15.933377+0000 mon.a (mon.0) 2457 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:16 smithi161 bash[27879]: audit 2024-02-13T03:35:15.934977+0000 mon.b (mon.1) 413 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:16 smithi161 bash[27879]: audit 2024-02-13T03:35:16.239884+0000 mgr.y (mgr.24473) 558 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:17.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:16 smithi161 bash[19769]: cluster 2024-02-13T03:35:15.463602+0000 mgr.y (mgr.24473) 557 : cluster [DBG] pgmap v828: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:17.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:16 smithi161 bash[19769]: cluster 2024-02-13T03:35:15.926519+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-02-13T03:35:17.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:16 smithi161 bash[19769]: audit 2024-02-13T03:35:15.933377+0000 mon.a (mon.0) 2457 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:17.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:16 smithi161 bash[19769]: audit 2024-02-13T03:35:15.934977+0000 mon.b (mon.1) 413 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:17.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:16 smithi161 bash[19769]: audit 2024-02-13T03:35:16.239884+0000 mgr.y (mgr.24473) 558 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:17 smithi183 bash[20824]: audit 2024-02-13T03:35:16.929365+0000 mon.a (mon.0) 2458 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:17 smithi183 bash[20824]: cluster 2024-02-13T03:35:16.930885+0000 mon.a (mon.0) 2459 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-02-13T03:35:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:17 smithi183 bash[20824]: audit 2024-02-13T03:35:16.935243+0000 mon.b (mon.1) 414 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:35:18.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:17 smithi183 bash[20824]: audit 2024-02-13T03:35:16.945008+0000 mon.a (mon.0) 2460 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:35:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:17 smithi161 bash[27879]: audit 2024-02-13T03:35:16.929365+0000 mon.a (mon.0) 2458 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:17 smithi161 bash[27879]: cluster 2024-02-13T03:35:16.930885+0000 mon.a (mon.0) 2459 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-02-13T03:35:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:17 smithi161 bash[27879]: audit 2024-02-13T03:35:16.935243+0000 mon.b (mon.1) 414 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:35:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:17 smithi161 bash[27879]: audit 2024-02-13T03:35:16.945008+0000 mon.a (mon.0) 2460 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:35:18.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:17 smithi161 bash[19769]: audit 2024-02-13T03:35:16.929365+0000 mon.a (mon.0) 2458 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:18.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:17 smithi161 bash[19769]: cluster 2024-02-13T03:35:16.930885+0000 mon.a (mon.0) 2459 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-02-13T03:35:18.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:17 smithi161 bash[19769]: audit 2024-02-13T03:35:16.935243+0000 mon.b (mon.1) 414 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:35:18.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:17 smithi161 bash[19769]: audit 2024-02-13T03:35:16.945008+0000 mon.a (mon.0) 2460 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:35:19.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:18 smithi183 bash[20824]: cluster 2024-02-13T03:35:17.464446+0000 mgr.y (mgr.24473) 559 : cluster [DBG] pgmap v831: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:19.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:18 smithi183 bash[20824]: audit 2024-02-13T03:35:17.951965+0000 mon.a (mon.0) 2461 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:35:19.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:18 smithi183 bash[20824]: cluster 2024-02-13T03:35:17.952058+0000 mon.a (mon.0) 2462 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-02-13T03:35:19.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:18 smithi183 bash[20824]: audit 2024-02-13T03:35:17.956957+0000 mon.b (mon.1) 415 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-90"}]: dispatch 2024-02-13T03:35:19.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:18 smithi183 bash[20824]: audit 2024-02-13T03:35:17.957725+0000 mon.a (mon.0) 2463 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-90"}]: dispatch 2024-02-13T03:35:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:18 smithi161 bash[27879]: cluster 2024-02-13T03:35:17.464446+0000 mgr.y (mgr.24473) 559 : cluster [DBG] pgmap v831: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:18 smithi161 bash[27879]: audit 2024-02-13T03:35:17.951965+0000 mon.a (mon.0) 2461 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:35:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:18 smithi161 bash[27879]: cluster 2024-02-13T03:35:17.952058+0000 mon.a (mon.0) 2462 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-02-13T03:35:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:18 smithi161 bash[27879]: audit 2024-02-13T03:35:17.956957+0000 mon.b (mon.1) 415 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-90"}]: dispatch 2024-02-13T03:35:19.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:18 smithi161 bash[27879]: audit 2024-02-13T03:35:17.957725+0000 mon.a (mon.0) 2463 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-90"}]: dispatch 2024-02-13T03:35:19.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:18 smithi161 bash[19769]: cluster 2024-02-13T03:35:17.464446+0000 mgr.y (mgr.24473) 559 : cluster [DBG] pgmap v831: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:19.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:18 smithi161 bash[19769]: audit 2024-02-13T03:35:17.951965+0000 mon.a (mon.0) 2461 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:35:19.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:18 smithi161 bash[19769]: cluster 2024-02-13T03:35:17.952058+0000 mon.a (mon.0) 2462 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-02-13T03:35:19.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:18 smithi161 bash[19769]: audit 2024-02-13T03:35:17.956957+0000 mon.b (mon.1) 415 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-90"}]: dispatch 2024-02-13T03:35:19.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:18 smithi161 bash[19769]: audit 2024-02-13T03:35:17.957725+0000 mon.a (mon.0) 2463 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-90"}]: dispatch 2024-02-13T03:35:19.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:35:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:35:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:35:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:19 smithi183 bash[20824]: audit 2024-02-13T03:35:18.955210+0000 mon.a (mon.0) 2464 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-90"}]': finished 2024-02-13T03:35:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:19 smithi183 bash[20824]: cluster 2024-02-13T03:35:18.955504+0000 mon.a (mon.0) 2465 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-02-13T03:35:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:19 smithi183 bash[20824]: audit 2024-02-13T03:35:18.956828+0000 mon.a (mon.0) 2466 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-90", "mode": "writeback"}]: dispatch 2024-02-13T03:35:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:19 smithi183 bash[20824]: audit 2024-02-13T03:35:18.958583+0000 mon.b (mon.1) 416 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-90", "mode": "writeback"}]: dispatch 2024-02-13T03:35:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:19 smithi183 bash[20824]: audit 2024-02-13T03:35:19.385097+0000 mon.c (mon.2) 530 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:35:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:19 smithi183 bash[20824]: audit 2024-02-13T03:35:19.704350+0000 mon.a (mon.0) 2467 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:35:20.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:19 smithi183 bash[20824]: cluster 2024-02-13T03:35:19.948839+0000 mon.a (mon.0) 2468 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:35:20.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[19769]: audit 2024-02-13T03:35:18.955210+0000 mon.a (mon.0) 2464 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-90"}]': finished 2024-02-13T03:35:20.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[19769]: cluster 2024-02-13T03:35:18.955504+0000 mon.a (mon.0) 2465 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-02-13T03:35:20.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[19769]: audit 2024-02-13T03:35:18.956828+0000 mon.a (mon.0) 2466 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-90", "mode": "writeback"}]: dispatch 2024-02-13T03:35:20.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[19769]: audit 2024-02-13T03:35:18.958583+0000 mon.b (mon.1) 416 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-90", "mode": "writeback"}]: dispatch 2024-02-13T03:35:20.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[19769]: audit 2024-02-13T03:35:19.385097+0000 mon.c (mon.2) 530 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:35:20.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[19769]: audit 2024-02-13T03:35:19.704350+0000 mon.a (mon.0) 2467 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:35:20.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[19769]: cluster 2024-02-13T03:35:19.948839+0000 mon.a (mon.0) 2468 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:35:20.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[27879]: audit 2024-02-13T03:35:18.955210+0000 mon.a (mon.0) 2464 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-6", "overlaypool": "test-rados-api-smithi161-61906-90"}]': finished 2024-02-13T03:35:20.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[27879]: cluster 2024-02-13T03:35:18.955504+0000 mon.a (mon.0) 2465 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-02-13T03:35:20.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[27879]: audit 2024-02-13T03:35:18.956828+0000 mon.a (mon.0) 2466 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-90", "mode": "writeback"}]: dispatch 2024-02-13T03:35:20.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[27879]: audit 2024-02-13T03:35:18.958583+0000 mon.b (mon.1) 416 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-90", "mode": "writeback"}]: dispatch 2024-02-13T03:35:20.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[27879]: audit 2024-02-13T03:35:19.385097+0000 mon.c (mon.2) 530 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:35:20.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[27879]: audit 2024-02-13T03:35:19.704350+0000 mon.a (mon.0) 2467 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:35:20.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:19 smithi161 bash[27879]: cluster 2024-02-13T03:35:19.948839+0000 mon.a (mon.0) 2468 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:35:21.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:20 smithi183 bash[20824]: cluster 2024-02-13T03:35:19.465268+0000 mgr.y (mgr.24473) 560 : cluster [DBG] pgmap v834: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:20 smithi183 bash[20824]: audit 2024-02-13T03:35:19.958004+0000 mon.a (mon.0) 2469 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-90", "mode": "writeback"}]': finished 2024-02-13T03:35:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:20 smithi183 bash[20824]: cluster 2024-02-13T03:35:19.958193+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-02-13T03:35:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:20 smithi183 bash[20824]: audit 2024-02-13T03:35:19.961569+0000 mon.b (mon.1) 417 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:35:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:20 smithi183 bash[20824]: audit 2024-02-13T03:35:19.962392+0000 mon.a (mon.0) 2471 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:35:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:20 smithi183 bash[20824]: audit 2024-02-13T03:35:19.994297+0000 mon.c (mon.2) 531 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:35:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:20 smithi183 bash[20824]: audit 2024-02-13T03:35:19.995574+0000 mon.c (mon.2) 532 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:35:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:20 smithi183 bash[20824]: audit 2024-02-13T03:35:20.003270+0000 mon.a (mon.0) 2472 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:35:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:20 smithi183 bash[20824]: audit 2024-02-13T03:35:20.959602+0000 mon.a (mon.0) 2473 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:35:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:20 smithi183 bash[20824]: cluster 2024-02-13T03:35:20.959927+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-02-13T03:35:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:20 smithi183 bash[20824]: audit 2024-02-13T03:35:20.962165+0000 mon.a (mon.0) 2475 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:20 smithi183 bash[20824]: audit 2024-02-13T03:35:20.963722+0000 mon.b (mon.1) 418 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:21.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:35:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:35:21] "GET /metrics HTTP/1.1" 200 188601 "" "Prometheus/2.33.4" 2024-02-13T03:35:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[27879]: cluster 2024-02-13T03:35:19.465268+0000 mgr.y (mgr.24473) 560 : cluster [DBG] pgmap v834: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[27879]: audit 2024-02-13T03:35:19.958004+0000 mon.a (mon.0) 2469 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-90", "mode": "writeback"}]': finished 2024-02-13T03:35:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[27879]: cluster 2024-02-13T03:35:19.958193+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-02-13T03:35:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[27879]: audit 2024-02-13T03:35:19.961569+0000 mon.b (mon.1) 417 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:35:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[27879]: audit 2024-02-13T03:35:19.962392+0000 mon.a (mon.0) 2471 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:35:21.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[27879]: audit 2024-02-13T03:35:19.994297+0000 mon.c (mon.2) 531 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:35:21.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[27879]: audit 2024-02-13T03:35:19.995574+0000 mon.c (mon.2) 532 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:35:21.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[27879]: audit 2024-02-13T03:35:20.003270+0000 mon.a (mon.0) 2472 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:35:21.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[27879]: audit 2024-02-13T03:35:20.959602+0000 mon.a (mon.0) 2473 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:35:21.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[27879]: cluster 2024-02-13T03:35:20.959927+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-02-13T03:35:21.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[27879]: audit 2024-02-13T03:35:20.962165+0000 mon.a (mon.0) 2475 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:21.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[27879]: audit 2024-02-13T03:35:20.963722+0000 mon.b (mon.1) 418 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[19769]: cluster 2024-02-13T03:35:19.465268+0000 mgr.y (mgr.24473) 560 : cluster [DBG] pgmap v834: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[19769]: audit 2024-02-13T03:35:19.958004+0000 mon.a (mon.0) 2469 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-90", "mode": "writeback"}]': finished 2024-02-13T03:35:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[19769]: cluster 2024-02-13T03:35:19.958193+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-02-13T03:35:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[19769]: audit 2024-02-13T03:35:19.961569+0000 mon.b (mon.1) 417 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:35:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[19769]: audit 2024-02-13T03:35:19.962392+0000 mon.a (mon.0) 2471 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:35:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[19769]: audit 2024-02-13T03:35:19.994297+0000 mon.c (mon.2) 531 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:35:21.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[19769]: audit 2024-02-13T03:35:19.995574+0000 mon.c (mon.2) 532 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:35:21.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[19769]: audit 2024-02-13T03:35:20.003270+0000 mon.a (mon.0) 2472 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:35:21.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[19769]: audit 2024-02-13T03:35:20.959602+0000 mon.a (mon.0) 2473 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:35:21.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[19769]: cluster 2024-02-13T03:35:20.959927+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-02-13T03:35:21.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[19769]: audit 2024-02-13T03:35:20.962165+0000 mon.a (mon.0) 2475 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:21.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:20 smithi161 bash[19769]: audit 2024-02-13T03:35:20.963722+0000 mon.b (mon.1) 418 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:22 smithi183 bash[20824]: cluster 2024-02-13T03:35:21.466176+0000 mgr.y (mgr.24473) 561 : cluster [DBG] pgmap v837: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:22 smithi183 bash[20824]: audit 2024-02-13T03:35:21.963299+0000 mon.a (mon.0) 2476 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:35:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:22 smithi183 bash[20824]: cluster 2024-02-13T03:35:21.963503+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-02-13T03:35:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:22 smithi183 bash[20824]: audit 2024-02-13T03:35:21.966174+0000 mon.a (mon.0) 2478 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:35:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:22 smithi183 bash[20824]: audit 2024-02-13T03:35:21.967945+0000 mon.b (mon.1) 419 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:35:23.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:22 smithi161 bash[27879]: cluster 2024-02-13T03:35:21.466176+0000 mgr.y (mgr.24473) 561 : cluster [DBG] pgmap v837: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:23.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:22 smithi161 bash[27879]: audit 2024-02-13T03:35:21.963299+0000 mon.a (mon.0) 2476 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:35:23.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:22 smithi161 bash[27879]: cluster 2024-02-13T03:35:21.963503+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-02-13T03:35:23.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:22 smithi161 bash[27879]: audit 2024-02-13T03:35:21.966174+0000 mon.a (mon.0) 2478 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:35:23.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:22 smithi161 bash[27879]: audit 2024-02-13T03:35:21.967945+0000 mon.b (mon.1) 419 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:35:23.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:22 smithi161 bash[19769]: cluster 2024-02-13T03:35:21.466176+0000 mgr.y (mgr.24473) 561 : cluster [DBG] pgmap v837: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:22 smithi161 bash[19769]: audit 2024-02-13T03:35:21.963299+0000 mon.a (mon.0) 2476 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:35:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:22 smithi161 bash[19769]: cluster 2024-02-13T03:35:21.963503+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-02-13T03:35:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:22 smithi161 bash[19769]: audit 2024-02-13T03:35:21.966174+0000 mon.a (mon.0) 2478 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:35:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:22 smithi161 bash[19769]: audit 2024-02-13T03:35:21.967945+0000 mon.b (mon.1) 419 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:35:24.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:23 smithi183 bash[20824]: cluster 2024-02-13T03:35:22.957737+0000 mon.a (mon.0) 2479 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:35:24.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:23 smithi183 bash[20824]: audit 2024-02-13T03:35:22.971914+0000 mon.a (mon.0) 2480 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-13T03:35:24.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:23 smithi183 bash[20824]: cluster 2024-02-13T03:35:22.972043+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-02-13T03:35:24.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:23 smithi183 bash[20824]: audit 2024-02-13T03:35:22.975493+0000 mon.a (mon.0) 2482 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-13T03:35:24.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:23 smithi183 bash[20824]: audit 2024-02-13T03:35:22.977294+0000 mon.b (mon.1) 420 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-13T03:35:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:23 smithi161 bash[27879]: cluster 2024-02-13T03:35:22.957737+0000 mon.a (mon.0) 2479 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:35:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:23 smithi161 bash[27879]: audit 2024-02-13T03:35:22.971914+0000 mon.a (mon.0) 2480 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-13T03:35:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:23 smithi161 bash[27879]: cluster 2024-02-13T03:35:22.972043+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-02-13T03:35:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:23 smithi161 bash[27879]: audit 2024-02-13T03:35:22.975493+0000 mon.a (mon.0) 2482 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-13T03:35:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:23 smithi161 bash[27879]: audit 2024-02-13T03:35:22.977294+0000 mon.b (mon.1) 420 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-13T03:35:24.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:23 smithi161 bash[19769]: cluster 2024-02-13T03:35:22.957737+0000 mon.a (mon.0) 2479 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:35:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:23 smithi161 bash[19769]: audit 2024-02-13T03:35:22.971914+0000 mon.a (mon.0) 2480 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-13T03:35:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:23 smithi161 bash[19769]: cluster 2024-02-13T03:35:22.972043+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-02-13T03:35:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:23 smithi161 bash[19769]: audit 2024-02-13T03:35:22.975493+0000 mon.a (mon.0) 2482 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-13T03:35:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:23 smithi161 bash[19769]: audit 2024-02-13T03:35:22.977294+0000 mon.b (mon.1) 420 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-13T03:35:25.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:24 smithi161 bash[27879]: cluster 2024-02-13T03:35:23.467040+0000 mgr.y (mgr.24473) 562 : cluster [DBG] pgmap v840: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:25.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:24 smithi161 bash[27879]: audit 2024-02-13T03:35:23.989075+0000 mon.a (mon.0) 2483 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-13T03:35:25.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:24 smithi161 bash[27879]: cluster 2024-02-13T03:35:23.989242+0000 mon.a (mon.0) 2484 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-02-13T03:35:25.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:24 smithi161 bash[27879]: audit 2024-02-13T03:35:24.036828+0000 mon.a (mon.0) 2485 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:25.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:24 smithi161 bash[27879]: audit 2024-02-13T03:35:24.038647+0000 mon.b (mon.1) 421 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:25.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:24 smithi161 bash[19769]: cluster 2024-02-13T03:35:23.467040+0000 mgr.y (mgr.24473) 562 : cluster [DBG] pgmap v840: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:25.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:24 smithi161 bash[19769]: audit 2024-02-13T03:35:23.989075+0000 mon.a (mon.0) 2483 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-13T03:35:25.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:24 smithi161 bash[19769]: cluster 2024-02-13T03:35:23.989242+0000 mon.a (mon.0) 2484 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-02-13T03:35:25.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:24 smithi161 bash[19769]: audit 2024-02-13T03:35:24.036828+0000 mon.a (mon.0) 2485 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:25.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:24 smithi161 bash[19769]: audit 2024-02-13T03:35:24.038647+0000 mon.b (mon.1) 421 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:24 smithi183 bash[20824]: cluster 2024-02-13T03:35:23.467040+0000 mgr.y (mgr.24473) 562 : cluster [DBG] pgmap v840: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:24 smithi183 bash[20824]: audit 2024-02-13T03:35:23.989075+0000 mon.a (mon.0) 2483 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-13T03:35:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:24 smithi183 bash[20824]: cluster 2024-02-13T03:35:23.989242+0000 mon.a (mon.0) 2484 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-02-13T03:35:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:24 smithi183 bash[20824]: audit 2024-02-13T03:35:24.036828+0000 mon.a (mon.0) 2485 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:24 smithi183 bash[20824]: audit 2024-02-13T03:35:24.038647+0000 mon.b (mon.1) 421 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:26 smithi161 bash[27879]: audit 2024-02-13T03:35:24.990153+0000 mon.a (mon.0) 2486 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:35:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:26 smithi161 bash[27879]: cluster 2024-02-13T03:35:24.990238+0000 mon.a (mon.0) 2487 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-02-13T03:35:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:26 smithi161 bash[27879]: audit 2024-02-13T03:35:24.993651+0000 mon.b (mon.1) 422 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90"}]: dispatch 2024-02-13T03:35:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:26 smithi161 bash[27879]: audit 2024-02-13T03:35:24.994032+0000 mon.a (mon.0) 2488 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90"}]: dispatch 2024-02-13T03:35:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:26 smithi161 bash[27879]: audit 2024-02-13T03:35:25.992497+0000 mon.a (mon.0) 2489 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90"}]': finished 2024-02-13T03:35:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:26 smithi161 bash[27879]: cluster 2024-02-13T03:35:25.992691+0000 mon.a (mon.0) 2490 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-02-13T03:35:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:26 smithi161 bash[19769]: audit 2024-02-13T03:35:24.990153+0000 mon.a (mon.0) 2486 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:35:26.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:26 smithi161 bash[19769]: cluster 2024-02-13T03:35:24.990238+0000 mon.a (mon.0) 2487 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-02-13T03:35:26.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:26 smithi161 bash[19769]: audit 2024-02-13T03:35:24.993651+0000 mon.b (mon.1) 422 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90"}]: dispatch 2024-02-13T03:35:26.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:26 smithi161 bash[19769]: audit 2024-02-13T03:35:24.994032+0000 mon.a (mon.0) 2488 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90"}]: dispatch 2024-02-13T03:35:26.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:26 smithi161 bash[19769]: audit 2024-02-13T03:35:25.992497+0000 mon.a (mon.0) 2489 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90"}]': finished 2024-02-13T03:35:26.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:26 smithi161 bash[19769]: cluster 2024-02-13T03:35:25.992691+0000 mon.a (mon.0) 2490 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-02-13T03:35:26.487 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:35:26 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:35:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:26 smithi183 bash[20824]: audit 2024-02-13T03:35:24.990153+0000 mon.a (mon.0) 2486 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]': finished 2024-02-13T03:35:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:26 smithi183 bash[20824]: cluster 2024-02-13T03:35:24.990238+0000 mon.a (mon.0) 2487 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-02-13T03:35:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:26 smithi183 bash[20824]: audit 2024-02-13T03:35:24.993651+0000 mon.b (mon.1) 422 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90"}]: dispatch 2024-02-13T03:35:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:26 smithi183 bash[20824]: audit 2024-02-13T03:35:24.994032+0000 mon.a (mon.0) 2488 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90"}]: dispatch 2024-02-13T03:35:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:26 smithi183 bash[20824]: audit 2024-02-13T03:35:25.992497+0000 mon.a (mon.0) 2489 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-90"}]': finished 2024-02-13T03:35:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:26 smithi183 bash[20824]: cluster 2024-02-13T03:35:25.992691+0000 mon.a (mon.0) 2490 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-02-13T03:35:27.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:27 smithi183 bash[20824]: cluster 2024-02-13T03:35:25.467852+0000 mgr.y (mgr.24473) 563 : cluster [DBG] pgmap v843: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:27 smithi183 bash[20824]: audit 2024-02-13T03:35:26.244649+0000 mgr.y (mgr.24473) 564 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:27.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:27 smithi161 bash[27879]: cluster 2024-02-13T03:35:25.467852+0000 mgr.y (mgr.24473) 563 : cluster [DBG] pgmap v843: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:27.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:27 smithi161 bash[27879]: audit 2024-02-13T03:35:26.244649+0000 mgr.y (mgr.24473) 564 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:27.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:27 smithi161 bash[19769]: cluster 2024-02-13T03:35:25.467852+0000 mgr.y (mgr.24473) 563 : cluster [DBG] pgmap v843: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:27.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:27 smithi161 bash[19769]: audit 2024-02-13T03:35:26.244649+0000 mgr.y (mgr.24473) 564 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:28 smithi183 bash[20824]: cluster 2024-02-13T03:35:27.016728+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-02-13T03:35:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:28 smithi183 bash[20824]: cluster 2024-02-13T03:35:27.468494+0000 mgr.y (mgr.24473) 565 : cluster [DBG] pgmap v846: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:35:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:28 smithi183 bash[20824]: cluster 2024-02-13T03:35:28.023184+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-02-13T03:35:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:28 smithi183 bash[20824]: audit 2024-02-13T03:35:28.028536+0000 mon.a (mon.0) 2493 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:28 smithi183 bash[20824]: audit 2024-02-13T03:35:28.029639+0000 mon.b (mon.1) 423 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:28.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:28 smithi161 bash[27879]: cluster 2024-02-13T03:35:27.016728+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-02-13T03:35:28.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:28 smithi161 bash[27879]: cluster 2024-02-13T03:35:27.468494+0000 mgr.y (mgr.24473) 565 : cluster [DBG] pgmap v846: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:35:28.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:28 smithi161 bash[27879]: cluster 2024-02-13T03:35:28.023184+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-02-13T03:35:28.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:28 smithi161 bash[27879]: audit 2024-02-13T03:35:28.028536+0000 mon.a (mon.0) 2493 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:28.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:28 smithi161 bash[27879]: audit 2024-02-13T03:35:28.029639+0000 mon.b (mon.1) 423 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:28.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:28 smithi161 bash[19769]: cluster 2024-02-13T03:35:27.016728+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-02-13T03:35:28.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:28 smithi161 bash[19769]: cluster 2024-02-13T03:35:27.468494+0000 mgr.y (mgr.24473) 565 : cluster [DBG] pgmap v846: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:35:28.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:28 smithi161 bash[19769]: cluster 2024-02-13T03:35:28.023184+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-02-13T03:35:28.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:28 smithi161 bash[19769]: audit 2024-02-13T03:35:28.028536+0000 mon.a (mon.0) 2493 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:28.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:28 smithi161 bash[19769]: audit 2024-02-13T03:35:28.029639+0000 mon.b (mon.1) 423 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:29.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:35:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:35:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:35:30.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:30 smithi161 bash[27879]: audit 2024-02-13T03:35:29.024244+0000 mon.a (mon.0) 2494 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:30.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:30 smithi161 bash[27879]: cluster 2024-02-13T03:35:29.024768+0000 mon.a (mon.0) 2495 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-02-13T03:35:30.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:30 smithi161 bash[27879]: audit 2024-02-13T03:35:29.030245+0000 mon.b (mon.1) 424 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:35:30.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:30 smithi161 bash[27879]: audit 2024-02-13T03:35:29.032592+0000 mon.a (mon.0) 2496 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:35:30.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:30 smithi161 bash[27879]: audit 2024-02-13T03:35:29.034171+0000 mon.b (mon.1) 425 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:35:30.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:30 smithi161 bash[19769]: audit 2024-02-13T03:35:29.024244+0000 mon.a (mon.0) 2494 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:30 smithi161 bash[19769]: cluster 2024-02-13T03:35:29.024768+0000 mon.a (mon.0) 2495 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-02-13T03:35:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:30 smithi161 bash[19769]: audit 2024-02-13T03:35:29.030245+0000 mon.b (mon.1) 424 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:35:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:30 smithi161 bash[19769]: audit 2024-02-13T03:35:29.032592+0000 mon.a (mon.0) 2496 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:35:30.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:30 smithi161 bash[19769]: audit 2024-02-13T03:35:29.034171+0000 mon.b (mon.1) 425 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:35:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:30 smithi183 bash[20824]: audit 2024-02-13T03:35:29.024244+0000 mon.a (mon.0) 2494 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:30 smithi183 bash[20824]: cluster 2024-02-13T03:35:29.024768+0000 mon.a (mon.0) 2495 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-02-13T03:35:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:30 smithi183 bash[20824]: audit 2024-02-13T03:35:29.030245+0000 mon.b (mon.1) 424 : audit [DBG] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-13T03:35:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:30 smithi183 bash[20824]: audit 2024-02-13T03:35:29.032592+0000 mon.a (mon.0) 2496 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:35:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:30 smithi183 bash[20824]: audit 2024-02-13T03:35:29.034171+0000 mon.b (mon.1) 425 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-13T03:35:31.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:35:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:35:31] "GET /metrics HTTP/1.1" 200 188601 "" "Prometheus/2.33.4" 2024-02-13T03:35:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:31 smithi161 bash[27879]: cluster 2024-02-13T03:35:29.469311+0000 mgr.y (mgr.24473) 566 : cluster [DBG] pgmap v849: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:35:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:31 smithi161 bash[27879]: audit 2024-02-13T03:35:30.026843+0000 mon.a (mon.0) 2497 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:35:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:31 smithi161 bash[27879]: cluster 2024-02-13T03:35:30.026999+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-02-13T03:35:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:31 smithi161 bash[27879]: audit 2024-02-13T03:35:30.028563+0000 mon.a (mon.0) 2499 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:35:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:31 smithi161 bash[27879]: audit 2024-02-13T03:35:30.030365+0000 mon.b (mon.1) 426 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:35:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:31 smithi161 bash[19769]: cluster 2024-02-13T03:35:29.469311+0000 mgr.y (mgr.24473) 566 : cluster [DBG] pgmap v849: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:35:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:31 smithi161 bash[19769]: audit 2024-02-13T03:35:30.026843+0000 mon.a (mon.0) 2497 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:35:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:31 smithi161 bash[19769]: cluster 2024-02-13T03:35:30.026999+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-02-13T03:35:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:31 smithi161 bash[19769]: audit 2024-02-13T03:35:30.028563+0000 mon.a (mon.0) 2499 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:35:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:31 smithi161 bash[19769]: audit 2024-02-13T03:35:30.030365+0000 mon.b (mon.1) 426 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:35:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:31 smithi183 bash[20824]: cluster 2024-02-13T03:35:29.469311+0000 mgr.y (mgr.24473) 566 : cluster [DBG] pgmap v849: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:35:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:31 smithi183 bash[20824]: audit 2024-02-13T03:35:30.026843+0000 mon.a (mon.0) 2497 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-13T03:35:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:31 smithi183 bash[20824]: cluster 2024-02-13T03:35:30.026999+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-02-13T03:35:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:31 smithi183 bash[20824]: audit 2024-02-13T03:35:30.028563+0000 mon.a (mon.0) 2499 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:35:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:31 smithi183 bash[20824]: audit 2024-02-13T03:35:30.030365+0000 mon.b (mon.1) 426 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-13T03:35:32.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:32 smithi161 bash[27879]: audit 2024-02-13T03:35:31.034390+0000 mon.a (mon.0) 2500 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:35:32.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:32 smithi161 bash[27879]: cluster 2024-02-13T03:35:31.034515+0000 mon.a (mon.0) 2501 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-02-13T03:35:32.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:32 smithi161 bash[27879]: audit 2024-02-13T03:35:31.036535+0000 mon.a (mon.0) 2502 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:35:32.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:32 smithi161 bash[27879]: audit 2024-02-13T03:35:31.037709+0000 mon.b (mon.1) 427 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:35:32.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:32 smithi161 bash[27879]: audit 2024-02-13T03:35:32.031631+0000 mon.a (mon.0) 2503 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:35:32.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:32 smithi161 bash[27879]: cluster 2024-02-13T03:35:32.031758+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-02-13T03:35:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:32 smithi161 bash[19769]: audit 2024-02-13T03:35:31.034390+0000 mon.a (mon.0) 2500 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:35:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:32 smithi161 bash[19769]: cluster 2024-02-13T03:35:31.034515+0000 mon.a (mon.0) 2501 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-02-13T03:35:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:32 smithi161 bash[19769]: audit 2024-02-13T03:35:31.036535+0000 mon.a (mon.0) 2502 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:35:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:32 smithi161 bash[19769]: audit 2024-02-13T03:35:31.037709+0000 mon.b (mon.1) 427 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:35:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:32 smithi161 bash[19769]: audit 2024-02-13T03:35:32.031631+0000 mon.a (mon.0) 2503 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:35:32.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:32 smithi161 bash[19769]: cluster 2024-02-13T03:35:32.031758+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-02-13T03:35:32.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:32 smithi183 bash[20824]: audit 2024-02-13T03:35:31.034390+0000 mon.a (mon.0) 2500 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-13T03:35:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:32 smithi183 bash[20824]: cluster 2024-02-13T03:35:31.034515+0000 mon.a (mon.0) 2501 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-02-13T03:35:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:32 smithi183 bash[20824]: audit 2024-02-13T03:35:31.036535+0000 mon.a (mon.0) 2502 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:35:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:32 smithi183 bash[20824]: audit 2024-02-13T03:35:31.037709+0000 mon.b (mon.1) 427 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-13T03:35:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:32 smithi183 bash[20824]: audit 2024-02-13T03:35:32.031631+0000 mon.a (mon.0) 2503 : audit [INF] from='client.26113 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-13T03:35:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:32 smithi183 bash[20824]: cluster 2024-02-13T03:35:32.031758+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-02-13T03:35:33.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:33 smithi161 bash[27879]: cluster 2024-02-13T03:35:31.470182+0000 mgr.y (mgr.24473) 567 : cluster [DBG] pgmap v852: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:33.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:33 smithi161 bash[27879]: audit 2024-02-13T03:35:32.072122+0000 mon.a (mon.0) 2505 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:33.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:33 smithi161 bash[27879]: audit 2024-02-13T03:35:32.073616+0000 mon.a (mon.0) 2506 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-92"}]: dispatch 2024-02-13T03:35:33.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:33 smithi161 bash[27879]: audit 2024-02-13T03:35:32.073884+0000 mon.b (mon.1) 428 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:33.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:33 smithi161 bash[27879]: audit 2024-02-13T03:35:32.075423+0000 mon.b (mon.1) 429 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-92"}]: dispatch 2024-02-13T03:35:33.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:33 smithi161 bash[19769]: cluster 2024-02-13T03:35:31.470182+0000 mgr.y (mgr.24473) 567 : cluster [DBG] pgmap v852: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:33.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:33 smithi161 bash[19769]: audit 2024-02-13T03:35:32.072122+0000 mon.a (mon.0) 2505 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:33.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:33 smithi161 bash[19769]: audit 2024-02-13T03:35:32.073616+0000 mon.a (mon.0) 2506 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-92"}]: dispatch 2024-02-13T03:35:33.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:33 smithi161 bash[19769]: audit 2024-02-13T03:35:32.073884+0000 mon.b (mon.1) 428 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:33.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:33 smithi161 bash[19769]: audit 2024-02-13T03:35:32.075423+0000 mon.b (mon.1) 429 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-92"}]: dispatch 2024-02-13T03:35:33.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:33 smithi183 bash[20824]: cluster 2024-02-13T03:35:31.470182+0000 mgr.y (mgr.24473) 567 : cluster [DBG] pgmap v852: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:33 smithi183 bash[20824]: audit 2024-02-13T03:35:32.072122+0000 mon.a (mon.0) 2505 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:33 smithi183 bash[20824]: audit 2024-02-13T03:35:32.073616+0000 mon.a (mon.0) 2506 : audit [INF] from='client.26113 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-92"}]: dispatch 2024-02-13T03:35:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:33 smithi183 bash[20824]: audit 2024-02-13T03:35:32.073884+0000 mon.b (mon.1) 428 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-6"}]: dispatch 2024-02-13T03:35:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:33 smithi183 bash[20824]: audit 2024-02-13T03:35:32.075423+0000 mon.b (mon.1) 429 : audit [INF] from='client.? v1:172.21.15.161:0/508217837' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-6", "tierpool": "test-rados-api-smithi161-61906-92"}]: dispatch 2024-02-13T03:35:34.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:34 smithi161 bash[27879]: cluster 2024-02-13T03:35:33.053294+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-02-13T03:35:34.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:34 smithi161 bash[19769]: cluster 2024-02-13T03:35:33.053294+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-02-13T03:35:34.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:34 smithi183 bash[20824]: cluster 2024-02-13T03:35:33.053294+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-02-13T03:35:35.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[27879]: cluster 2024-02-13T03:35:33.470971+0000 mgr.y (mgr.24473) 568 : cluster [DBG] pgmap v855: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:35.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[27879]: cluster 2024-02-13T03:35:34.074828+0000 mon.a (mon.0) 2508 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-02-13T03:35:35.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[27879]: audit 2024-02-13T03:35:34.097414+0000 mon.c (mon.2) 533 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:35.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[27879]: audit 2024-02-13T03:35:34.097818+0000 mon.a (mon.0) 2509 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:35.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[27879]: audit 2024-02-13T03:35:34.099084+0000 mon.c (mon.2) 534 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:35.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[27879]: audit 2024-02-13T03:35:34.099376+0000 mon.a (mon.0) 2510 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:35.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[27879]: audit 2024-02-13T03:35:34.099904+0000 mon.c (mon.2) 535 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:35:35.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[27879]: audit 2024-02-13T03:35:34.100204+0000 mon.a (mon.0) 2511 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:35:35.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[19769]: cluster 2024-02-13T03:35:33.470971+0000 mgr.y (mgr.24473) 568 : cluster [DBG] pgmap v855: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:35.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[19769]: cluster 2024-02-13T03:35:34.074828+0000 mon.a (mon.0) 2508 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-02-13T03:35:35.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[19769]: audit 2024-02-13T03:35:34.097414+0000 mon.c (mon.2) 533 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:35.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[19769]: audit 2024-02-13T03:35:34.097818+0000 mon.a (mon.0) 2509 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:35.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[19769]: audit 2024-02-13T03:35:34.099084+0000 mon.c (mon.2) 534 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:35.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[19769]: audit 2024-02-13T03:35:34.099376+0000 mon.a (mon.0) 2510 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:35.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[19769]: audit 2024-02-13T03:35:34.099904+0000 mon.c (mon.2) 535 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:35:35.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:35 smithi161 bash[19769]: audit 2024-02-13T03:35:34.100204+0000 mon.a (mon.0) 2511 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:35:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:35 smithi183 bash[20824]: cluster 2024-02-13T03:35:33.470971+0000 mgr.y (mgr.24473) 568 : cluster [DBG] pgmap v855: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:35 smithi183 bash[20824]: cluster 2024-02-13T03:35:34.074828+0000 mon.a (mon.0) 2508 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-02-13T03:35:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:35 smithi183 bash[20824]: audit 2024-02-13T03:35:34.097414+0000 mon.c (mon.2) 533 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:35 smithi183 bash[20824]: audit 2024-02-13T03:35:34.097818+0000 mon.a (mon.0) 2509 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:35 smithi183 bash[20824]: audit 2024-02-13T03:35:34.099084+0000 mon.c (mon.2) 534 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:35 smithi183 bash[20824]: audit 2024-02-13T03:35:34.099376+0000 mon.a (mon.0) 2510 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:35 smithi183 bash[20824]: audit 2024-02-13T03:35:34.099904+0000 mon.c (mon.2) 535 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:35:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:35 smithi183 bash[20824]: audit 2024-02-13T03:35:34.100204+0000 mon.a (mon.0) 2511 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:35:36.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:35:36 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:35:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:36 smithi161 bash[27879]: audit 2024-02-13T03:35:35.084744+0000 mon.a (mon.0) 2512 : audit [INF] from='client.48374 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:35:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:36 smithi161 bash[27879]: cluster 2024-02-13T03:35:35.084901+0000 mon.a (mon.0) 2513 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-02-13T03:35:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:36 smithi161 bash[27879]: audit 2024-02-13T03:35:35.088587+0000 mon.c (mon.2) 536 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi161-61906-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:36 smithi161 bash[27879]: audit 2024-02-13T03:35:35.089145+0000 mon.a (mon.0) 2514 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi161-61906-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:36.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:36 smithi161 bash[19769]: audit 2024-02-13T03:35:35.084744+0000 mon.a (mon.0) 2512 : audit [INF] from='client.48374 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:35:36.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:36 smithi161 bash[19769]: cluster 2024-02-13T03:35:35.084901+0000 mon.a (mon.0) 2513 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-02-13T03:35:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:36 smithi161 bash[19769]: audit 2024-02-13T03:35:35.088587+0000 mon.c (mon.2) 536 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi161-61906-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:36.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:36 smithi161 bash[19769]: audit 2024-02-13T03:35:35.089145+0000 mon.a (mon.0) 2514 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi161-61906-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:36.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:36 smithi183 bash[20824]: audit 2024-02-13T03:35:35.084744+0000 mon.a (mon.0) 2512 : audit [INF] from='client.48374 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:35:36.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:36 smithi183 bash[20824]: cluster 2024-02-13T03:35:35.084901+0000 mon.a (mon.0) 2513 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-02-13T03:35:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:36 smithi183 bash[20824]: audit 2024-02-13T03:35:35.088587+0000 mon.c (mon.2) 536 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi161-61906-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:36.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:36 smithi183 bash[20824]: audit 2024-02-13T03:35:35.089145+0000 mon.a (mon.0) 2514 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi161-61906-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:35:37.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:37 smithi161 bash[19769]: cluster 2024-02-13T03:35:35.471876+0000 mgr.y (mgr.24473) 569 : cluster [DBG] pgmap v858: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:37.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:37 smithi161 bash[19769]: audit 2024-02-13T03:35:36.251744+0000 mgr.y (mgr.24473) 570 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:37.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:37 smithi161 bash[19769]: cluster 2024-02-13T03:35:36.524832+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-02-13T03:35:37.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:37 smithi161 bash[19769]: audit 2024-02-13T03:35:37.094945+0000 mon.a (mon.0) 2516 : audit [INF] from='client.48374 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi161-61906-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi161-61906-94"}]': finished 2024-02-13T03:35:37.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:37 smithi161 bash[19769]: cluster 2024-02-13T03:35:37.095214+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-02-13T03:35:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:37 smithi161 bash[27879]: cluster 2024-02-13T03:35:35.471876+0000 mgr.y (mgr.24473) 569 : cluster [DBG] pgmap v858: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:37 smithi161 bash[27879]: audit 2024-02-13T03:35:36.251744+0000 mgr.y (mgr.24473) 570 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:37.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:37 smithi161 bash[27879]: cluster 2024-02-13T03:35:36.524832+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-02-13T03:35:37.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:37 smithi161 bash[27879]: audit 2024-02-13T03:35:37.094945+0000 mon.a (mon.0) 2516 : audit [INF] from='client.48374 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi161-61906-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi161-61906-94"}]': finished 2024-02-13T03:35:37.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:37 smithi161 bash[27879]: cluster 2024-02-13T03:35:37.095214+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-02-13T03:35:37.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:37 smithi183 bash[20824]: cluster 2024-02-13T03:35:35.471876+0000 mgr.y (mgr.24473) 569 : cluster [DBG] pgmap v858: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:37 smithi183 bash[20824]: audit 2024-02-13T03:35:36.251744+0000 mgr.y (mgr.24473) 570 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:37 smithi183 bash[20824]: cluster 2024-02-13T03:35:36.524832+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-02-13T03:35:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:37 smithi183 bash[20824]: audit 2024-02-13T03:35:37.094945+0000 mon.a (mon.0) 2516 : audit [INF] from='client.48374 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi161-61906-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi161-61906-94"}]': finished 2024-02-13T03:35:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:37 smithi183 bash[20824]: cluster 2024-02-13T03:35:37.095214+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-02-13T03:35:39.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:39 smithi161 bash[27879]: cluster 2024-02-13T03:35:37.472804+0000 mgr.y (mgr.24473) 571 : cluster [DBG] pgmap v861: 172 pgs: 8 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:39.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:39 smithi161 bash[27879]: cluster 2024-02-13T03:35:38.104548+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-02-13T03:35:39.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:39 smithi161 bash[19769]: cluster 2024-02-13T03:35:37.472804+0000 mgr.y (mgr.24473) 571 : cluster [DBG] pgmap v861: 172 pgs: 8 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:39.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:39 smithi161 bash[19769]: cluster 2024-02-13T03:35:38.104548+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-02-13T03:35:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:39 smithi183 bash[20824]: cluster 2024-02-13T03:35:37.472804+0000 mgr.y (mgr.24473) 571 : cluster [DBG] pgmap v861: 172 pgs: 8 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:39 smithi183 bash[20824]: cluster 2024-02-13T03:35:38.104548+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-02-13T03:35:39.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:35:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:35:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:35:40.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:40 smithi161 bash[27879]: cluster 2024-02-13T03:35:39.114254+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-02-13T03:35:40.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:40 smithi161 bash[27879]: audit 2024-02-13T03:35:39.119102+0000 mon.c (mon.2) 537 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:40.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:40 smithi161 bash[27879]: audit 2024-02-13T03:35:39.121390+0000 mon.a (mon.0) 2520 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:40.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:40 smithi161 bash[19769]: cluster 2024-02-13T03:35:39.114254+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-02-13T03:35:40.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:40 smithi161 bash[19769]: audit 2024-02-13T03:35:39.119102+0000 mon.c (mon.2) 537 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:40.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:40 smithi161 bash[19769]: audit 2024-02-13T03:35:39.121390+0000 mon.a (mon.0) 2520 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:40.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:40 smithi183 bash[20824]: cluster 2024-02-13T03:35:39.114254+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-02-13T03:35:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:40 smithi183 bash[20824]: audit 2024-02-13T03:35:39.119102+0000 mon.c (mon.2) 537 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:40 smithi183 bash[20824]: audit 2024-02-13T03:35:39.121390+0000 mon.a (mon.0) 2520 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:41.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:35:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:35:41] "GET /metrics HTTP/1.1" 200 183761 "" "Prometheus/2.33.4" 2024-02-13T03:35:41.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:41 smithi161 bash[27879]: cluster 2024-02-13T03:35:39.473696+0000 mgr.y (mgr.24473) 572 : cluster [DBG] pgmap v864: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:41.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:41 smithi161 bash[27879]: audit 2024-02-13T03:35:40.118902+0000 mon.a (mon.0) 2521 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:41.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:41 smithi161 bash[27879]: cluster 2024-02-13T03:35:40.119273+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-02-13T03:35:41.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:41 smithi161 bash[19769]: cluster 2024-02-13T03:35:39.473696+0000 mgr.y (mgr.24473) 572 : cluster [DBG] pgmap v864: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:41.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:41 smithi161 bash[19769]: audit 2024-02-13T03:35:40.118902+0000 mon.a (mon.0) 2521 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:41.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:41 smithi161 bash[19769]: cluster 2024-02-13T03:35:40.119273+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-02-13T03:35:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:41 smithi183 bash[20824]: cluster 2024-02-13T03:35:39.473696+0000 mgr.y (mgr.24473) 572 : cluster [DBG] pgmap v864: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:41 smithi183 bash[20824]: audit 2024-02-13T03:35:40.118902+0000 mon.a (mon.0) 2521 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:41 smithi183 bash[20824]: cluster 2024-02-13T03:35:40.119273+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-02-13T03:35:42.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[27879]: cluster 2024-02-13T03:35:41.136735+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-02-13T03:35:42.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[27879]: audit 2024-02-13T03:35:41.139179+0000 mon.c (mon.2) 538 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:42.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[27879]: audit 2024-02-13T03:35:41.143952+0000 mon.a (mon.0) 2524 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:42.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[27879]: audit 2024-02-13T03:35:42.137552+0000 mon.a (mon.0) 2525 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:42.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[27879]: cluster 2024-02-13T03:35:42.137688+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-02-13T03:35:42.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[27879]: audit 2024-02-13T03:35:42.141938+0000 mon.c (mon.2) 539 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:42.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[27879]: audit 2024-02-13T03:35:42.143054+0000 mon.a (mon.0) 2527 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:42.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[19769]: cluster 2024-02-13T03:35:41.136735+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-02-13T03:35:42.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[19769]: audit 2024-02-13T03:35:41.139179+0000 mon.c (mon.2) 538 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:42.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[19769]: audit 2024-02-13T03:35:41.143952+0000 mon.a (mon.0) 2524 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:42.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[19769]: audit 2024-02-13T03:35:42.137552+0000 mon.a (mon.0) 2525 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:42.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[19769]: cluster 2024-02-13T03:35:42.137688+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-02-13T03:35:42.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[19769]: audit 2024-02-13T03:35:42.141938+0000 mon.c (mon.2) 539 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:42.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:42 smithi161 bash[19769]: audit 2024-02-13T03:35:42.143054+0000 mon.a (mon.0) 2527 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:42 smithi183 bash[20824]: cluster 2024-02-13T03:35:41.136735+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-02-13T03:35:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:42 smithi183 bash[20824]: audit 2024-02-13T03:35:41.139179+0000 mon.c (mon.2) 538 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:42 smithi183 bash[20824]: audit 2024-02-13T03:35:41.143952+0000 mon.a (mon.0) 2524 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:42 smithi183 bash[20824]: audit 2024-02-13T03:35:42.137552+0000 mon.a (mon.0) 2525 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:42 smithi183 bash[20824]: cluster 2024-02-13T03:35:42.137688+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-02-13T03:35:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:42 smithi183 bash[20824]: audit 2024-02-13T03:35:42.141938+0000 mon.c (mon.2) 539 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:42 smithi183 bash[20824]: audit 2024-02-13T03:35:42.143054+0000 mon.a (mon.0) 2527 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:43.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:43 smithi161 bash[27879]: cluster 2024-02-13T03:35:41.474600+0000 mgr.y (mgr.24473) 573 : cluster [DBG] pgmap v867: 236 pgs: 32 unknown, 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:35:43.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:43 smithi161 bash[27879]: cluster 2024-02-13T03:35:42.145185+0000 mon.a (mon.0) 2528 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:35:43.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:43 smithi161 bash[27879]: audit 2024-02-13T03:35:43.143925+0000 mon.a (mon.0) 2529 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]': finished 2024-02-13T03:35:43.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:43 smithi161 bash[27879]: cluster 2024-02-13T03:35:43.144118+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-02-13T03:35:43.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:43 smithi161 bash[27879]: audit 2024-02-13T03:35:43.145900+0000 mon.c (mon.2) 540 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-97", "overlaypool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:43.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:43 smithi161 bash[27879]: audit 2024-02-13T03:35:43.147057+0000 mon.a (mon.0) 2531 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-97", "overlaypool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:43.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:43 smithi161 bash[19769]: cluster 2024-02-13T03:35:41.474600+0000 mgr.y (mgr.24473) 573 : cluster [DBG] pgmap v867: 236 pgs: 32 unknown, 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:35:43.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:43 smithi161 bash[19769]: cluster 2024-02-13T03:35:42.145185+0000 mon.a (mon.0) 2528 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:35:43.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:43 smithi161 bash[19769]: audit 2024-02-13T03:35:43.143925+0000 mon.a (mon.0) 2529 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]': finished 2024-02-13T03:35:43.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:43 smithi161 bash[19769]: cluster 2024-02-13T03:35:43.144118+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-02-13T03:35:43.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:43 smithi161 bash[19769]: audit 2024-02-13T03:35:43.145900+0000 mon.c (mon.2) 540 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-97", "overlaypool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:43.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:43 smithi161 bash[19769]: audit 2024-02-13T03:35:43.147057+0000 mon.a (mon.0) 2531 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-97", "overlaypool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:43.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:43 smithi183 bash[20824]: cluster 2024-02-13T03:35:41.474600+0000 mgr.y (mgr.24473) 573 : cluster [DBG] pgmap v867: 236 pgs: 32 unknown, 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:35:43.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:43 smithi183 bash[20824]: cluster 2024-02-13T03:35:42.145185+0000 mon.a (mon.0) 2528 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:35:43.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:43 smithi183 bash[20824]: audit 2024-02-13T03:35:43.143925+0000 mon.a (mon.0) 2529 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]': finished 2024-02-13T03:35:43.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:43 smithi183 bash[20824]: cluster 2024-02-13T03:35:43.144118+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-02-13T03:35:43.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:43 smithi183 bash[20824]: audit 2024-02-13T03:35:43.145900+0000 mon.c (mon.2) 540 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-97", "overlaypool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:43.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:43 smithi183 bash[20824]: audit 2024-02-13T03:35:43.147057+0000 mon.a (mon.0) 2531 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-97", "overlaypool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:45.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:45 smithi161 bash[27879]: cluster 2024-02-13T03:35:43.475493+0000 mgr.y (mgr.24473) 574 : cluster [DBG] pgmap v870: 236 pgs: 32 unknown, 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:35:45.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:45 smithi161 bash[27879]: audit 2024-02-13T03:35:44.142917+0000 mon.a (mon.0) 2532 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-97", "overlaypool": "test-rados-api-smithi161-61906-97-cache"}]': finished 2024-02-13T03:35:45.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:45 smithi161 bash[27879]: cluster 2024-02-13T03:35:44.143014+0000 mon.a (mon.0) 2533 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-02-13T03:35:45.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:45 smithi161 bash[27879]: audit 2024-02-13T03:35:44.146692+0000 mon.c (mon.2) 541 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-97-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:35:45.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:45 smithi161 bash[27879]: audit 2024-02-13T03:35:44.147066+0000 mon.a (mon.0) 2534 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-97-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:35:45.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:45 smithi161 bash[19769]: cluster 2024-02-13T03:35:43.475493+0000 mgr.y (mgr.24473) 574 : cluster [DBG] pgmap v870: 236 pgs: 32 unknown, 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:35:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:45 smithi161 bash[19769]: audit 2024-02-13T03:35:44.142917+0000 mon.a (mon.0) 2532 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-97", "overlaypool": "test-rados-api-smithi161-61906-97-cache"}]': finished 2024-02-13T03:35:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:45 smithi161 bash[19769]: cluster 2024-02-13T03:35:44.143014+0000 mon.a (mon.0) 2533 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-02-13T03:35:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:45 smithi161 bash[19769]: audit 2024-02-13T03:35:44.146692+0000 mon.c (mon.2) 541 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-97-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:35:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:45 smithi161 bash[19769]: audit 2024-02-13T03:35:44.147066+0000 mon.a (mon.0) 2534 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-97-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:35:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:45 smithi183 bash[20824]: cluster 2024-02-13T03:35:43.475493+0000 mgr.y (mgr.24473) 574 : cluster [DBG] pgmap v870: 236 pgs: 32 unknown, 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:35:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:45 smithi183 bash[20824]: audit 2024-02-13T03:35:44.142917+0000 mon.a (mon.0) 2532 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-97", "overlaypool": "test-rados-api-smithi161-61906-97-cache"}]': finished 2024-02-13T03:35:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:45 smithi183 bash[20824]: cluster 2024-02-13T03:35:44.143014+0000 mon.a (mon.0) 2533 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-02-13T03:35:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:45 smithi183 bash[20824]: audit 2024-02-13T03:35:44.146692+0000 mon.c (mon.2) 541 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-97-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:35:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:45 smithi183 bash[20824]: audit 2024-02-13T03:35:44.147066+0000 mon.a (mon.0) 2534 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-97-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:35:46.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:46 smithi161 bash[27879]: cluster 2024-02-13T03:35:45.137309+0000 mon.a (mon.0) 2535 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:35:46.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:46 smithi161 bash[27879]: audit 2024-02-13T03:35:45.147602+0000 mon.a (mon.0) 2536 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-97-cache", "mode": "writeback"}]': finished 2024-02-13T03:35:46.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:46 smithi161 bash[27879]: cluster 2024-02-13T03:35:45.147713+0000 mon.a (mon.0) 2537 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-02-13T03:35:46.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:46 smithi161 bash[27879]: audit 2024-02-13T03:35:45.171896+0000 mon.c (mon.2) 542 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-97"}]: dispatch 2024-02-13T03:35:46.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:46 smithi161 bash[27879]: audit 2024-02-13T03:35:45.172289+0000 mon.a (mon.0) 2538 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-97"}]: dispatch 2024-02-13T03:35:46.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:46 smithi161 bash[19769]: cluster 2024-02-13T03:35:45.137309+0000 mon.a (mon.0) 2535 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:35:46.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:46 smithi161 bash[19769]: audit 2024-02-13T03:35:45.147602+0000 mon.a (mon.0) 2536 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-97-cache", "mode": "writeback"}]': finished 2024-02-13T03:35:46.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:46 smithi161 bash[19769]: cluster 2024-02-13T03:35:45.147713+0000 mon.a (mon.0) 2537 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-02-13T03:35:46.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:46 smithi161 bash[19769]: audit 2024-02-13T03:35:45.171896+0000 mon.c (mon.2) 542 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-97"}]: dispatch 2024-02-13T03:35:46.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:46 smithi161 bash[19769]: audit 2024-02-13T03:35:45.172289+0000 mon.a (mon.0) 2538 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-97"}]: dispatch 2024-02-13T03:35:46.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:35:46 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:35:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:46 smithi183 bash[20824]: cluster 2024-02-13T03:35:45.137309+0000 mon.a (mon.0) 2535 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:35:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:46 smithi183 bash[20824]: audit 2024-02-13T03:35:45.147602+0000 mon.a (mon.0) 2536 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-97-cache", "mode": "writeback"}]': finished 2024-02-13T03:35:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:46 smithi183 bash[20824]: cluster 2024-02-13T03:35:45.147713+0000 mon.a (mon.0) 2537 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-02-13T03:35:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:46 smithi183 bash[20824]: audit 2024-02-13T03:35:45.171896+0000 mon.c (mon.2) 542 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-97"}]: dispatch 2024-02-13T03:35:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:46 smithi183 bash[20824]: audit 2024-02-13T03:35:45.172289+0000 mon.a (mon.0) 2538 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-97"}]: dispatch 2024-02-13T03:35:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:47 smithi183 bash[20824]: cluster 2024-02-13T03:35:45.476285+0000 mgr.y (mgr.24473) 575 : cluster [DBG] pgmap v873: 236 pgs: 32 unknown, 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:47 smithi183 bash[20824]: audit 2024-02-13T03:35:46.161256+0000 mon.a (mon.0) 2539 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-97"}]': finished 2024-02-13T03:35:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:47 smithi183 bash[20824]: cluster 2024-02-13T03:35:46.161428+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-02-13T03:35:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:47 smithi183 bash[20824]: audit 2024-02-13T03:35:46.168404+0000 mon.c (mon.2) 543 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:47 smithi183 bash[20824]: audit 2024-02-13T03:35:46.175348+0000 mon.a (mon.0) 2541 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:47.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:47 smithi183 bash[20824]: audit 2024-02-13T03:35:46.258289+0000 mgr.y (mgr.24473) 576 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:47 smithi161 bash[27879]: cluster 2024-02-13T03:35:45.476285+0000 mgr.y (mgr.24473) 575 : cluster [DBG] pgmap v873: 236 pgs: 32 unknown, 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:47 smithi161 bash[27879]: audit 2024-02-13T03:35:46.161256+0000 mon.a (mon.0) 2539 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-97"}]': finished 2024-02-13T03:35:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:47 smithi161 bash[27879]: cluster 2024-02-13T03:35:46.161428+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-02-13T03:35:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:47 smithi161 bash[27879]: audit 2024-02-13T03:35:46.168404+0000 mon.c (mon.2) 543 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:47 smithi161 bash[27879]: audit 2024-02-13T03:35:46.175348+0000 mon.a (mon.0) 2541 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:47 smithi161 bash[27879]: audit 2024-02-13T03:35:46.258289+0000 mgr.y (mgr.24473) 576 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:47 smithi161 bash[19769]: cluster 2024-02-13T03:35:45.476285+0000 mgr.y (mgr.24473) 575 : cluster [DBG] pgmap v873: 236 pgs: 32 unknown, 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:47 smithi161 bash[19769]: audit 2024-02-13T03:35:46.161256+0000 mon.a (mon.0) 2539 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-97"}]': finished 2024-02-13T03:35:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:47 smithi161 bash[19769]: cluster 2024-02-13T03:35:46.161428+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-02-13T03:35:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:47 smithi161 bash[19769]: audit 2024-02-13T03:35:46.168404+0000 mon.c (mon.2) 543 : audit [INF] from='client.? v1:172.21.15.161:0/4052452575' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:47 smithi161 bash[19769]: audit 2024-02-13T03:35:46.175348+0000 mon.a (mon.0) 2541 : audit [INF] from='client.48380 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]: dispatch 2024-02-13T03:35:47.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:47 smithi161 bash[19769]: audit 2024-02-13T03:35:46.258289+0000 mgr.y (mgr.24473) 576 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:48 smithi183 bash[20824]: cluster 2024-02-13T03:35:47.171223+0000 mon.a (mon.0) 2542 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:35:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:48 smithi183 bash[20824]: audit 2024-02-13T03:35:47.179333+0000 mon.a (mon.0) 2543 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]': finished 2024-02-13T03:35:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:48 smithi183 bash[20824]: cluster 2024-02-13T03:35:47.179449+0000 mon.a (mon.0) 2544 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-02-13T03:35:48.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:48 smithi161 bash[27879]: cluster 2024-02-13T03:35:47.171223+0000 mon.a (mon.0) 2542 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:35:48.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:48 smithi161 bash[27879]: audit 2024-02-13T03:35:47.179333+0000 mon.a (mon.0) 2543 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]': finished 2024-02-13T03:35:48.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:48 smithi161 bash[27879]: cluster 2024-02-13T03:35:47.179449+0000 mon.a (mon.0) 2544 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-02-13T03:35:48.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:48 smithi161 bash[19769]: cluster 2024-02-13T03:35:47.171223+0000 mon.a (mon.0) 2542 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:35:48.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:48 smithi161 bash[19769]: audit 2024-02-13T03:35:47.179333+0000 mon.a (mon.0) 2543 : audit [INF] from='client.48380 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-97", "tierpool": "test-rados-api-smithi161-61906-97-cache"}]': finished 2024-02-13T03:35:48.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:48 smithi161 bash[19769]: cluster 2024-02-13T03:35:47.179449+0000 mon.a (mon.0) 2544 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-02-13T03:35:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:49 smithi183 bash[20824]: cluster 2024-02-13T03:35:47.477185+0000 mgr.y (mgr.24473) 577 : cluster [DBG] pgmap v876: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:35:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:49 smithi183 bash[20824]: cluster 2024-02-13T03:35:48.194701+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-02-13T03:35:49.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:35:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:35:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:35:49.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:49 smithi161 bash[19769]: cluster 2024-02-13T03:35:47.477185+0000 mgr.y (mgr.24473) 577 : cluster [DBG] pgmap v876: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:35:49.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:49 smithi161 bash[19769]: cluster 2024-02-13T03:35:48.194701+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-02-13T03:35:49.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:49 smithi161 bash[27879]: cluster 2024-02-13T03:35:47.477185+0000 mgr.y (mgr.24473) 577 : cluster [DBG] pgmap v876: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:35:49.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:49 smithi161 bash[27879]: cluster 2024-02-13T03:35:48.194701+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-02-13T03:35:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:50 smithi183 bash[20824]: cluster 2024-02-13T03:35:49.216145+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-02-13T03:35:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:50 smithi183 bash[20824]: audit 2024-02-13T03:35:49.242035+0000 mon.c (mon.2) 544 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:50 smithi183 bash[20824]: audit 2024-02-13T03:35:49.242414+0000 mon.a (mon.0) 2547 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:50 smithi183 bash[20824]: audit 2024-02-13T03:35:49.244081+0000 mon.c (mon.2) 545 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:50 smithi183 bash[20824]: audit 2024-02-13T03:35:49.244415+0000 mon.a (mon.0) 2548 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:50 smithi183 bash[20824]: audit 2024-02-13T03:35:49.244949+0000 mon.c (mon.2) 546 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:35:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:50 smithi183 bash[20824]: audit 2024-02-13T03:35:49.245257+0000 mon.a (mon.0) 2549 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:35:50.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[27879]: cluster 2024-02-13T03:35:49.216145+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-02-13T03:35:50.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[27879]: audit 2024-02-13T03:35:49.242035+0000 mon.c (mon.2) 544 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:50.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[27879]: audit 2024-02-13T03:35:49.242414+0000 mon.a (mon.0) 2547 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:50.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[27879]: audit 2024-02-13T03:35:49.244081+0000 mon.c (mon.2) 545 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:50.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[27879]: audit 2024-02-13T03:35:49.244415+0000 mon.a (mon.0) 2548 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:50.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[27879]: audit 2024-02-13T03:35:49.244949+0000 mon.c (mon.2) 546 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:35:50.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[27879]: audit 2024-02-13T03:35:49.245257+0000 mon.a (mon.0) 2549 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:35:50.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[19769]: cluster 2024-02-13T03:35:49.216145+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-02-13T03:35:50.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[19769]: audit 2024-02-13T03:35:49.242035+0000 mon.c (mon.2) 544 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:50.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[19769]: audit 2024-02-13T03:35:49.242414+0000 mon.a (mon.0) 2547 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:50.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[19769]: audit 2024-02-13T03:35:49.244081+0000 mon.c (mon.2) 545 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:50.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[19769]: audit 2024-02-13T03:35:49.244415+0000 mon.a (mon.0) 2548 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:50.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[19769]: audit 2024-02-13T03:35:49.244949+0000 mon.c (mon.2) 546 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:35:50.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:50 smithi161 bash[19769]: audit 2024-02-13T03:35:49.245257+0000 mon.a (mon.0) 2549 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:35:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:35:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:35:51] "GET /metrics HTTP/1.1" 200 186300 "" "Prometheus/2.33.4" 2024-02-13T03:35:51.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:51 smithi161 bash[27879]: cluster 2024-02-13T03:35:49.478014+0000 mgr.y (mgr.24473) 578 : cluster [DBG] pgmap v879: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:51.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:51 smithi161 bash[27879]: audit 2024-02-13T03:35:50.236574+0000 mon.a (mon.0) 2550 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:35:51.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:51 smithi161 bash[27879]: cluster 2024-02-13T03:35:50.236684+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-02-13T03:35:51.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:51 smithi161 bash[27879]: audit 2024-02-13T03:35:50.238053+0000 mon.c (mon.2) 547 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:51.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:51 smithi161 bash[27879]: audit 2024-02-13T03:35:50.238603+0000 mon.a (mon.0) 2552 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:51.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:51 smithi161 bash[19769]: cluster 2024-02-13T03:35:49.478014+0000 mgr.y (mgr.24473) 578 : cluster [DBG] pgmap v879: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:51.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:51 smithi161 bash[19769]: audit 2024-02-13T03:35:50.236574+0000 mon.a (mon.0) 2550 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:35:51.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:51 smithi161 bash[19769]: cluster 2024-02-13T03:35:50.236684+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-02-13T03:35:51.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:51 smithi161 bash[19769]: audit 2024-02-13T03:35:50.238053+0000 mon.c (mon.2) 547 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:51.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:51 smithi161 bash[19769]: audit 2024-02-13T03:35:50.238603+0000 mon.a (mon.0) 2552 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:51 smithi183 bash[20824]: cluster 2024-02-13T03:35:49.478014+0000 mgr.y (mgr.24473) 578 : cluster [DBG] pgmap v879: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:51 smithi183 bash[20824]: audit 2024-02-13T03:35:50.236574+0000 mon.a (mon.0) 2550 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:35:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:51 smithi183 bash[20824]: cluster 2024-02-13T03:35:50.236684+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-02-13T03:35:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:51 smithi183 bash[20824]: audit 2024-02-13T03:35:50.238053+0000 mon.c (mon.2) 547 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:51.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:51 smithi183 bash[20824]: audit 2024-02-13T03:35:50.238603+0000 mon.a (mon.0) 2552 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:35:52.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:52 smithi183 bash[20824]: cluster 2024-02-13T03:35:51.656433+0000 mon.a (mon.0) 2553 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-02-13T03:35:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:52 smithi183 bash[20824]: audit 2024-02-13T03:35:52.239565+0000 mon.a (mon.0) 2554 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-99"}]': finished 2024-02-13T03:35:52.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:52 smithi183 bash[20824]: cluster 2024-02-13T03:35:52.239815+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-02-13T03:35:53.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:52 smithi161 bash[27879]: cluster 2024-02-13T03:35:51.656433+0000 mon.a (mon.0) 2553 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-02-13T03:35:53.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:52 smithi161 bash[27879]: audit 2024-02-13T03:35:52.239565+0000 mon.a (mon.0) 2554 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-99"}]': finished 2024-02-13T03:35:53.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:52 smithi161 bash[27879]: cluster 2024-02-13T03:35:52.239815+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-02-13T03:35:53.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:52 smithi161 bash[19769]: cluster 2024-02-13T03:35:51.656433+0000 mon.a (mon.0) 2553 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-02-13T03:35:53.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:52 smithi161 bash[19769]: audit 2024-02-13T03:35:52.239565+0000 mon.a (mon.0) 2554 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-99"}]': finished 2024-02-13T03:35:53.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:52 smithi161 bash[19769]: cluster 2024-02-13T03:35:52.239815+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-02-13T03:35:53.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:53 smithi183 bash[20824]: cluster 2024-02-13T03:35:51.478717+0000 mgr.y (mgr.24473) 579 : cluster [DBG] pgmap v882: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:53 smithi183 bash[20824]: cluster 2024-02-13T03:35:53.245726+0000 mon.a (mon.0) 2556 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-02-13T03:35:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:53 smithi183 bash[20824]: audit 2024-02-13T03:35:53.247371+0000 mon.c (mon.2) 548 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:53 smithi183 bash[20824]: audit 2024-02-13T03:35:53.250266+0000 mon.a (mon.0) 2557 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:54.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:53 smithi161 bash[27879]: cluster 2024-02-13T03:35:51.478717+0000 mgr.y (mgr.24473) 579 : cluster [DBG] pgmap v882: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:54.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:53 smithi161 bash[27879]: cluster 2024-02-13T03:35:53.245726+0000 mon.a (mon.0) 2556 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-02-13T03:35:54.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:53 smithi161 bash[27879]: audit 2024-02-13T03:35:53.247371+0000 mon.c (mon.2) 548 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:54.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:53 smithi161 bash[27879]: audit 2024-02-13T03:35:53.250266+0000 mon.a (mon.0) 2557 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:54.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:53 smithi161 bash[19769]: cluster 2024-02-13T03:35:51.478717+0000 mgr.y (mgr.24473) 579 : cluster [DBG] pgmap v882: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:54.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:53 smithi161 bash[19769]: cluster 2024-02-13T03:35:53.245726+0000 mon.a (mon.0) 2556 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-02-13T03:35:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:53 smithi161 bash[19769]: audit 2024-02-13T03:35:53.247371+0000 mon.c (mon.2) 548 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:54.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:53 smithi161 bash[19769]: audit 2024-02-13T03:35:53.250266+0000 mon.a (mon.0) 2557 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:35:55.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:55 smithi161 bash[27879]: cluster 2024-02-13T03:35:53.479573+0000 mgr.y (mgr.24473) 580 : cluster [DBG] pgmap v885: 212 pgs: 40 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:55.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:55 smithi161 bash[27879]: audit 2024-02-13T03:35:54.246073+0000 mon.a (mon.0) 2558 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:55.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:55 smithi161 bash[27879]: cluster 2024-02-13T03:35:54.246405+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-02-13T03:35:55.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:55 smithi161 bash[27879]: audit 2024-02-13T03:35:54.256480+0000 mon.c (mon.2) 549 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:35:55.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:55 smithi161 bash[27879]: audit 2024-02-13T03:35:54.257953+0000 mon.a (mon.0) 2560 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:35:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:55 smithi161 bash[19769]: cluster 2024-02-13T03:35:53.479573+0000 mgr.y (mgr.24473) 580 : cluster [DBG] pgmap v885: 212 pgs: 40 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:55 smithi161 bash[19769]: audit 2024-02-13T03:35:54.246073+0000 mon.a (mon.0) 2558 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:55 smithi161 bash[19769]: cluster 2024-02-13T03:35:54.246405+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-02-13T03:35:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:55 smithi161 bash[19769]: audit 2024-02-13T03:35:54.256480+0000 mon.c (mon.2) 549 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:35:55.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:55 smithi161 bash[19769]: audit 2024-02-13T03:35:54.257953+0000 mon.a (mon.0) 2560 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:35:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:55 smithi183 bash[20824]: cluster 2024-02-13T03:35:53.479573+0000 mgr.y (mgr.24473) 580 : cluster [DBG] pgmap v885: 212 pgs: 40 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:55 smithi183 bash[20824]: audit 2024-02-13T03:35:54.246073+0000 mon.a (mon.0) 2558 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:35:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:55 smithi183 bash[20824]: cluster 2024-02-13T03:35:54.246405+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-02-13T03:35:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:55 smithi183 bash[20824]: audit 2024-02-13T03:35:54.256480+0000 mon.c (mon.2) 549 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:35:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:55 smithi183 bash[20824]: audit 2024-02-13T03:35:54.257953+0000 mon.a (mon.0) 2560 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:35:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:56 smithi161 bash[27879]: audit 2024-02-13T03:35:55.259198+0000 mon.a (mon.0) 2561 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]': finished 2024-02-13T03:35:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:56 smithi161 bash[27879]: cluster 2024-02-13T03:35:55.259381+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-02-13T03:35:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:56 smithi161 bash[27879]: audit 2024-02-13T03:35:55.263620+0000 mon.c (mon.2) 550 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-99", "overlaypool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:35:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:56 smithi161 bash[27879]: audit 2024-02-13T03:35:55.264283+0000 mon.a (mon.0) 2563 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-99", "overlaypool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:35:56.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:56 smithi161 bash[19769]: audit 2024-02-13T03:35:55.259198+0000 mon.a (mon.0) 2561 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]': finished 2024-02-13T03:35:56.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:56 smithi161 bash[19769]: cluster 2024-02-13T03:35:55.259381+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-02-13T03:35:56.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:56 smithi161 bash[19769]: audit 2024-02-13T03:35:55.263620+0000 mon.c (mon.2) 550 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-99", "overlaypool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:35:56.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:56 smithi161 bash[19769]: audit 2024-02-13T03:35:55.264283+0000 mon.a (mon.0) 2563 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-99", "overlaypool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:35:56.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:35:56 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:35:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:56 smithi183 bash[20824]: audit 2024-02-13T03:35:55.259198+0000 mon.a (mon.0) 2561 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]': finished 2024-02-13T03:35:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:56 smithi183 bash[20824]: cluster 2024-02-13T03:35:55.259381+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-02-13T03:35:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:56 smithi183 bash[20824]: audit 2024-02-13T03:35:55.263620+0000 mon.c (mon.2) 550 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-99", "overlaypool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:35:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:56 smithi183 bash[20824]: audit 2024-02-13T03:35:55.264283+0000 mon.a (mon.0) 2563 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-99", "overlaypool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:35:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:57 smithi161 bash[27879]: cluster 2024-02-13T03:35:55.480394+0000 mgr.y (mgr.24473) 581 : cluster [DBG] pgmap v888: 212 pgs: 40 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:57 smithi161 bash[27879]: audit 2024-02-13T03:35:56.261522+0000 mgr.y (mgr.24473) 582 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:57 smithi161 bash[27879]: audit 2024-02-13T03:35:56.270256+0000 mon.a (mon.0) 2564 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-99", "overlaypool": "test-rados-api-smithi161-61906-99-cache"}]': finished 2024-02-13T03:35:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:57 smithi161 bash[27879]: cluster 2024-02-13T03:35:56.270376+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-02-13T03:35:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:57 smithi161 bash[27879]: audit 2024-02-13T03:35:56.275132+0000 mon.c (mon.2) 551 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-99-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:35:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:57 smithi161 bash[27879]: audit 2024-02-13T03:35:56.276064+0000 mon.a (mon.0) 2566 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-99-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:35:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:57 smithi161 bash[19769]: cluster 2024-02-13T03:35:55.480394+0000 mgr.y (mgr.24473) 581 : cluster [DBG] pgmap v888: 212 pgs: 40 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:57 smithi161 bash[19769]: audit 2024-02-13T03:35:56.261522+0000 mgr.y (mgr.24473) 582 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:57 smithi161 bash[19769]: audit 2024-02-13T03:35:56.270256+0000 mon.a (mon.0) 2564 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-99", "overlaypool": "test-rados-api-smithi161-61906-99-cache"}]': finished 2024-02-13T03:35:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:57 smithi161 bash[19769]: cluster 2024-02-13T03:35:56.270376+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-02-13T03:35:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:57 smithi161 bash[19769]: audit 2024-02-13T03:35:56.275132+0000 mon.c (mon.2) 551 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-99-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:35:57.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:57 smithi161 bash[19769]: audit 2024-02-13T03:35:56.276064+0000 mon.a (mon.0) 2566 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-99-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:35:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:57 smithi183 bash[20824]: cluster 2024-02-13T03:35:55.480394+0000 mgr.y (mgr.24473) 581 : cluster [DBG] pgmap v888: 212 pgs: 40 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:35:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:57 smithi183 bash[20824]: audit 2024-02-13T03:35:56.261522+0000 mgr.y (mgr.24473) 582 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:35:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:57 smithi183 bash[20824]: audit 2024-02-13T03:35:56.270256+0000 mon.a (mon.0) 2564 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-99", "overlaypool": "test-rados-api-smithi161-61906-99-cache"}]': finished 2024-02-13T03:35:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:57 smithi183 bash[20824]: cluster 2024-02-13T03:35:56.270376+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-02-13T03:35:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:57 smithi183 bash[20824]: audit 2024-02-13T03:35:56.275132+0000 mon.c (mon.2) 551 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-99-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:35:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:57 smithi183 bash[20824]: audit 2024-02-13T03:35:56.276064+0000 mon.a (mon.0) 2566 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-99-cache", "mode": "writeback"}]: dispatch 2024-02-13T03:35:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:58 smithi161 bash[27879]: cluster 2024-02-13T03:35:57.267181+0000 mon.a (mon.0) 2567 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:35:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:58 smithi161 bash[27879]: audit 2024-02-13T03:35:57.277042+0000 mon.a (mon.0) 2568 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-99-cache", "mode": "writeback"}]': finished 2024-02-13T03:35:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:58 smithi161 bash[27879]: cluster 2024-02-13T03:35:57.277269+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-02-13T03:35:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:58 smithi161 bash[27879]: audit 2024-02-13T03:35:57.282960+0000 mon.c (mon.2) 552 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:35:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:58 smithi161 bash[27879]: audit 2024-02-13T03:35:57.289332+0000 mon.a (mon.0) 2570 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:35:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:58 smithi161 bash[19769]: cluster 2024-02-13T03:35:57.267181+0000 mon.a (mon.0) 2567 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:35:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:58 smithi161 bash[19769]: audit 2024-02-13T03:35:57.277042+0000 mon.a (mon.0) 2568 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-99-cache", "mode": "writeback"}]': finished 2024-02-13T03:35:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:58 smithi161 bash[19769]: cluster 2024-02-13T03:35:57.277269+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-02-13T03:35:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:58 smithi161 bash[19769]: audit 2024-02-13T03:35:57.282960+0000 mon.c (mon.2) 552 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:35:58.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:58 smithi161 bash[19769]: audit 2024-02-13T03:35:57.289332+0000 mon.a (mon.0) 2570 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:35:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:58 smithi183 bash[20824]: cluster 2024-02-13T03:35:57.267181+0000 mon.a (mon.0) 2567 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:35:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:58 smithi183 bash[20824]: audit 2024-02-13T03:35:57.277042+0000 mon.a (mon.0) 2568 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-99-cache", "mode": "writeback"}]': finished 2024-02-13T03:35:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:58 smithi183 bash[20824]: cluster 2024-02-13T03:35:57.277269+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-02-13T03:35:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:58 smithi183 bash[20824]: audit 2024-02-13T03:35:57.282960+0000 mon.c (mon.2) 552 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:35:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:58 smithi183 bash[20824]: audit 2024-02-13T03:35:57.289332+0000 mon.a (mon.0) 2570 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:35:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:59 smithi161 bash[27879]: cluster 2024-02-13T03:35:57.481255+0000 mgr.y (mgr.24473) 583 : cluster [DBG] pgmap v891: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:59 smithi161 bash[27879]: audit 2024-02-13T03:35:58.290633+0000 mon.a (mon.0) 2571 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:35:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:59 smithi161 bash[27879]: cluster 2024-02-13T03:35:58.290729+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-02-13T03:35:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:59 smithi161 bash[27879]: audit 2024-02-13T03:35:58.292838+0000 mon.c (mon.2) 553 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:35:59 smithi161 bash[27879]: audit 2024-02-13T03:35:58.298750+0000 mon.a (mon.0) 2573 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:59 smithi161 bash[19769]: cluster 2024-02-13T03:35:57.481255+0000 mgr.y (mgr.24473) 583 : cluster [DBG] pgmap v891: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:59 smithi161 bash[19769]: audit 2024-02-13T03:35:58.290633+0000 mon.a (mon.0) 2571 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:35:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:59 smithi161 bash[19769]: cluster 2024-02-13T03:35:58.290729+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-02-13T03:35:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:59 smithi161 bash[19769]: audit 2024-02-13T03:35:58.292838+0000 mon.c (mon.2) 553 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:35:59 smithi161 bash[19769]: audit 2024-02-13T03:35:58.298750+0000 mon.a (mon.0) 2573 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:59.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:35:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:35:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:35:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:59 smithi183 bash[20824]: cluster 2024-02-13T03:35:57.481255+0000 mgr.y (mgr.24473) 583 : cluster [DBG] pgmap v891: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:35:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:59 smithi183 bash[20824]: audit 2024-02-13T03:35:58.290633+0000 mon.a (mon.0) 2571 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:35:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:59 smithi183 bash[20824]: cluster 2024-02-13T03:35:58.290729+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-02-13T03:35:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:59 smithi183 bash[20824]: audit 2024-02-13T03:35:58.292838+0000 mon.c (mon.2) 553 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:35:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:35:59 smithi183 bash[20824]: audit 2024-02-13T03:35:58.298750+0000 mon.a (mon.0) 2573 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:36:00.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:00 smithi161 bash[27879]: audit 2024-02-13T03:35:59.303008+0000 mon.a (mon.0) 2574 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:36:00.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:00 smithi161 bash[27879]: cluster 2024-02-13T03:35:59.303184+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-02-13T03:36:00.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:00 smithi161 bash[27879]: audit 2024-02-13T03:35:59.304192+0000 mon.c (mon.2) 554 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:36:00.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:00 smithi161 bash[27879]: audit 2024-02-13T03:35:59.304729+0000 mon.a (mon.0) 2576 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:36:00.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:00 smithi161 bash[27879]: cluster 2024-02-13T03:36:00.296990+0000 mon.a (mon.0) 2577 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:36:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:00 smithi161 bash[19769]: audit 2024-02-13T03:35:59.303008+0000 mon.a (mon.0) 2574 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:36:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:00 smithi161 bash[19769]: cluster 2024-02-13T03:35:59.303184+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-02-13T03:36:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:00 smithi161 bash[19769]: audit 2024-02-13T03:35:59.304192+0000 mon.c (mon.2) 554 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:36:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:00 smithi161 bash[19769]: audit 2024-02-13T03:35:59.304729+0000 mon.a (mon.0) 2576 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:36:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:00 smithi161 bash[19769]: cluster 2024-02-13T03:36:00.296990+0000 mon.a (mon.0) 2577 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:36:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:00 smithi183 bash[20824]: audit 2024-02-13T03:35:59.303008+0000 mon.a (mon.0) 2574 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:36:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:00 smithi183 bash[20824]: cluster 2024-02-13T03:35:59.303184+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-02-13T03:36:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:00 smithi183 bash[20824]: audit 2024-02-13T03:35:59.304192+0000 mon.c (mon.2) 554 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:36:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:00 smithi183 bash[20824]: audit 2024-02-13T03:35:59.304729+0000 mon.a (mon.0) 2576 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:36:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:00 smithi183 bash[20824]: cluster 2024-02-13T03:36:00.296990+0000 mon.a (mon.0) 2577 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:36:01.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:36:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:36:01] "GET /metrics HTTP/1.1" 200 186300 "" "Prometheus/2.33.4" 2024-02-13T03:36:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:01 smithi161 bash[19769]: cluster 2024-02-13T03:35:59.482108+0000 mgr.y (mgr.24473) 584 : cluster [DBG] pgmap v894: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:01 smithi161 bash[19769]: audit 2024-02-13T03:36:00.306897+0000 mon.a (mon.0) 2578 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-13T03:36:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:01 smithi161 bash[19769]: cluster 2024-02-13T03:36:00.307214+0000 mon.a (mon.0) 2579 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-02-13T03:36:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:01 smithi161 bash[19769]: audit 2024-02-13T03:36:00.309902+0000 mon.c (mon.2) 555 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-13T03:36:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:01 smithi161 bash[19769]: audit 2024-02-13T03:36:00.314564+0000 mon.a (mon.0) 2580 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-13T03:36:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:01 smithi161 bash[27879]: cluster 2024-02-13T03:35:59.482108+0000 mgr.y (mgr.24473) 584 : cluster [DBG] pgmap v894: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:01 smithi161 bash[27879]: audit 2024-02-13T03:36:00.306897+0000 mon.a (mon.0) 2578 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-13T03:36:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:01 smithi161 bash[27879]: cluster 2024-02-13T03:36:00.307214+0000 mon.a (mon.0) 2579 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-02-13T03:36:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:01 smithi161 bash[27879]: audit 2024-02-13T03:36:00.309902+0000 mon.c (mon.2) 555 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-13T03:36:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:01 smithi161 bash[27879]: audit 2024-02-13T03:36:00.314564+0000 mon.a (mon.0) 2580 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-13T03:36:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:01 smithi183 bash[20824]: cluster 2024-02-13T03:35:59.482108+0000 mgr.y (mgr.24473) 584 : cluster [DBG] pgmap v894: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:01 smithi183 bash[20824]: audit 2024-02-13T03:36:00.306897+0000 mon.a (mon.0) 2578 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-13T03:36:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:01 smithi183 bash[20824]: cluster 2024-02-13T03:36:00.307214+0000 mon.a (mon.0) 2579 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-02-13T03:36:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:01 smithi183 bash[20824]: audit 2024-02-13T03:36:00.309902+0000 mon.c (mon.2) 555 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-13T03:36:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:01 smithi183 bash[20824]: audit 2024-02-13T03:36:00.314564+0000 mon.a (mon.0) 2580 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-13T03:36:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:02 smithi161 bash[19769]: audit 2024-02-13T03:36:01.319387+0000 mon.a (mon.0) 2581 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-13T03:36:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:02 smithi161 bash[19769]: cluster 2024-02-13T03:36:01.319639+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-02-13T03:36:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:02 smithi161 bash[19769]: audit 2024-02-13T03:36:01.368078+0000 mon.c (mon.2) 556 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:02 smithi161 bash[19769]: audit 2024-02-13T03:36:01.368572+0000 mon.a (mon.0) 2583 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:02 smithi161 bash[27879]: audit 2024-02-13T03:36:01.319387+0000 mon.a (mon.0) 2581 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-13T03:36:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:02 smithi161 bash[27879]: cluster 2024-02-13T03:36:01.319639+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-02-13T03:36:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:02 smithi161 bash[27879]: audit 2024-02-13T03:36:01.368078+0000 mon.c (mon.2) 556 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:02 smithi161 bash[27879]: audit 2024-02-13T03:36:01.368572+0000 mon.a (mon.0) 2583 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:02 smithi183 bash[20824]: audit 2024-02-13T03:36:01.319387+0000 mon.a (mon.0) 2581 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-13T03:36:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:02 smithi183 bash[20824]: cluster 2024-02-13T03:36:01.319639+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-02-13T03:36:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:02 smithi183 bash[20824]: audit 2024-02-13T03:36:01.368078+0000 mon.c (mon.2) 556 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:02 smithi183 bash[20824]: audit 2024-02-13T03:36:01.368572+0000 mon.a (mon.0) 2583 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:03 smithi161 bash[27879]: cluster 2024-02-13T03:36:01.482958+0000 mgr.y (mgr.24473) 585 : cluster [DBG] pgmap v897: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:03 smithi161 bash[27879]: audit 2024-02-13T03:36:02.334134+0000 mon.a (mon.0) 2584 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-99"}]': finished 2024-02-13T03:36:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:03 smithi161 bash[27879]: cluster 2024-02-13T03:36:02.334271+0000 mon.a (mon.0) 2585 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-02-13T03:36:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:03 smithi161 bash[27879]: audit 2024-02-13T03:36:02.336983+0000 mon.c (mon.2) 557 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:36:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:03 smithi161 bash[27879]: audit 2024-02-13T03:36:02.343495+0000 mon.a (mon.0) 2586 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:36:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:03 smithi161 bash[19769]: cluster 2024-02-13T03:36:01.482958+0000 mgr.y (mgr.24473) 585 : cluster [DBG] pgmap v897: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:03 smithi161 bash[19769]: audit 2024-02-13T03:36:02.334134+0000 mon.a (mon.0) 2584 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-99"}]': finished 2024-02-13T03:36:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:03 smithi161 bash[19769]: cluster 2024-02-13T03:36:02.334271+0000 mon.a (mon.0) 2585 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-02-13T03:36:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:03 smithi161 bash[19769]: audit 2024-02-13T03:36:02.336983+0000 mon.c (mon.2) 557 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:36:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:03 smithi161 bash[19769]: audit 2024-02-13T03:36:02.343495+0000 mon.a (mon.0) 2586 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:36:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:03 smithi183 bash[20824]: cluster 2024-02-13T03:36:01.482958+0000 mgr.y (mgr.24473) 585 : cluster [DBG] pgmap v897: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:03 smithi183 bash[20824]: audit 2024-02-13T03:36:02.334134+0000 mon.a (mon.0) 2584 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-99"}]': finished 2024-02-13T03:36:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:03 smithi183 bash[20824]: cluster 2024-02-13T03:36:02.334271+0000 mon.a (mon.0) 2585 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-02-13T03:36:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:03 smithi183 bash[20824]: audit 2024-02-13T03:36:02.336983+0000 mon.c (mon.2) 557 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:36:03.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:03 smithi183 bash[20824]: audit 2024-02-13T03:36:02.343495+0000 mon.a (mon.0) 2586 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]: dispatch 2024-02-13T03:36:04.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:04 smithi161 bash[27879]: audit 2024-02-13T03:36:03.348336+0000 mon.a (mon.0) 2587 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]': finished 2024-02-13T03:36:04.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:04 smithi161 bash[27879]: cluster 2024-02-13T03:36:03.348504+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-02-13T03:36:04.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:04 smithi161 bash[27879]: cluster 2024-02-13T03:36:04.349111+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-02-13T03:36:04.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:04 smithi161 bash[19769]: audit 2024-02-13T03:36:03.348336+0000 mon.a (mon.0) 2587 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]': finished 2024-02-13T03:36:04.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:04 smithi161 bash[19769]: cluster 2024-02-13T03:36:03.348504+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-02-13T03:36:04.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:04 smithi161 bash[19769]: cluster 2024-02-13T03:36:04.349111+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-02-13T03:36:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:04 smithi183 bash[20824]: audit 2024-02-13T03:36:03.348336+0000 mon.a (mon.0) 2587 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-99", "tierpool": "test-rados-api-smithi161-61906-99-cache"}]': finished 2024-02-13T03:36:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:04 smithi183 bash[20824]: cluster 2024-02-13T03:36:03.348504+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-02-13T03:36:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:04 smithi183 bash[20824]: cluster 2024-02-13T03:36:04.349111+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-02-13T03:36:05.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:05 smithi161 bash[27879]: cluster 2024-02-13T03:36:03.483851+0000 mgr.y (mgr.24473) 586 : cluster [DBG] pgmap v900: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:05.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:05 smithi161 bash[27879]: cluster 2024-02-13T03:36:05.351829+0000 mon.a (mon.0) 2590 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-02-13T03:36:05.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:05 smithi161 bash[27879]: audit 2024-02-13T03:36:05.355152+0000 mon.c (mon.2) 558 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:05.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:05 smithi161 bash[27879]: audit 2024-02-13T03:36:05.355644+0000 mon.a (mon.0) 2591 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:05 smithi161 bash[19769]: cluster 2024-02-13T03:36:03.483851+0000 mgr.y (mgr.24473) 586 : cluster [DBG] pgmap v900: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:05 smithi161 bash[19769]: cluster 2024-02-13T03:36:05.351829+0000 mon.a (mon.0) 2590 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-02-13T03:36:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:05 smithi161 bash[19769]: audit 2024-02-13T03:36:05.355152+0000 mon.c (mon.2) 558 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:05 smithi161 bash[19769]: audit 2024-02-13T03:36:05.355644+0000 mon.a (mon.0) 2591 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:05.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:05 smithi183 bash[20824]: cluster 2024-02-13T03:36:03.483851+0000 mgr.y (mgr.24473) 586 : cluster [DBG] pgmap v900: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:05 smithi183 bash[20824]: cluster 2024-02-13T03:36:05.351829+0000 mon.a (mon.0) 2590 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-02-13T03:36:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:05 smithi183 bash[20824]: audit 2024-02-13T03:36:05.355152+0000 mon.c (mon.2) 558 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:05 smithi183 bash[20824]: audit 2024-02-13T03:36:05.355644+0000 mon.a (mon.0) 2591 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:06.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:36:06 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:36:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:07 smithi161 bash[27879]: cluster 2024-02-13T03:36:05.484687+0000 mgr.y (mgr.24473) 587 : cluster [DBG] pgmap v903: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:07 smithi161 bash[27879]: audit 2024-02-13T03:36:06.265021+0000 mgr.y (mgr.24473) 588 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:07 smithi161 bash[27879]: audit 2024-02-13T03:36:06.355990+0000 mon.a (mon.0) 2592 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]': finished 2024-02-13T03:36:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:07 smithi161 bash[27879]: cluster 2024-02-13T03:36:06.356185+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-02-13T03:36:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:07 smithi161 bash[27879]: audit 2024-02-13T03:36:06.358775+0000 mon.c (mon.2) 559 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:07 smithi161 bash[27879]: audit 2024-02-13T03:36:06.359368+0000 mon.a (mon.0) 2594 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:07 smithi161 bash[19769]: cluster 2024-02-13T03:36:05.484687+0000 mgr.y (mgr.24473) 587 : cluster [DBG] pgmap v903: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:07 smithi161 bash[19769]: audit 2024-02-13T03:36:06.265021+0000 mgr.y (mgr.24473) 588 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:07 smithi161 bash[19769]: audit 2024-02-13T03:36:06.355990+0000 mon.a (mon.0) 2592 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]': finished 2024-02-13T03:36:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:07 smithi161 bash[19769]: cluster 2024-02-13T03:36:06.356185+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-02-13T03:36:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:07 smithi161 bash[19769]: audit 2024-02-13T03:36:06.358775+0000 mon.c (mon.2) 559 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:07 smithi161 bash[19769]: audit 2024-02-13T03:36:06.359368+0000 mon.a (mon.0) 2594 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:07.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:07 smithi183 bash[20824]: cluster 2024-02-13T03:36:05.484687+0000 mgr.y (mgr.24473) 587 : cluster [DBG] pgmap v903: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:07 smithi183 bash[20824]: audit 2024-02-13T03:36:06.265021+0000 mgr.y (mgr.24473) 588 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:07 smithi183 bash[20824]: audit 2024-02-13T03:36:06.355990+0000 mon.a (mon.0) 2592 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-99"}]': finished 2024-02-13T03:36:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:07 smithi183 bash[20824]: cluster 2024-02-13T03:36:06.356185+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-02-13T03:36:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:07 smithi183 bash[20824]: audit 2024-02-13T03:36:06.358775+0000 mon.c (mon.2) 559 : audit [INF] from='client.? v1:172.21.15.161:0/1510756986' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:07 smithi183 bash[20824]: audit 2024-02-13T03:36:06.359368+0000 mon.a (mon.0) 2594 : audit [INF] from='client.48386 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]: dispatch 2024-02-13T03:36:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:08 smithi161 bash[27879]: audit 2024-02-13T03:36:07.360691+0000 mon.a (mon.0) 2595 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]': finished 2024-02-13T03:36:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:08 smithi161 bash[27879]: cluster 2024-02-13T03:36:07.360819+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-02-13T03:36:08.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:08 smithi161 bash[19769]: audit 2024-02-13T03:36:07.360691+0000 mon.a (mon.0) 2595 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]': finished 2024-02-13T03:36:08.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:08 smithi161 bash[19769]: cluster 2024-02-13T03:36:07.360819+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-02-13T03:36:08.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:08 smithi183 bash[20824]: audit 2024-02-13T03:36:07.360691+0000 mon.a (mon.0) 2595 : audit [INF] from='client.48386 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-99"}]': finished 2024-02-13T03:36:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:08 smithi183 bash[20824]: cluster 2024-02-13T03:36:07.360819+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-02-13T03:36:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:09 smithi161 bash[27879]: cluster 2024-02-13T03:36:07.485526+0000 mgr.y (mgr.24473) 589 : cluster [DBG] pgmap v906: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:09 smithi161 bash[27879]: cluster 2024-02-13T03:36:08.375034+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-02-13T03:36:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:09 smithi161 bash[27879]: audit 2024-02-13T03:36:08.381627+0000 mon.c (mon.2) 560 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:36:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:09 smithi161 bash[27879]: audit 2024-02-13T03:36:08.387600+0000 mon.a (mon.0) 2598 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:36:09.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:09 smithi161 bash[19769]: cluster 2024-02-13T03:36:07.485526+0000 mgr.y (mgr.24473) 589 : cluster [DBG] pgmap v906: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:09.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:09 smithi161 bash[19769]: cluster 2024-02-13T03:36:08.375034+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-02-13T03:36:09.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:09 smithi161 bash[19769]: audit 2024-02-13T03:36:08.381627+0000 mon.c (mon.2) 560 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:36:09.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:09 smithi161 bash[19769]: audit 2024-02-13T03:36:08.387600+0000 mon.a (mon.0) 2598 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:36:09.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:36:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:36:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:36:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:09 smithi183 bash[20824]: cluster 2024-02-13T03:36:07.485526+0000 mgr.y (mgr.24473) 589 : cluster [DBG] pgmap v906: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:09 smithi183 bash[20824]: cluster 2024-02-13T03:36:08.375034+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-02-13T03:36:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:09 smithi183 bash[20824]: audit 2024-02-13T03:36:08.381627+0000 mon.c (mon.2) 560 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:36:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:09 smithi183 bash[20824]: audit 2024-02-13T03:36:08.387600+0000 mon.a (mon.0) 2598 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:36:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:10 smithi161 bash[27879]: audit 2024-02-13T03:36:09.390423+0000 mon.a (mon.0) 2599 : audit [INF] from='client.48374 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]': finished 2024-02-13T03:36:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:10 smithi161 bash[27879]: cluster 2024-02-13T03:36:09.390593+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-02-13T03:36:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:10 smithi161 bash[27879]: audit 2024-02-13T03:36:09.392873+0000 mon.c (mon.2) 561 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:36:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:10 smithi161 bash[27879]: audit 2024-02-13T03:36:09.398162+0000 mon.a (mon.0) 2601 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:36:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:10 smithi161 bash[27879]: cluster 2024-02-13T03:36:09.486459+0000 mgr.y (mgr.24473) 590 : cluster [DBG] pgmap v909: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-13T03:36:10.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:10 smithi161 bash[19769]: audit 2024-02-13T03:36:09.390423+0000 mon.a (mon.0) 2599 : audit [INF] from='client.48374 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]': finished 2024-02-13T03:36:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:10 smithi161 bash[19769]: cluster 2024-02-13T03:36:09.390593+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-02-13T03:36:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:10 smithi161 bash[19769]: audit 2024-02-13T03:36:09.392873+0000 mon.c (mon.2) 561 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:36:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:10 smithi161 bash[19769]: audit 2024-02-13T03:36:09.398162+0000 mon.a (mon.0) 2601 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:36:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:10 smithi161 bash[19769]: cluster 2024-02-13T03:36:09.486459+0000 mgr.y (mgr.24473) 590 : cluster [DBG] pgmap v909: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-13T03:36:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:10 smithi183 bash[20824]: audit 2024-02-13T03:36:09.390423+0000 mon.a (mon.0) 2599 : audit [INF] from='client.48374 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi161-61906-94"}]': finished 2024-02-13T03:36:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:10 smithi183 bash[20824]: cluster 2024-02-13T03:36:09.390593+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-02-13T03:36:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:10 smithi183 bash[20824]: audit 2024-02-13T03:36:09.392873+0000 mon.c (mon.2) 561 : audit [INF] from='client.? v1:172.21.15.161:0/2649538060' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:36:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:10 smithi183 bash[20824]: audit 2024-02-13T03:36:09.398162+0000 mon.a (mon.0) 2601 : audit [INF] from='client.48374 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]: dispatch 2024-02-13T03:36:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:10 smithi183 bash[20824]: cluster 2024-02-13T03:36:09.486459+0000 mgr.y (mgr.24473) 590 : cluster [DBG] pgmap v909: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-13T03:36:11.407 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:36:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:36:11] "GET /metrics HTTP/1.1" 200 188855 "" "Prometheus/2.33.4" 2024-02-13T03:36:11.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:11 smithi161 bash[27879]: cluster 2024-02-13T03:36:10.393386+0000 mon.a (mon.0) 2602 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:36:11.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:11 smithi161 bash[27879]: audit 2024-02-13T03:36:10.412287+0000 mon.a (mon.0) 2603 : audit [INF] from='client.48374 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]': finished 2024-02-13T03:36:11.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:11 smithi161 bash[27879]: cluster 2024-02-13T03:36:10.412377+0000 mon.a (mon.0) 2604 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-02-13T03:36:11.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:11 smithi161 bash[27879]: audit 2024-02-13T03:36:10.431577+0000 mon.a (mon.0) 2605 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:11.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:11 smithi161 bash[27879]: audit 2024-02-13T03:36:10.432987+0000 mon.a (mon.0) 2606 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:11.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:11 smithi161 bash[27879]: audit 2024-02-13T03:36:10.433401+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:36:11.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:11 smithi161 bash[19769]: cluster 2024-02-13T03:36:10.393386+0000 mon.a (mon.0) 2602 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:36:11.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:11 smithi161 bash[19769]: audit 2024-02-13T03:36:10.412287+0000 mon.a (mon.0) 2603 : audit [INF] from='client.48374 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]': finished 2024-02-13T03:36:11.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:11 smithi161 bash[19769]: cluster 2024-02-13T03:36:10.412377+0000 mon.a (mon.0) 2604 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-02-13T03:36:11.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:11 smithi161 bash[19769]: audit 2024-02-13T03:36:10.431577+0000 mon.a (mon.0) 2605 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:11.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:11 smithi161 bash[19769]: audit 2024-02-13T03:36:10.432987+0000 mon.a (mon.0) 2606 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:11.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:11 smithi161 bash[19769]: audit 2024-02-13T03:36:10.433401+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:36:11.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:11 smithi183 bash[20824]: cluster 2024-02-13T03:36:10.393386+0000 mon.a (mon.0) 2602 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:36:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:11 smithi183 bash[20824]: audit 2024-02-13T03:36:10.412287+0000 mon.a (mon.0) 2603 : audit [INF] from='client.48374 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi161-61906-94"}]': finished 2024-02-13T03:36:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:11 smithi183 bash[20824]: cluster 2024-02-13T03:36:10.412377+0000 mon.a (mon.0) 2604 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-02-13T03:36:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:11 smithi183 bash[20824]: audit 2024-02-13T03:36:10.431577+0000 mon.a (mon.0) 2605 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:11 smithi183 bash[20824]: audit 2024-02-13T03:36:10.432987+0000 mon.a (mon.0) 2606 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:11 smithi183 bash[20824]: audit 2024-02-13T03:36:10.433401+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T03:36:13.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:12 smithi161 bash[27879]: audit 2024-02-13T03:36:11.418684+0000 mon.a (mon.0) 2608 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:36:13.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:12 smithi161 bash[27879]: cluster 2024-02-13T03:36:11.418772+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-02-13T03:36:13.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:12 smithi161 bash[27879]: audit 2024-02-13T03:36:11.419703+0000 mon.a (mon.0) 2610 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:13.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:12 smithi161 bash[27879]: cluster 2024-02-13T03:36:11.487253+0000 mgr.y (mgr.24473) 591 : cluster [DBG] pgmap v912: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-13T03:36:13.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:12 smithi161 bash[19769]: audit 2024-02-13T03:36:11.418684+0000 mon.a (mon.0) 2608 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:36:13.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:12 smithi161 bash[19769]: cluster 2024-02-13T03:36:11.418772+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-02-13T03:36:13.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:12 smithi161 bash[19769]: audit 2024-02-13T03:36:11.419703+0000 mon.a (mon.0) 2610 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:13.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:12 smithi161 bash[19769]: cluster 2024-02-13T03:36:11.487253+0000 mgr.y (mgr.24473) 591 : cluster [DBG] pgmap v912: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-13T03:36:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:12 smithi183 bash[20824]: audit 2024-02-13T03:36:11.418684+0000 mon.a (mon.0) 2608 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi161-61906-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T03:36:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:12 smithi183 bash[20824]: cluster 2024-02-13T03:36:11.418772+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-02-13T03:36:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:12 smithi183 bash[20824]: audit 2024-02-13T03:36:11.419703+0000 mon.a (mon.0) 2610 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:13.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:12 smithi183 bash[20824]: cluster 2024-02-13T03:36:11.487253+0000 mgr.y (mgr.24473) 591 : cluster [DBG] pgmap v912: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-13T03:36:14.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:13 smithi161 bash[27879]: cluster 2024-02-13T03:36:12.847069+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-02-13T03:36:14.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:13 smithi161 bash[27879]: audit 2024-02-13T03:36:13.428880+0000 mon.a (mon.0) 2612 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:36:14.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:13 smithi161 bash[27879]: cluster 2024-02-13T03:36:13.429166+0000 mon.a (mon.0) 2613 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-02-13T03:36:14.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:13 smithi161 bash[19769]: cluster 2024-02-13T03:36:12.847069+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-02-13T03:36:14.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:13 smithi161 bash[19769]: audit 2024-02-13T03:36:13.428880+0000 mon.a (mon.0) 2612 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:36:14.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:13 smithi161 bash[19769]: cluster 2024-02-13T03:36:13.429166+0000 mon.a (mon.0) 2613 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-02-13T03:36:14.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:13 smithi183 bash[20824]: cluster 2024-02-13T03:36:12.847069+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-02-13T03:36:14.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:13 smithi183 bash[20824]: audit 2024-02-13T03:36:13.428880+0000 mon.a (mon.0) 2612 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi161-61906-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:36:14.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:13 smithi183 bash[20824]: cluster 2024-02-13T03:36:13.429166+0000 mon.a (mon.0) 2613 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-02-13T03:36:15.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:14 smithi161 bash[27879]: cluster 2024-02-13T03:36:13.487995+0000 mgr.y (mgr.24473) 592 : cluster [DBG] pgmap v915: 172 pgs: 8 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:15.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:14 smithi161 bash[27879]: cluster 2024-02-13T03:36:14.435805+0000 mon.a (mon.0) 2614 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-02-13T03:36:15.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:14 smithi161 bash[27879]: audit 2024-02-13T03:36:14.442330+0000 mon.a (mon.0) 2615 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:36:15.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:14 smithi161 bash[19769]: cluster 2024-02-13T03:36:13.487995+0000 mgr.y (mgr.24473) 592 : cluster [DBG] pgmap v915: 172 pgs: 8 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:15.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:14 smithi161 bash[19769]: cluster 2024-02-13T03:36:14.435805+0000 mon.a (mon.0) 2614 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-02-13T03:36:15.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:14 smithi161 bash[19769]: audit 2024-02-13T03:36:14.442330+0000 mon.a (mon.0) 2615 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:36:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:14 smithi183 bash[20824]: cluster 2024-02-13T03:36:13.487995+0000 mgr.y (mgr.24473) 592 : cluster [DBG] pgmap v915: 172 pgs: 8 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:36:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:14 smithi183 bash[20824]: cluster 2024-02-13T03:36:14.435805+0000 mon.a (mon.0) 2614 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-02-13T03:36:15.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:14 smithi183 bash[20824]: audit 2024-02-13T03:36:14.442330+0000 mon.a (mon.0) 2615 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:36:16.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:36:16 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:36:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:16 smithi183 bash[20824]: audit 2024-02-13T03:36:15.439185+0000 mon.a (mon.0) 2616 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:36:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:16 smithi183 bash[20824]: cluster 2024-02-13T03:36:15.439467+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-02-13T03:36:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:16 smithi183 bash[20824]: audit 2024-02-13T03:36:15.463524+0000 mon.a (mon.0) 2618 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:36:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:16 smithi183 bash[20824]: cluster 2024-02-13T03:36:15.488697+0000 mgr.y (mgr.24473) 593 : cluster [DBG] pgmap v918: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:16 smithi183 bash[20824]: audit 2024-02-13T03:36:16.270645+0000 mgr.y (mgr.24473) 594 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:16.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:16 smithi161 bash[27879]: audit 2024-02-13T03:36:15.439185+0000 mon.a (mon.0) 2616 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:36:16.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:16 smithi161 bash[27879]: cluster 2024-02-13T03:36:15.439467+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-02-13T03:36:16.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:16 smithi161 bash[27879]: audit 2024-02-13T03:36:15.463524+0000 mon.a (mon.0) 2618 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:36:16.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:16 smithi161 bash[27879]: cluster 2024-02-13T03:36:15.488697+0000 mgr.y (mgr.24473) 593 : cluster [DBG] pgmap v918: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:16.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:16 smithi161 bash[27879]: audit 2024-02-13T03:36:16.270645+0000 mgr.y (mgr.24473) 594 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:16 smithi161 bash[19769]: audit 2024-02-13T03:36:15.439185+0000 mon.a (mon.0) 2616 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:36:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:16 smithi161 bash[19769]: cluster 2024-02-13T03:36:15.439467+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-02-13T03:36:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:16 smithi161 bash[19769]: audit 2024-02-13T03:36:15.463524+0000 mon.a (mon.0) 2618 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:36:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:16 smithi161 bash[19769]: cluster 2024-02-13T03:36:15.488697+0000 mgr.y (mgr.24473) 593 : cluster [DBG] pgmap v918: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:16 smithi161 bash[19769]: audit 2024-02-13T03:36:16.270645+0000 mgr.y (mgr.24473) 594 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:17.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:17 smithi183 bash[20824]: audit 2024-02-13T03:36:16.451151+0000 mon.a (mon.0) 2619 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:36:17.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:17 smithi183 bash[20824]: cluster 2024-02-13T03:36:16.451375+0000 mon.a (mon.0) 2620 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-02-13T03:36:17.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:17 smithi183 bash[20824]: audit 2024-02-13T03:36:16.452163+0000 mon.a (mon.0) 2621 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-102"}]: dispatch 2024-02-13T03:36:17.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:17 smithi161 bash[27879]: audit 2024-02-13T03:36:16.451151+0000 mon.a (mon.0) 2619 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:36:17.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:17 smithi161 bash[27879]: cluster 2024-02-13T03:36:16.451375+0000 mon.a (mon.0) 2620 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-02-13T03:36:17.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:17 smithi161 bash[27879]: audit 2024-02-13T03:36:16.452163+0000 mon.a (mon.0) 2621 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-102"}]: dispatch 2024-02-13T03:36:17.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:17 smithi161 bash[19769]: audit 2024-02-13T03:36:16.451151+0000 mon.a (mon.0) 2619 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:36:17.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:17 smithi161 bash[19769]: cluster 2024-02-13T03:36:16.451375+0000 mon.a (mon.0) 2620 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-02-13T03:36:17.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:17 smithi161 bash[19769]: audit 2024-02-13T03:36:16.452163+0000 mon.a (mon.0) 2621 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-102"}]: dispatch 2024-02-13T03:36:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:18 smithi183 bash[20824]: audit 2024-02-13T03:36:17.460209+0000 mon.a (mon.0) 2622 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-102"}]': finished 2024-02-13T03:36:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:18 smithi183 bash[20824]: cluster 2024-02-13T03:36:17.460369+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-02-13T03:36:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:18 smithi183 bash[20824]: cluster 2024-02-13T03:36:17.489343+0000 mgr.y (mgr.24473) 595 : cluster [DBG] pgmap v921: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:36:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:18 smithi183 bash[20824]: audit 2024-02-13T03:36:17.498977+0000 mon.a (mon.0) 2624 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:18.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:18 smithi161 bash[27879]: audit 2024-02-13T03:36:17.460209+0000 mon.a (mon.0) 2622 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-102"}]': finished 2024-02-13T03:36:18.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:18 smithi161 bash[27879]: cluster 2024-02-13T03:36:17.460369+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-02-13T03:36:18.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:18 smithi161 bash[27879]: cluster 2024-02-13T03:36:17.489343+0000 mgr.y (mgr.24473) 595 : cluster [DBG] pgmap v921: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:36:18.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:18 smithi161 bash[27879]: audit 2024-02-13T03:36:17.498977+0000 mon.a (mon.0) 2624 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:18.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:18 smithi161 bash[19769]: audit 2024-02-13T03:36:17.460209+0000 mon.a (mon.0) 2622 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-102"}]': finished 2024-02-13T03:36:18.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:18 smithi161 bash[19769]: cluster 2024-02-13T03:36:17.460369+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-02-13T03:36:18.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:18 smithi161 bash[19769]: cluster 2024-02-13T03:36:17.489343+0000 mgr.y (mgr.24473) 595 : cluster [DBG] pgmap v921: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:36:18.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:18 smithi161 bash[19769]: audit 2024-02-13T03:36:17.498977+0000 mon.a (mon.0) 2624 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:19.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:36:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:36:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:36:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:19 smithi183 bash[20824]: cluster 2024-02-13T03:36:18.456282+0000 mon.a (mon.0) 2625 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:36:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:19 smithi183 bash[20824]: audit 2024-02-13T03:36:18.465585+0000 mon.a (mon.0) 2626 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:36:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:19 smithi183 bash[20824]: cluster 2024-02-13T03:36:18.465783+0000 mon.a (mon.0) 2627 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-02-13T03:36:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:19 smithi183 bash[20824]: audit 2024-02-13T03:36:18.467113+0000 mon.a (mon.0) 2628 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-102"}]: dispatch 2024-02-13T03:36:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:19 smithi183 bash[20824]: audit 2024-02-13T03:36:19.466773+0000 mon.a (mon.0) 2629 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-102"}]': finished 2024-02-13T03:36:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:19 smithi183 bash[20824]: cluster 2024-02-13T03:36:19.467358+0000 mon.a (mon.0) 2630 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-02-13T03:36:19.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:19 smithi161 bash[27879]: cluster 2024-02-13T03:36:18.456282+0000 mon.a (mon.0) 2625 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:36:19.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:19 smithi161 bash[27879]: audit 2024-02-13T03:36:18.465585+0000 mon.a (mon.0) 2626 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:36:19.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:19 smithi161 bash[27879]: cluster 2024-02-13T03:36:18.465783+0000 mon.a (mon.0) 2627 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-02-13T03:36:19.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:19 smithi161 bash[27879]: audit 2024-02-13T03:36:18.467113+0000 mon.a (mon.0) 2628 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-102"}]: dispatch 2024-02-13T03:36:19.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:19 smithi161 bash[27879]: audit 2024-02-13T03:36:19.466773+0000 mon.a (mon.0) 2629 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-102"}]': finished 2024-02-13T03:36:19.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:19 smithi161 bash[27879]: cluster 2024-02-13T03:36:19.467358+0000 mon.a (mon.0) 2630 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-02-13T03:36:19.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:19 smithi161 bash[19769]: cluster 2024-02-13T03:36:18.456282+0000 mon.a (mon.0) 2625 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:36:19.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:19 smithi161 bash[19769]: audit 2024-02-13T03:36:18.465585+0000 mon.a (mon.0) 2626 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:36:19.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:19 smithi161 bash[19769]: cluster 2024-02-13T03:36:18.465783+0000 mon.a (mon.0) 2627 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-02-13T03:36:19.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:19 smithi161 bash[19769]: audit 2024-02-13T03:36:18.467113+0000 mon.a (mon.0) 2628 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-102"}]: dispatch 2024-02-13T03:36:19.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:19 smithi161 bash[19769]: audit 2024-02-13T03:36:19.466773+0000 mon.a (mon.0) 2629 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-102"}]': finished 2024-02-13T03:36:19.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:19 smithi161 bash[19769]: cluster 2024-02-13T03:36:19.467358+0000 mon.a (mon.0) 2630 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-02-13T03:36:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:20 smithi183 bash[20824]: cluster 2024-02-13T03:36:19.489840+0000 mgr.y (mgr.24473) 596 : cluster [DBG] pgmap v924: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:36:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:20 smithi183 bash[20824]: audit 2024-02-13T03:36:20.013227+0000 mon.c (mon.2) 562 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:36:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:20 smithi183 bash[20824]: audit 2024-02-13T03:36:20.313077+0000 mon.c (mon.2) 563 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:36:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:20 smithi183 bash[20824]: audit 2024-02-13T03:36:20.314294+0000 mon.c (mon.2) 564 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:36:20.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:20 smithi183 bash[20824]: audit 2024-02-13T03:36:20.322170+0000 mon.a (mon.0) 2631 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:36:20.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:20 smithi161 bash[27879]: cluster 2024-02-13T03:36:19.489840+0000 mgr.y (mgr.24473) 596 : cluster [DBG] pgmap v924: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:36:20.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:20 smithi161 bash[27879]: audit 2024-02-13T03:36:20.013227+0000 mon.c (mon.2) 562 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:36:20.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:20 smithi161 bash[27879]: audit 2024-02-13T03:36:20.313077+0000 mon.c (mon.2) 563 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:36:20.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:20 smithi161 bash[27879]: audit 2024-02-13T03:36:20.314294+0000 mon.c (mon.2) 564 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:36:20.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:20 smithi161 bash[27879]: audit 2024-02-13T03:36:20.322170+0000 mon.a (mon.0) 2631 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:36:20.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:20 smithi161 bash[19769]: cluster 2024-02-13T03:36:19.489840+0000 mgr.y (mgr.24473) 596 : cluster [DBG] pgmap v924: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:36:20.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:20 smithi161 bash[19769]: audit 2024-02-13T03:36:20.013227+0000 mon.c (mon.2) 562 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:36:20.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:20 smithi161 bash[19769]: audit 2024-02-13T03:36:20.313077+0000 mon.c (mon.2) 563 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:36:20.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:20 smithi161 bash[19769]: audit 2024-02-13T03:36:20.314294+0000 mon.c (mon.2) 564 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:36:20.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:20 smithi161 bash[19769]: audit 2024-02-13T03:36:20.322170+0000 mon.a (mon.0) 2631 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:36:21.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:36:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:36:21] "GET /metrics HTTP/1.1" 200 189154 "" "Prometheus/2.33.4" 2024-02-13T03:36:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:21 smithi183 bash[20824]: cluster 2024-02-13T03:36:20.488647+0000 mon.a (mon.0) 2632 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-02-13T03:36:21.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:21 smithi161 bash[27879]: cluster 2024-02-13T03:36:20.488647+0000 mon.a (mon.0) 2632 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-02-13T03:36:21.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:21 smithi161 bash[19769]: cluster 2024-02-13T03:36:20.488647+0000 mon.a (mon.0) 2632 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-02-13T03:36:22.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:22 smithi161 bash[27879]: cluster 2024-02-13T03:36:21.491878+0000 mgr.y (mgr.24473) 597 : cluster [DBG] pgmap v926: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T03:36:22.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:22 smithi161 bash[27879]: cluster 2024-02-13T03:36:21.497431+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-02-13T03:36:22.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:22 smithi161 bash[27879]: audit 2024-02-13T03:36:21.506767+0000 mon.a (mon.0) 2634 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:36:22.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:22 smithi161 bash[19769]: cluster 2024-02-13T03:36:21.491878+0000 mgr.y (mgr.24473) 597 : cluster [DBG] pgmap v926: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T03:36:22.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:22 smithi161 bash[19769]: cluster 2024-02-13T03:36:21.497431+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-02-13T03:36:22.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:22 smithi161 bash[19769]: audit 2024-02-13T03:36:21.506767+0000 mon.a (mon.0) 2634 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:36:22.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:22 smithi183 bash[20824]: cluster 2024-02-13T03:36:21.491878+0000 mgr.y (mgr.24473) 597 : cluster [DBG] pgmap v926: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T03:36:22.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:22 smithi183 bash[20824]: cluster 2024-02-13T03:36:21.497431+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-02-13T03:36:22.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:22 smithi183 bash[20824]: audit 2024-02-13T03:36:21.506767+0000 mon.a (mon.0) 2634 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:36:23.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:23 smithi161 bash[19769]: audit 2024-02-13T03:36:22.503624+0000 mon.a (mon.0) 2635 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:36:23.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:23 smithi161 bash[19769]: cluster 2024-02-13T03:36:22.504060+0000 mon.a (mon.0) 2636 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-02-13T03:36:23.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:23 smithi161 bash[19769]: audit 2024-02-13T03:36:22.522423+0000 mon.a (mon.0) 2637 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:36:23.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:23 smithi161 bash[19769]: audit 2024-02-13T03:36:23.511541+0000 mon.a (mon.0) 2638 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:36:23.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:23 smithi161 bash[19769]: cluster 2024-02-13T03:36:23.511757+0000 mon.a (mon.0) 2639 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-02-13T03:36:23.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:23 smithi161 bash[19769]: audit 2024-02-13T03:36:23.512366+0000 mon.a (mon.0) 2640 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-104"}]: dispatch 2024-02-13T03:36:23.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:23 smithi161 bash[27879]: audit 2024-02-13T03:36:22.503624+0000 mon.a (mon.0) 2635 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:36:23.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:23 smithi161 bash[27879]: cluster 2024-02-13T03:36:22.504060+0000 mon.a (mon.0) 2636 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-02-13T03:36:23.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:23 smithi161 bash[27879]: audit 2024-02-13T03:36:22.522423+0000 mon.a (mon.0) 2637 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:36:23.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:23 smithi161 bash[27879]: audit 2024-02-13T03:36:23.511541+0000 mon.a (mon.0) 2638 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:36:23.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:23 smithi161 bash[27879]: cluster 2024-02-13T03:36:23.511757+0000 mon.a (mon.0) 2639 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-02-13T03:36:23.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:23 smithi161 bash[27879]: audit 2024-02-13T03:36:23.512366+0000 mon.a (mon.0) 2640 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-104"}]: dispatch 2024-02-13T03:36:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:23 smithi183 bash[20824]: audit 2024-02-13T03:36:22.503624+0000 mon.a (mon.0) 2635 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:36:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:23 smithi183 bash[20824]: cluster 2024-02-13T03:36:22.504060+0000 mon.a (mon.0) 2636 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-02-13T03:36:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:23 smithi183 bash[20824]: audit 2024-02-13T03:36:22.522423+0000 mon.a (mon.0) 2637 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:36:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:23 smithi183 bash[20824]: audit 2024-02-13T03:36:23.511541+0000 mon.a (mon.0) 2638 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:36:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:23 smithi183 bash[20824]: cluster 2024-02-13T03:36:23.511757+0000 mon.a (mon.0) 2639 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-02-13T03:36:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:23 smithi183 bash[20824]: audit 2024-02-13T03:36:23.512366+0000 mon.a (mon.0) 2640 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-104"}]: dispatch 2024-02-13T03:36:25.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:24 smithi161 bash[27879]: cluster 2024-02-13T03:36:23.492670+0000 mgr.y (mgr.24473) 598 : cluster [DBG] pgmap v929: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T03:36:25.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:24 smithi161 bash[27879]: audit 2024-02-13T03:36:24.506229+0000 mon.a (mon.0) 2641 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-104"}]': finished 2024-02-13T03:36:25.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:24 smithi161 bash[27879]: cluster 2024-02-13T03:36:24.506321+0000 mon.a (mon.0) 2642 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-02-13T03:36:25.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:24 smithi161 bash[27879]: audit 2024-02-13T03:36:24.509612+0000 mon.a (mon.0) 2643 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-104", "mode": "writeback"}]: dispatch 2024-02-13T03:36:25.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:24 smithi161 bash[19769]: cluster 2024-02-13T03:36:23.492670+0000 mgr.y (mgr.24473) 598 : cluster [DBG] pgmap v929: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T03:36:25.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:24 smithi161 bash[19769]: audit 2024-02-13T03:36:24.506229+0000 mon.a (mon.0) 2641 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-104"}]': finished 2024-02-13T03:36:25.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:24 smithi161 bash[19769]: cluster 2024-02-13T03:36:24.506321+0000 mon.a (mon.0) 2642 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-02-13T03:36:25.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:24 smithi161 bash[19769]: audit 2024-02-13T03:36:24.509612+0000 mon.a (mon.0) 2643 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-104", "mode": "writeback"}]: dispatch 2024-02-13T03:36:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:24 smithi183 bash[20824]: cluster 2024-02-13T03:36:23.492670+0000 mgr.y (mgr.24473) 598 : cluster [DBG] pgmap v929: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T03:36:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:24 smithi183 bash[20824]: audit 2024-02-13T03:36:24.506229+0000 mon.a (mon.0) 2641 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-104"}]': finished 2024-02-13T03:36:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:24 smithi183 bash[20824]: cluster 2024-02-13T03:36:24.506321+0000 mon.a (mon.0) 2642 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-02-13T03:36:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:24 smithi183 bash[20824]: audit 2024-02-13T03:36:24.509612+0000 mon.a (mon.0) 2643 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-104", "mode": "writeback"}]: dispatch 2024-02-13T03:36:26.284 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:26 smithi183 bash[20824]: cluster 2024-02-13T03:36:25.501312+0000 mon.a (mon.0) 2644 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:36:26.284 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:26 smithi183 bash[20824]: audit 2024-02-13T03:36:25.510915+0000 mon.a (mon.0) 2645 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-104", "mode": "writeback"}]': finished 2024-02-13T03:36:26.284 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:26 smithi183 bash[20824]: cluster 2024-02-13T03:36:25.511222+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-02-13T03:36:26.284 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:26 smithi183 bash[20824]: audit 2024-02-13T03:36:25.565969+0000 mon.a (mon.0) 2647 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:26 smithi161 bash[27879]: cluster 2024-02-13T03:36:25.501312+0000 mon.a (mon.0) 2644 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:36:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:26 smithi161 bash[27879]: audit 2024-02-13T03:36:25.510915+0000 mon.a (mon.0) 2645 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-104", "mode": "writeback"}]': finished 2024-02-13T03:36:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:26 smithi161 bash[27879]: cluster 2024-02-13T03:36:25.511222+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-02-13T03:36:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:26 smithi161 bash[27879]: audit 2024-02-13T03:36:25.565969+0000 mon.a (mon.0) 2647 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:26 smithi161 bash[19769]: cluster 2024-02-13T03:36:25.501312+0000 mon.a (mon.0) 2644 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:36:26.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:26 smithi161 bash[19769]: audit 2024-02-13T03:36:25.510915+0000 mon.a (mon.0) 2645 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-104", "mode": "writeback"}]': finished 2024-02-13T03:36:26.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:26 smithi161 bash[19769]: cluster 2024-02-13T03:36:25.511222+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-02-13T03:36:26.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:26 smithi161 bash[19769]: audit 2024-02-13T03:36:25.565969+0000 mon.a (mon.0) 2647 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:26.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:36:26 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:36:27.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:27 smithi161 bash[27879]: cluster 2024-02-13T03:36:25.493417+0000 mgr.y (mgr.24473) 599 : cluster [DBG] pgmap v932: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:27.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:27 smithi161 bash[27879]: audit 2024-02-13T03:36:26.282038+0000 mgr.y (mgr.24473) 600 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:27.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:27 smithi161 bash[27879]: audit 2024-02-13T03:36:26.510879+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:36:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:27 smithi161 bash[27879]: cluster 2024-02-13T03:36:26.511096+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-02-13T03:36:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:27 smithi161 bash[27879]: audit 2024-02-13T03:36:26.513224+0000 mon.a (mon.0) 2650 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-104"}]: dispatch 2024-02-13T03:36:27.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:27 smithi161 bash[19769]: cluster 2024-02-13T03:36:25.493417+0000 mgr.y (mgr.24473) 599 : cluster [DBG] pgmap v932: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:27.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:27 smithi161 bash[19769]: audit 2024-02-13T03:36:26.282038+0000 mgr.y (mgr.24473) 600 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:27.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:27 smithi161 bash[19769]: audit 2024-02-13T03:36:26.510879+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:36:27.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:27 smithi161 bash[19769]: cluster 2024-02-13T03:36:26.511096+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-02-13T03:36:27.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:27 smithi161 bash[19769]: audit 2024-02-13T03:36:26.513224+0000 mon.a (mon.0) 2650 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-104"}]: dispatch 2024-02-13T03:36:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:27 smithi183 bash[20824]: cluster 2024-02-13T03:36:25.493417+0000 mgr.y (mgr.24473) 599 : cluster [DBG] pgmap v932: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:27 smithi183 bash[20824]: audit 2024-02-13T03:36:26.282038+0000 mgr.y (mgr.24473) 600 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:27 smithi183 bash[20824]: audit 2024-02-13T03:36:26.510879+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:36:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:27 smithi183 bash[20824]: cluster 2024-02-13T03:36:26.511096+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-02-13T03:36:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:27 smithi183 bash[20824]: audit 2024-02-13T03:36:26.513224+0000 mon.a (mon.0) 2650 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-104"}]: dispatch 2024-02-13T03:36:28.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:28 smithi161 bash[27879]: cluster 2024-02-13T03:36:27.506359+0000 mon.a (mon.0) 2651 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:36:28.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:28 smithi161 bash[27879]: audit 2024-02-13T03:36:27.515795+0000 mon.a (mon.0) 2652 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-104"}]': finished 2024-02-13T03:36:28.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:28 smithi161 bash[27879]: cluster 2024-02-13T03:36:27.516083+0000 mon.a (mon.0) 2653 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-02-13T03:36:28.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:28 smithi161 bash[19769]: cluster 2024-02-13T03:36:27.506359+0000 mon.a (mon.0) 2651 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:36:28.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:28 smithi161 bash[19769]: audit 2024-02-13T03:36:27.515795+0000 mon.a (mon.0) 2652 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-104"}]': finished 2024-02-13T03:36:28.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:28 smithi161 bash[19769]: cluster 2024-02-13T03:36:27.516083+0000 mon.a (mon.0) 2653 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-02-13T03:36:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:28 smithi183 bash[20824]: cluster 2024-02-13T03:36:27.506359+0000 mon.a (mon.0) 2651 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:36:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:28 smithi183 bash[20824]: audit 2024-02-13T03:36:27.515795+0000 mon.a (mon.0) 2652 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-104"}]': finished 2024-02-13T03:36:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:28 smithi183 bash[20824]: cluster 2024-02-13T03:36:27.516083+0000 mon.a (mon.0) 2653 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-02-13T03:36:29.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:29 smithi183 bash[20824]: cluster 2024-02-13T03:36:27.494260+0000 mgr.y (mgr.24473) 601 : cluster [DBG] pgmap v935: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:36:29.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:29 smithi183 bash[20824]: cluster 2024-02-13T03:36:28.517365+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-02-13T03:36:29.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:29 smithi161 bash[27879]: cluster 2024-02-13T03:36:27.494260+0000 mgr.y (mgr.24473) 601 : cluster [DBG] pgmap v935: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:36:29.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:29 smithi161 bash[27879]: cluster 2024-02-13T03:36:28.517365+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-02-13T03:36:29.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:29 smithi161 bash[19769]: cluster 2024-02-13T03:36:27.494260+0000 mgr.y (mgr.24473) 601 : cluster [DBG] pgmap v935: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:36:29.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:29 smithi161 bash[19769]: cluster 2024-02-13T03:36:28.517365+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-02-13T03:36:29.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:36:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:36:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:36:30.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:30 smithi161 bash[27879]: cluster 2024-02-13T03:36:29.495041+0000 mgr.y (mgr.24473) 602 : cluster [DBG] pgmap v938: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:36:30.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:30 smithi161 bash[27879]: cluster 2024-02-13T03:36:29.522330+0000 mon.a (mon.0) 2655 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-02-13T03:36:30.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:30 smithi161 bash[27879]: audit 2024-02-13T03:36:29.530365+0000 mon.a (mon.0) 2656 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:36:30.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:30 smithi161 bash[19769]: cluster 2024-02-13T03:36:29.495041+0000 mgr.y (mgr.24473) 602 : cluster [DBG] pgmap v938: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:36:30.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:30 smithi161 bash[19769]: cluster 2024-02-13T03:36:29.522330+0000 mon.a (mon.0) 2655 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-02-13T03:36:30.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:30 smithi161 bash[19769]: audit 2024-02-13T03:36:29.530365+0000 mon.a (mon.0) 2656 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:36:30.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:30 smithi183 bash[20824]: cluster 2024-02-13T03:36:29.495041+0000 mgr.y (mgr.24473) 602 : cluster [DBG] pgmap v938: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:36:30.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:30 smithi183 bash[20824]: cluster 2024-02-13T03:36:29.522330+0000 mon.a (mon.0) 2655 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-02-13T03:36:30.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:30 smithi183 bash[20824]: audit 2024-02-13T03:36:29.530365+0000 mon.a (mon.0) 2656 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:36:31.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:36:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:36:31] "GET /metrics HTTP/1.1" 200 189154 "" "Prometheus/2.33.4" 2024-02-13T03:36:31.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:31 smithi161 bash[27879]: audit 2024-02-13T03:36:30.525369+0000 mon.a (mon.0) 2657 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:36:31.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:31 smithi161 bash[27879]: cluster 2024-02-13T03:36:30.525706+0000 mon.a (mon.0) 2658 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-02-13T03:36:31.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:31 smithi161 bash[19769]: audit 2024-02-13T03:36:30.525369+0000 mon.a (mon.0) 2657 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:36:31.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:31 smithi161 bash[19769]: cluster 2024-02-13T03:36:30.525706+0000 mon.a (mon.0) 2658 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-02-13T03:36:31.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:31 smithi183 bash[20824]: audit 2024-02-13T03:36:30.525369+0000 mon.a (mon.0) 2657 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:36:31.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:31 smithi183 bash[20824]: cluster 2024-02-13T03:36:30.525706+0000 mon.a (mon.0) 2658 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-02-13T03:36:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:32 smithi161 bash[27879]: cluster 2024-02-13T03:36:31.497045+0000 mgr.y (mgr.24473) 603 : cluster [DBG] pgmap v941: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-02-13T03:36:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:32 smithi161 bash[27879]: cluster 2024-02-13T03:36:31.547029+0000 mon.a (mon.0) 2659 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-02-13T03:36:32.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:32 smithi161 bash[27879]: audit 2024-02-13T03:36:31.561376+0000 mon.a (mon.0) 2660 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:36:32.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:32 smithi161 bash[19769]: cluster 2024-02-13T03:36:31.497045+0000 mgr.y (mgr.24473) 603 : cluster [DBG] pgmap v941: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-02-13T03:36:32.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:32 smithi161 bash[19769]: cluster 2024-02-13T03:36:31.547029+0000 mon.a (mon.0) 2659 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-02-13T03:36:32.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:32 smithi161 bash[19769]: audit 2024-02-13T03:36:31.561376+0000 mon.a (mon.0) 2660 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:36:32.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:32 smithi183 bash[20824]: cluster 2024-02-13T03:36:31.497045+0000 mgr.y (mgr.24473) 603 : cluster [DBG] pgmap v941: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-02-13T03:36:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:32 smithi183 bash[20824]: cluster 2024-02-13T03:36:31.547029+0000 mon.a (mon.0) 2659 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-02-13T03:36:32.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:32 smithi183 bash[20824]: audit 2024-02-13T03:36:31.561376+0000 mon.a (mon.0) 2660 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:36:33.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:33 smithi161 bash[27879]: audit 2024-02-13T03:36:32.550414+0000 mon.a (mon.0) 2661 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:36:33.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:33 smithi161 bash[27879]: cluster 2024-02-13T03:36:32.550520+0000 mon.a (mon.0) 2662 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-02-13T03:36:33.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:33 smithi161 bash[27879]: audit 2024-02-13T03:36:32.557128+0000 mon.a (mon.0) 2663 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-106"}]: dispatch 2024-02-13T03:36:33.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:33 smithi161 bash[19769]: audit 2024-02-13T03:36:32.550414+0000 mon.a (mon.0) 2661 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:36:33.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:33 smithi161 bash[19769]: cluster 2024-02-13T03:36:32.550520+0000 mon.a (mon.0) 2662 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-02-13T03:36:33.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:33 smithi161 bash[19769]: audit 2024-02-13T03:36:32.557128+0000 mon.a (mon.0) 2663 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-106"}]: dispatch 2024-02-13T03:36:33.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:33 smithi183 bash[20824]: audit 2024-02-13T03:36:32.550414+0000 mon.a (mon.0) 2661 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:36:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:33 smithi183 bash[20824]: cluster 2024-02-13T03:36:32.550520+0000 mon.a (mon.0) 2662 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-02-13T03:36:33.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:33 smithi183 bash[20824]: audit 2024-02-13T03:36:32.557128+0000 mon.a (mon.0) 2663 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-106"}]: dispatch 2024-02-13T03:36:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:34 smithi161 bash[27879]: cluster 2024-02-13T03:36:33.497877+0000 mgr.y (mgr.24473) 604 : cluster [DBG] pgmap v944: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-13T03:36:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:34 smithi161 bash[27879]: audit 2024-02-13T03:36:33.573098+0000 mon.a (mon.0) 2664 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-106"}]': finished 2024-02-13T03:36:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:34 smithi161 bash[27879]: cluster 2024-02-13T03:36:33.573197+0000 mon.a (mon.0) 2665 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-02-13T03:36:34.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:34 smithi161 bash[27879]: audit 2024-02-13T03:36:33.577913+0000 mon.a (mon.0) 2666 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-106", "mode": "writeback"}]: dispatch 2024-02-13T03:36:34.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:34 smithi161 bash[19769]: cluster 2024-02-13T03:36:33.497877+0000 mgr.y (mgr.24473) 604 : cluster [DBG] pgmap v944: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-13T03:36:34.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:34 smithi161 bash[19769]: audit 2024-02-13T03:36:33.573098+0000 mon.a (mon.0) 2664 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-106"}]': finished 2024-02-13T03:36:34.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:34 smithi161 bash[19769]: cluster 2024-02-13T03:36:33.573197+0000 mon.a (mon.0) 2665 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-02-13T03:36:34.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:34 smithi161 bash[19769]: audit 2024-02-13T03:36:33.577913+0000 mon.a (mon.0) 2666 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-106", "mode": "writeback"}]: dispatch 2024-02-13T03:36:34.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:34 smithi183 bash[20824]: cluster 2024-02-13T03:36:33.497877+0000 mgr.y (mgr.24473) 604 : cluster [DBG] pgmap v944: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-13T03:36:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:34 smithi183 bash[20824]: audit 2024-02-13T03:36:33.573098+0000 mon.a (mon.0) 2664 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-106"}]': finished 2024-02-13T03:36:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:34 smithi183 bash[20824]: cluster 2024-02-13T03:36:33.573197+0000 mon.a (mon.0) 2665 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-02-13T03:36:34.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:34 smithi183 bash[20824]: audit 2024-02-13T03:36:33.577913+0000 mon.a (mon.0) 2666 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-106", "mode": "writeback"}]: dispatch 2024-02-13T03:36:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:35 smithi161 bash[27879]: cluster 2024-02-13T03:36:34.567479+0000 mon.a (mon.0) 2667 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:36:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:35 smithi161 bash[27879]: audit 2024-02-13T03:36:34.577497+0000 mon.a (mon.0) 2668 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-106", "mode": "writeback"}]': finished 2024-02-13T03:36:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:35 smithi161 bash[27879]: cluster 2024-02-13T03:36:34.577701+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-02-13T03:36:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:35 smithi161 bash[27879]: audit 2024-02-13T03:36:34.600839+0000 mon.a (mon.0) 2670 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.6"}]: dispatch 2024-02-13T03:36:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:35 smithi161 bash[27879]: audit 2024-02-13T03:36:34.601054+0000 mgr.y (mgr.24473) 605 : audit [DBG] from='mon.0 v1:172.21.15.161:0/4039416131' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.6"}]: dispatch 2024-02-13T03:36:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:35 smithi161 bash[27879]: audit 2024-02-13T03:36:34.791506+0000 mon.a (mon.0) 2671 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:36:35.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:35 smithi161 bash[19769]: cluster 2024-02-13T03:36:34.567479+0000 mon.a (mon.0) 2667 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:36:35.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:35 smithi161 bash[19769]: audit 2024-02-13T03:36:34.577497+0000 mon.a (mon.0) 2668 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-106", "mode": "writeback"}]': finished 2024-02-13T03:36:35.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:35 smithi161 bash[19769]: cluster 2024-02-13T03:36:34.577701+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-02-13T03:36:35.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:35 smithi161 bash[19769]: audit 2024-02-13T03:36:34.600839+0000 mon.a (mon.0) 2670 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.6"}]: dispatch 2024-02-13T03:36:35.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:35 smithi161 bash[19769]: audit 2024-02-13T03:36:34.601054+0000 mgr.y (mgr.24473) 605 : audit [DBG] from='mon.0 v1:172.21.15.161:0/4039416131' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.6"}]: dispatch 2024-02-13T03:36:35.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:35 smithi161 bash[19769]: audit 2024-02-13T03:36:34.791506+0000 mon.a (mon.0) 2671 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:36:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:35 smithi183 bash[20824]: cluster 2024-02-13T03:36:34.567479+0000 mon.a (mon.0) 2667 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:36:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:35 smithi183 bash[20824]: audit 2024-02-13T03:36:34.577497+0000 mon.a (mon.0) 2668 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-106", "mode": "writeback"}]': finished 2024-02-13T03:36:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:35 smithi183 bash[20824]: cluster 2024-02-13T03:36:34.577701+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-02-13T03:36:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:35 smithi183 bash[20824]: audit 2024-02-13T03:36:34.600839+0000 mon.a (mon.0) 2670 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.6"}]: dispatch 2024-02-13T03:36:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:35 smithi183 bash[20824]: audit 2024-02-13T03:36:34.601054+0000 mgr.y (mgr.24473) 605 : audit [DBG] from='mon.0 v1:172.21.15.161:0/4039416131' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.6"}]: dispatch 2024-02-13T03:36:35.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:35 smithi183 bash[20824]: audit 2024-02-13T03:36:34.791506+0000 mon.a (mon.0) 2671 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:36:36.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:36:36 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:36:37.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:36 smithi161 bash[19769]: cluster 2024-02-13T03:36:35.498673+0000 mgr.y (mgr.24473) 606 : cluster [DBG] pgmap v947: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:37.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:36 smithi161 bash[19769]: audit 2024-02-13T03:36:36.293306+0000 mgr.y (mgr.24473) 607 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:37.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:36 smithi161 bash[19769]: audit 2024-02-13T03:36:36.495175+0000 mon.c (mon.2) 565 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.0", "id": [7, 4]}]: dispatch 2024-02-13T03:36:37.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:36 smithi161 bash[19769]: audit 2024-02-13T03:36:36.495930+0000 mon.c (mon.2) 566 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1e", "id": [7, 3]}]: dispatch 2024-02-13T03:36:37.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:36 smithi161 bash[19769]: audit 2024-02-13T03:36:36.496027+0000 mon.a (mon.0) 2672 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.0", "id": [7, 4]}]: dispatch 2024-02-13T03:36:37.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:36 smithi161 bash[19769]: audit 2024-02-13T03:36:36.496926+0000 mon.a (mon.0) 2673 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1e", "id": [7, 3]}]: dispatch 2024-02-13T03:36:37.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:36 smithi161 bash[27879]: cluster 2024-02-13T03:36:35.498673+0000 mgr.y (mgr.24473) 606 : cluster [DBG] pgmap v947: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:37.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:36 smithi161 bash[27879]: audit 2024-02-13T03:36:36.293306+0000 mgr.y (mgr.24473) 607 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:37.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:36 smithi161 bash[27879]: audit 2024-02-13T03:36:36.495175+0000 mon.c (mon.2) 565 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.0", "id": [7, 4]}]: dispatch 2024-02-13T03:36:37.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:36 smithi161 bash[27879]: audit 2024-02-13T03:36:36.495930+0000 mon.c (mon.2) 566 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1e", "id": [7, 3]}]: dispatch 2024-02-13T03:36:37.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:36 smithi161 bash[27879]: audit 2024-02-13T03:36:36.496027+0000 mon.a (mon.0) 2672 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.0", "id": [7, 4]}]: dispatch 2024-02-13T03:36:37.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:36 smithi161 bash[27879]: audit 2024-02-13T03:36:36.496926+0000 mon.a (mon.0) 2673 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1e", "id": [7, 3]}]: dispatch 2024-02-13T03:36:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:36 smithi183 bash[20824]: cluster 2024-02-13T03:36:35.498673+0000 mgr.y (mgr.24473) 606 : cluster [DBG] pgmap v947: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:36 smithi183 bash[20824]: audit 2024-02-13T03:36:36.293306+0000 mgr.y (mgr.24473) 607 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:36 smithi183 bash[20824]: audit 2024-02-13T03:36:36.495175+0000 mon.c (mon.2) 565 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.0", "id": [7, 4]}]: dispatch 2024-02-13T03:36:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:36 smithi183 bash[20824]: audit 2024-02-13T03:36:36.495930+0000 mon.c (mon.2) 566 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1e", "id": [7, 3]}]: dispatch 2024-02-13T03:36:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:36 smithi183 bash[20824]: audit 2024-02-13T03:36:36.496027+0000 mon.a (mon.0) 2672 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.0", "id": [7, 4]}]: dispatch 2024-02-13T03:36:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:36 smithi183 bash[20824]: audit 2024-02-13T03:36:36.496926+0000 mon.a (mon.0) 2673 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1e", "id": [7, 3]}]: dispatch 2024-02-13T03:36:38.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:37 smithi161 bash[27879]: audit 2024-02-13T03:36:36.806775+0000 mon.a (mon.0) 2674 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.0", "id": [7, 4]}]': finished 2024-02-13T03:36:38.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:37 smithi161 bash[27879]: audit 2024-02-13T03:36:36.807000+0000 mon.a (mon.0) 2675 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1e", "id": [7, 3]}]': finished 2024-02-13T03:36:38.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:37 smithi161 bash[27879]: cluster 2024-02-13T03:36:36.807159+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-02-13T03:36:38.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:37 smithi161 bash[19769]: audit 2024-02-13T03:36:36.806775+0000 mon.a (mon.0) 2674 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.0", "id": [7, 4]}]': finished 2024-02-13T03:36:38.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:37 smithi161 bash[19769]: audit 2024-02-13T03:36:36.807000+0000 mon.a (mon.0) 2675 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1e", "id": [7, 3]}]': finished 2024-02-13T03:36:38.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:37 smithi161 bash[19769]: cluster 2024-02-13T03:36:36.807159+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-02-13T03:36:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:37 smithi183 bash[20824]: audit 2024-02-13T03:36:36.806775+0000 mon.a (mon.0) 2674 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.0", "id": [7, 4]}]': finished 2024-02-13T03:36:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:37 smithi183 bash[20824]: audit 2024-02-13T03:36:36.807000+0000 mon.a (mon.0) 2675 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1e", "id": [7, 3]}]': finished 2024-02-13T03:36:38.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:37 smithi183 bash[20824]: cluster 2024-02-13T03:36:36.807159+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-02-13T03:36:39.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:38 smithi161 bash[27879]: cluster 2024-02-13T03:36:37.499550+0000 mgr.y (mgr.24473) 608 : cluster [DBG] pgmap v949: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 620 B/s wr, 2 op/s 2024-02-13T03:36:39.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:38 smithi161 bash[27879]: cluster 2024-02-13T03:36:37.780138+0000 osd.6 (osd.6) 5 : cluster [DBG] 237.6 scrub starts 2024-02-13T03:36:39.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:38 smithi161 bash[27879]: cluster 2024-02-13T03:36:37.783728+0000 osd.6 (osd.6) 6 : cluster [DBG] 237.6 scrub ok 2024-02-13T03:36:39.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:38 smithi161 bash[27879]: cluster 2024-02-13T03:36:37.817678+0000 mon.a (mon.0) 2677 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-02-13T03:36:39.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:38 smithi161 bash[19769]: cluster 2024-02-13T03:36:37.499550+0000 mgr.y (mgr.24473) 608 : cluster [DBG] pgmap v949: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 620 B/s wr, 2 op/s 2024-02-13T03:36:39.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:38 smithi161 bash[19769]: cluster 2024-02-13T03:36:37.780138+0000 osd.6 (osd.6) 5 : cluster [DBG] 237.6 scrub starts 2024-02-13T03:36:39.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:38 smithi161 bash[19769]: cluster 2024-02-13T03:36:37.783728+0000 osd.6 (osd.6) 6 : cluster [DBG] 237.6 scrub ok 2024-02-13T03:36:39.177 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:38 smithi161 bash[19769]: cluster 2024-02-13T03:36:37.817678+0000 mon.a (mon.0) 2677 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-02-13T03:36:39.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:38 smithi183 bash[20824]: cluster 2024-02-13T03:36:37.499550+0000 mgr.y (mgr.24473) 608 : cluster [DBG] pgmap v949: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 620 B/s wr, 2 op/s 2024-02-13T03:36:39.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:38 smithi183 bash[20824]: cluster 2024-02-13T03:36:37.780138+0000 osd.6 (osd.6) 5 : cluster [DBG] 237.6 scrub starts 2024-02-13T03:36:39.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:38 smithi183 bash[20824]: cluster 2024-02-13T03:36:37.783728+0000 osd.6 (osd.6) 6 : cluster [DBG] 237.6 scrub ok 2024-02-13T03:36:39.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:38 smithi183 bash[20824]: cluster 2024-02-13T03:36:37.817678+0000 mon.a (mon.0) 2677 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-02-13T03:36:39.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:36:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:36:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:36:41.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:36:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:36:41] "GET /metrics HTTP/1.1" 200 189252 "" "Prometheus/2.33.4" 2024-02-13T03:36:41.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:41 smithi161 bash[27879]: cluster 2024-02-13T03:36:39.500368+0000 mgr.y (mgr.24473) 609 : cluster [DBG] pgmap v951: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 518 B/s wr, 2 op/s 2024-02-13T03:36:41.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:41 smithi161 bash[19769]: cluster 2024-02-13T03:36:39.500368+0000 mgr.y (mgr.24473) 609 : cluster [DBG] pgmap v951: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 518 B/s wr, 2 op/s 2024-02-13T03:36:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:41 smithi183 bash[20824]: cluster 2024-02-13T03:36:39.500368+0000 mgr.y (mgr.24473) 609 : cluster [DBG] pgmap v951: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 518 B/s wr, 2 op/s 2024-02-13T03:36:43.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:43 smithi161 bash[27879]: cluster 2024-02-13T03:36:41.502586+0000 mgr.y (mgr.24473) 610 : cluster [DBG] pgmap v952: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 443 B/s wr, 2 op/s 2024-02-13T03:36:43.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:43 smithi161 bash[19769]: cluster 2024-02-13T03:36:41.502586+0000 mgr.y (mgr.24473) 610 : cluster [DBG] pgmap v952: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 443 B/s wr, 2 op/s 2024-02-13T03:36:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:43 smithi183 bash[20824]: cluster 2024-02-13T03:36:41.502586+0000 mgr.y (mgr.24473) 610 : cluster [DBG] pgmap v952: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 443 B/s wr, 2 op/s 2024-02-13T03:36:45.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:45 smithi161 bash[27879]: cluster 2024-02-13T03:36:43.503550+0000 mgr.y (mgr.24473) 611 : cluster [DBG] pgmap v953: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-02-13T03:36:45.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:45 smithi161 bash[19769]: cluster 2024-02-13T03:36:43.503550+0000 mgr.y (mgr.24473) 611 : cluster [DBG] pgmap v953: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-02-13T03:36:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:45 smithi183 bash[20824]: cluster 2024-02-13T03:36:43.503550+0000 mgr.y (mgr.24473) 611 : cluster [DBG] pgmap v953: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-02-13T03:36:46.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:36:46 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:36:47.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:47 smithi161 bash[27879]: cluster 2024-02-13T03:36:45.504379+0000 mgr.y (mgr.24473) 612 : cluster [DBG] pgmap v954: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 588 B/s rd, 0 op/s 2024-02-13T03:36:47.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:47 smithi161 bash[27879]: audit 2024-02-13T03:36:46.304602+0000 mgr.y (mgr.24473) 613 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:47.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:47 smithi161 bash[19769]: cluster 2024-02-13T03:36:45.504379+0000 mgr.y (mgr.24473) 612 : cluster [DBG] pgmap v954: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 588 B/s rd, 0 op/s 2024-02-13T03:36:47.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:47 smithi161 bash[19769]: audit 2024-02-13T03:36:46.304602+0000 mgr.y (mgr.24473) 613 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:47 smithi183 bash[20824]: cluster 2024-02-13T03:36:45.504379+0000 mgr.y (mgr.24473) 612 : cluster [DBG] pgmap v954: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 588 B/s rd, 0 op/s 2024-02-13T03:36:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:47 smithi183 bash[20824]: audit 2024-02-13T03:36:46.304602+0000 mgr.y (mgr.24473) 613 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:49.330 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:49 smithi183 bash[20824]: cluster 2024-02-13T03:36:47.506439+0000 mgr.y (mgr.24473) 614 : cluster [DBG] pgmap v955: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:36:49.331 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:36:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:36:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:36:49.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:49 smithi161 bash[27879]: cluster 2024-02-13T03:36:47.506439+0000 mgr.y (mgr.24473) 614 : cluster [DBG] pgmap v955: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:36:49.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:49 smithi161 bash[19769]: cluster 2024-02-13T03:36:47.506439+0000 mgr.y (mgr.24473) 614 : cluster [DBG] pgmap v955: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:36:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:36:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:36:51] "GET /metrics HTTP/1.1" 200 189282 "" "Prometheus/2.33.4" 2024-02-13T03:36:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:51 smithi161 bash[27879]: cluster 2024-02-13T03:36:49.507283+0000 mgr.y (mgr.24473) 615 : cluster [DBG] pgmap v956: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 875 B/s rd, 0 op/s 2024-02-13T03:36:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:51 smithi161 bash[27879]: cluster 2024-02-13T03:36:50.050371+0000 mon.a (mon.0) 2678 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-02-13T03:36:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:51 smithi161 bash[27879]: audit 2024-02-13T03:36:50.083368+0000 mon.a (mon.0) 2679 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:51 smithi161 bash[19769]: cluster 2024-02-13T03:36:49.507283+0000 mgr.y (mgr.24473) 615 : cluster [DBG] pgmap v956: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 875 B/s rd, 0 op/s 2024-02-13T03:36:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:51 smithi161 bash[19769]: cluster 2024-02-13T03:36:50.050371+0000 mon.a (mon.0) 2678 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-02-13T03:36:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:51 smithi161 bash[19769]: audit 2024-02-13T03:36:50.083368+0000 mon.a (mon.0) 2679 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:51 smithi183 bash[20824]: cluster 2024-02-13T03:36:49.507283+0000 mgr.y (mgr.24473) 615 : cluster [DBG] pgmap v956: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 875 B/s rd, 0 op/s 2024-02-13T03:36:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:51 smithi183 bash[20824]: cluster 2024-02-13T03:36:50.050371+0000 mon.a (mon.0) 2678 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-02-13T03:36:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:51 smithi183 bash[20824]: audit 2024-02-13T03:36:50.083368+0000 mon.a (mon.0) 2679 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:36:52.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:52 smithi161 bash[27879]: audit 2024-02-13T03:36:51.056524+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:36:52.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:52 smithi161 bash[27879]: cluster 2024-02-13T03:36:51.056640+0000 mon.a (mon.0) 2681 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-02-13T03:36:52.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:52 smithi161 bash[27879]: audit 2024-02-13T03:36:51.059432+0000 mon.a (mon.0) 2682 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-106"}]: dispatch 2024-02-13T03:36:52.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:52 smithi161 bash[19769]: audit 2024-02-13T03:36:51.056524+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:36:52.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:52 smithi161 bash[19769]: cluster 2024-02-13T03:36:51.056640+0000 mon.a (mon.0) 2681 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-02-13T03:36:52.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:52 smithi161 bash[19769]: audit 2024-02-13T03:36:51.059432+0000 mon.a (mon.0) 2682 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-106"}]: dispatch 2024-02-13T03:36:52.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:52 smithi183 bash[20824]: audit 2024-02-13T03:36:51.056524+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:36:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:52 smithi183 bash[20824]: cluster 2024-02-13T03:36:51.056640+0000 mon.a (mon.0) 2681 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-02-13T03:36:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:52 smithi183 bash[20824]: audit 2024-02-13T03:36:51.059432+0000 mon.a (mon.0) 2682 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-106"}]: dispatch 2024-02-13T03:36:53.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:53 smithi161 bash[27879]: cluster 2024-02-13T03:36:51.508110+0000 mgr.y (mgr.24473) 616 : cluster [DBG] pgmap v959: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-13T03:36:53.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:53 smithi161 bash[27879]: cluster 2024-02-13T03:36:52.051712+0000 mon.a (mon.0) 2683 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:36:53.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:53 smithi161 bash[27879]: audit 2024-02-13T03:36:52.060219+0000 mon.a (mon.0) 2684 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-106"}]': finished 2024-02-13T03:36:53.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:53 smithi161 bash[27879]: cluster 2024-02-13T03:36:52.063598+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-02-13T03:36:53.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:53 smithi161 bash[19769]: cluster 2024-02-13T03:36:51.508110+0000 mgr.y (mgr.24473) 616 : cluster [DBG] pgmap v959: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-13T03:36:53.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:53 smithi161 bash[19769]: cluster 2024-02-13T03:36:52.051712+0000 mon.a (mon.0) 2683 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:36:53.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:53 smithi161 bash[19769]: audit 2024-02-13T03:36:52.060219+0000 mon.a (mon.0) 2684 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-106"}]': finished 2024-02-13T03:36:53.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:53 smithi161 bash[19769]: cluster 2024-02-13T03:36:52.063598+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-02-13T03:36:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:53 smithi183 bash[20824]: cluster 2024-02-13T03:36:51.508110+0000 mgr.y (mgr.24473) 616 : cluster [DBG] pgmap v959: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-13T03:36:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:53 smithi183 bash[20824]: cluster 2024-02-13T03:36:52.051712+0000 mon.a (mon.0) 2683 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:36:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:53 smithi183 bash[20824]: audit 2024-02-13T03:36:52.060219+0000 mon.a (mon.0) 2684 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-106"}]': finished 2024-02-13T03:36:53.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:53 smithi183 bash[20824]: cluster 2024-02-13T03:36:52.063598+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-02-13T03:36:54.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:54 smithi161 bash[27879]: cluster 2024-02-13T03:36:53.074377+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-02-13T03:36:54.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:54 smithi161 bash[27879]: cluster 2024-02-13T03:36:53.508866+0000 mgr.y (mgr.24473) 617 : cluster [DBG] pgmap v962: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-13T03:36:54.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:54 smithi161 bash[27879]: cluster 2024-02-13T03:36:54.015333+0000 mon.a (mon.0) 2687 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-02-13T03:36:54.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:54 smithi161 bash[27879]: audit 2024-02-13T03:36:54.022723+0000 mon.a (mon.0) 2688 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:36:54.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:54 smithi161 bash[19769]: cluster 2024-02-13T03:36:53.074377+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-02-13T03:36:54.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:54 smithi161 bash[19769]: cluster 2024-02-13T03:36:53.508866+0000 mgr.y (mgr.24473) 617 : cluster [DBG] pgmap v962: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-13T03:36:54.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:54 smithi161 bash[19769]: cluster 2024-02-13T03:36:54.015333+0000 mon.a (mon.0) 2687 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-02-13T03:36:54.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:54 smithi161 bash[19769]: audit 2024-02-13T03:36:54.022723+0000 mon.a (mon.0) 2688 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:36:54.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:54 smithi183 bash[20824]: cluster 2024-02-13T03:36:53.074377+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-02-13T03:36:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:54 smithi183 bash[20824]: cluster 2024-02-13T03:36:53.508866+0000 mgr.y (mgr.24473) 617 : cluster [DBG] pgmap v962: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-13T03:36:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:54 smithi183 bash[20824]: cluster 2024-02-13T03:36:54.015333+0000 mon.a (mon.0) 2687 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-02-13T03:36:54.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:54 smithi183 bash[20824]: audit 2024-02-13T03:36:54.022723+0000 mon.a (mon.0) 2688 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:36:56.316 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:56 smithi183 bash[20824]: audit 2024-02-13T03:36:55.022116+0000 mon.a (mon.0) 2689 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:36:56.316 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:56 smithi183 bash[20824]: cluster 2024-02-13T03:36:55.022771+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-02-13T03:36:56.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:56 smithi161 bash[27879]: audit 2024-02-13T03:36:55.022116+0000 mon.a (mon.0) 2689 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:36:56.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:56 smithi161 bash[27879]: cluster 2024-02-13T03:36:55.022771+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-02-13T03:36:56.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:56 smithi161 bash[19769]: audit 2024-02-13T03:36:55.022116+0000 mon.a (mon.0) 2689 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:36:56.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:56 smithi161 bash[19769]: cluster 2024-02-13T03:36:55.022771+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-02-13T03:36:56.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:36:56 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:36:57.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:57 smithi161 bash[19769]: cluster 2024-02-13T03:36:55.509811+0000 mgr.y (mgr.24473) 618 : cluster [DBG] pgmap v965: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:57.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:57 smithi161 bash[19769]: cluster 2024-02-13T03:36:56.021080+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-02-13T03:36:57.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:57 smithi161 bash[19769]: audit 2024-02-13T03:36:56.032661+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:36:57.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:57 smithi161 bash[19769]: audit 2024-02-13T03:36:56.314267+0000 mgr.y (mgr.24473) 619 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:57.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:57 smithi161 bash[27879]: cluster 2024-02-13T03:36:55.509811+0000 mgr.y (mgr.24473) 618 : cluster [DBG] pgmap v965: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:57 smithi161 bash[27879]: cluster 2024-02-13T03:36:56.021080+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-02-13T03:36:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:57 smithi161 bash[27879]: audit 2024-02-13T03:36:56.032661+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:36:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:57 smithi161 bash[27879]: audit 2024-02-13T03:36:56.314267+0000 mgr.y (mgr.24473) 619 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:57 smithi183 bash[20824]: cluster 2024-02-13T03:36:55.509811+0000 mgr.y (mgr.24473) 618 : cluster [DBG] pgmap v965: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:36:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:57 smithi183 bash[20824]: cluster 2024-02-13T03:36:56.021080+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-02-13T03:36:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:57 smithi183 bash[20824]: audit 2024-02-13T03:36:56.032661+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:36:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:57 smithi183 bash[20824]: audit 2024-02-13T03:36:56.314267+0000 mgr.y (mgr.24473) 619 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:36:58.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:58 smithi161 bash[27879]: audit 2024-02-13T03:36:57.040753+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:36:58.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:58 smithi161 bash[27879]: cluster 2024-02-13T03:36:57.040845+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-02-13T03:36:58.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:58 smithi161 bash[27879]: audit 2024-02-13T03:36:57.041630+0000 mon.a (mon.0) 2695 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-108"}]: dispatch 2024-02-13T03:36:58.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:58 smithi161 bash[19769]: audit 2024-02-13T03:36:57.040753+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:36:58.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:58 smithi161 bash[19769]: cluster 2024-02-13T03:36:57.040845+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-02-13T03:36:58.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:58 smithi161 bash[19769]: audit 2024-02-13T03:36:57.041630+0000 mon.a (mon.0) 2695 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-108"}]: dispatch 2024-02-13T03:36:58.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:58 smithi183 bash[20824]: audit 2024-02-13T03:36:57.040753+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:36:58.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:58 smithi183 bash[20824]: cluster 2024-02-13T03:36:57.040845+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-02-13T03:36:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:58 smithi183 bash[20824]: audit 2024-02-13T03:36:57.041630+0000 mon.a (mon.0) 2695 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-108"}]: dispatch 2024-02-13T03:36:59.330 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:59 smithi183 bash[20824]: cluster 2024-02-13T03:36:57.510718+0000 mgr.y (mgr.24473) 620 : cluster [DBG] pgmap v968: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:36:59.330 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:59 smithi183 bash[20824]: audit 2024-02-13T03:36:58.045246+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-108"}]': finished 2024-02-13T03:36:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:59 smithi183 bash[20824]: cluster 2024-02-13T03:36:58.045512+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-02-13T03:36:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:59 smithi183 bash[20824]: audit 2024-02-13T03:36:58.046707+0000 mon.a (mon.0) 2698 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-108", "mode": "writeback"}]: dispatch 2024-02-13T03:36:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:59 smithi183 bash[20824]: cluster 2024-02-13T03:36:59.039186+0000 mon.a (mon.0) 2699 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:36:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:59 smithi183 bash[20824]: audit 2024-02-13T03:36:59.046165+0000 mon.a (mon.0) 2700 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-108", "mode": "writeback"}]': finished 2024-02-13T03:36:59.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:36:59 smithi183 bash[20824]: cluster 2024-02-13T03:36:59.046548+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-02-13T03:36:59.331 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:36:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:36:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:36:59.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[27879]: cluster 2024-02-13T03:36:57.510718+0000 mgr.y (mgr.24473) 620 : cluster [DBG] pgmap v968: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:36:59.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[27879]: audit 2024-02-13T03:36:58.045246+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-108"}]': finished 2024-02-13T03:36:59.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[27879]: cluster 2024-02-13T03:36:58.045512+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-02-13T03:36:59.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[27879]: audit 2024-02-13T03:36:58.046707+0000 mon.a (mon.0) 2698 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-108", "mode": "writeback"}]: dispatch 2024-02-13T03:36:59.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[27879]: cluster 2024-02-13T03:36:59.039186+0000 mon.a (mon.0) 2699 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:36:59.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[27879]: audit 2024-02-13T03:36:59.046165+0000 mon.a (mon.0) 2700 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-108", "mode": "writeback"}]': finished 2024-02-13T03:36:59.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[27879]: cluster 2024-02-13T03:36:59.046548+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-02-13T03:36:59.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[19769]: cluster 2024-02-13T03:36:57.510718+0000 mgr.y (mgr.24473) 620 : cluster [DBG] pgmap v968: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:36:59.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[19769]: audit 2024-02-13T03:36:58.045246+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-108"}]': finished 2024-02-13T03:36:59.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[19769]: cluster 2024-02-13T03:36:58.045512+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-02-13T03:36:59.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[19769]: audit 2024-02-13T03:36:58.046707+0000 mon.a (mon.0) 2698 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-108", "mode": "writeback"}]: dispatch 2024-02-13T03:36:59.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[19769]: cluster 2024-02-13T03:36:59.039186+0000 mon.a (mon.0) 2699 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:36:59.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[19769]: audit 2024-02-13T03:36:59.046165+0000 mon.a (mon.0) 2700 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-108", "mode": "writeback"}]': finished 2024-02-13T03:36:59.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:36:59 smithi161 bash[19769]: cluster 2024-02-13T03:36:59.046548+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-02-13T03:37:01.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:37:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:37:01] "GET /metrics HTTP/1.1" 200 189282 "" "Prometheus/2.33.4" 2024-02-13T03:37:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:01 smithi161 bash[27879]: cluster 2024-02-13T03:36:59.511554+0000 mgr.y (mgr.24473) 621 : cluster [DBG] pgmap v971: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:01 smithi161 bash[27879]: cluster 2024-02-13T03:37:00.049538+0000 mon.a (mon.0) 2702 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-02-13T03:37:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:01 smithi161 bash[27879]: audit 2024-02-13T03:37:00.086997+0000 mon.a (mon.0) 2703 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:01 smithi161 bash[19769]: cluster 2024-02-13T03:36:59.511554+0000 mgr.y (mgr.24473) 621 : cluster [DBG] pgmap v971: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:01 smithi161 bash[19769]: cluster 2024-02-13T03:37:00.049538+0000 mon.a (mon.0) 2702 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-02-13T03:37:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:01 smithi161 bash[19769]: audit 2024-02-13T03:37:00.086997+0000 mon.a (mon.0) 2703 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:01 smithi183 bash[20824]: cluster 2024-02-13T03:36:59.511554+0000 mgr.y (mgr.24473) 621 : cluster [DBG] pgmap v971: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:01 smithi183 bash[20824]: cluster 2024-02-13T03:37:00.049538+0000 mon.a (mon.0) 2702 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-02-13T03:37:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:01 smithi183 bash[20824]: audit 2024-02-13T03:37:00.086997+0000 mon.a (mon.0) 2703 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:02.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:02 smithi161 bash[19769]: audit 2024-02-13T03:37:01.066820+0000 mon.a (mon.0) 2704 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:02.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:02 smithi161 bash[19769]: cluster 2024-02-13T03:37:01.066922+0000 mon.a (mon.0) 2705 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-02-13T03:37:02.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:02 smithi161 bash[19769]: audit 2024-02-13T03:37:01.067493+0000 mon.a (mon.0) 2706 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-108"}]: dispatch 2024-02-13T03:37:02.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:02 smithi161 bash[27879]: audit 2024-02-13T03:37:01.066820+0000 mon.a (mon.0) 2704 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:02.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:02 smithi161 bash[27879]: cluster 2024-02-13T03:37:01.066922+0000 mon.a (mon.0) 2705 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-02-13T03:37:02.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:02 smithi161 bash[27879]: audit 2024-02-13T03:37:01.067493+0000 mon.a (mon.0) 2706 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-108"}]: dispatch 2024-02-13T03:37:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:02 smithi183 bash[20824]: audit 2024-02-13T03:37:01.066820+0000 mon.a (mon.0) 2704 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:02 smithi183 bash[20824]: cluster 2024-02-13T03:37:01.066922+0000 mon.a (mon.0) 2705 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-02-13T03:37:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:02 smithi183 bash[20824]: audit 2024-02-13T03:37:01.067493+0000 mon.a (mon.0) 2706 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-108"}]: dispatch 2024-02-13T03:37:03.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:03 smithi161 bash[27879]: cluster 2024-02-13T03:37:01.512394+0000 mgr.y (mgr.24473) 622 : cluster [DBG] pgmap v974: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:03.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:03 smithi161 bash[27879]: cluster 2024-02-13T03:37:02.059599+0000 mon.a (mon.0) 2707 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:03.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:03 smithi161 bash[27879]: audit 2024-02-13T03:37:02.076816+0000 mon.a (mon.0) 2708 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-108"}]': finished 2024-02-13T03:37:03.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:03 smithi161 bash[27879]: cluster 2024-02-13T03:37:02.076926+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-02-13T03:37:03.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:03 smithi161 bash[19769]: cluster 2024-02-13T03:37:01.512394+0000 mgr.y (mgr.24473) 622 : cluster [DBG] pgmap v974: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:03.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:03 smithi161 bash[19769]: cluster 2024-02-13T03:37:02.059599+0000 mon.a (mon.0) 2707 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:03.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:03 smithi161 bash[19769]: audit 2024-02-13T03:37:02.076816+0000 mon.a (mon.0) 2708 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-108"}]': finished 2024-02-13T03:37:03.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:03 smithi161 bash[19769]: cluster 2024-02-13T03:37:02.076926+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-02-13T03:37:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:03 smithi183 bash[20824]: cluster 2024-02-13T03:37:01.512394+0000 mgr.y (mgr.24473) 622 : cluster [DBG] pgmap v974: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:03 smithi183 bash[20824]: cluster 2024-02-13T03:37:02.059599+0000 mon.a (mon.0) 2707 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:03 smithi183 bash[20824]: audit 2024-02-13T03:37:02.076816+0000 mon.a (mon.0) 2708 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-108"}]': finished 2024-02-13T03:37:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:03 smithi183 bash[20824]: cluster 2024-02-13T03:37:02.076926+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-02-13T03:37:04.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:04 smithi161 bash[27879]: cluster 2024-02-13T03:37:03.076216+0000 mon.a (mon.0) 2710 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-02-13T03:37:04.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:04 smithi161 bash[27879]: cluster 2024-02-13T03:37:04.025235+0000 mon.a (mon.0) 2711 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-02-13T03:37:04.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:04 smithi161 bash[27879]: audit 2024-02-13T03:37:04.028179+0000 mon.a (mon.0) 2712 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:04.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:04 smithi161 bash[19769]: cluster 2024-02-13T03:37:03.076216+0000 mon.a (mon.0) 2710 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-02-13T03:37:04.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:04 smithi161 bash[19769]: cluster 2024-02-13T03:37:04.025235+0000 mon.a (mon.0) 2711 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-02-13T03:37:04.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:04 smithi161 bash[19769]: audit 2024-02-13T03:37:04.028179+0000 mon.a (mon.0) 2712 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:04.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:04 smithi183 bash[20824]: cluster 2024-02-13T03:37:03.076216+0000 mon.a (mon.0) 2710 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-02-13T03:37:04.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:04 smithi183 bash[20824]: cluster 2024-02-13T03:37:04.025235+0000 mon.a (mon.0) 2711 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-02-13T03:37:04.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:04 smithi183 bash[20824]: audit 2024-02-13T03:37:04.028179+0000 mon.a (mon.0) 2712 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:05.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:05 smithi161 bash[27879]: cluster 2024-02-13T03:37:03.512840+0000 mgr.y (mgr.24473) 623 : cluster [DBG] pgmap v977: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:05.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:05 smithi161 bash[27879]: audit 2024-02-13T03:37:04.793174+0000 mon.a (mon.0) 2713 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:37:05.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:05 smithi161 bash[27879]: audit 2024-02-13T03:37:05.024196+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:05.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:05 smithi161 bash[27879]: cluster 2024-02-13T03:37:05.024711+0000 mon.a (mon.0) 2715 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-02-13T03:37:05.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:05 smithi161 bash[27879]: audit 2024-02-13T03:37:05.044289+0000 mon.a (mon.0) 2716 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:05.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:05 smithi161 bash[19769]: cluster 2024-02-13T03:37:03.512840+0000 mgr.y (mgr.24473) 623 : cluster [DBG] pgmap v977: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:05.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:05 smithi161 bash[19769]: audit 2024-02-13T03:37:04.793174+0000 mon.a (mon.0) 2713 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:37:05.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:05 smithi161 bash[19769]: audit 2024-02-13T03:37:05.024196+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:05.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:05 smithi161 bash[19769]: cluster 2024-02-13T03:37:05.024711+0000 mon.a (mon.0) 2715 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-02-13T03:37:05.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:05 smithi161 bash[19769]: audit 2024-02-13T03:37:05.044289+0000 mon.a (mon.0) 2716 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:05.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:05 smithi183 bash[20824]: cluster 2024-02-13T03:37:03.512840+0000 mgr.y (mgr.24473) 623 : cluster [DBG] pgmap v977: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:05.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:05 smithi183 bash[20824]: audit 2024-02-13T03:37:04.793174+0000 mon.a (mon.0) 2713 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:37:05.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:05 smithi183 bash[20824]: audit 2024-02-13T03:37:05.024196+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:05.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:05 smithi183 bash[20824]: cluster 2024-02-13T03:37:05.024711+0000 mon.a (mon.0) 2715 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-02-13T03:37:05.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:05 smithi183 bash[20824]: audit 2024-02-13T03:37:05.044289+0000 mon.a (mon.0) 2716 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:06.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:37:06 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:37:07.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:07 smithi161 bash[27879]: cluster 2024-02-13T03:37:05.513584+0000 mgr.y (mgr.24473) 624 : cluster [DBG] pgmap v980: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:07.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:07 smithi161 bash[27879]: audit 2024-02-13T03:37:06.026388+0000 mon.a (mon.0) 2717 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:07.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:07 smithi161 bash[27879]: cluster 2024-02-13T03:37:06.026447+0000 mon.a (mon.0) 2718 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-02-13T03:37:07.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:07 smithi161 bash[27879]: audit 2024-02-13T03:37:06.027007+0000 mon.a (mon.0) 2719 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-110"}]: dispatch 2024-02-13T03:37:07.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:07 smithi161 bash[27879]: audit 2024-02-13T03:37:06.325651+0000 mgr.y (mgr.24473) 625 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:07.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:07 smithi161 bash[19769]: cluster 2024-02-13T03:37:05.513584+0000 mgr.y (mgr.24473) 624 : cluster [DBG] pgmap v980: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:07.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:07 smithi161 bash[19769]: audit 2024-02-13T03:37:06.026388+0000 mon.a (mon.0) 2717 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:07.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:07 smithi161 bash[19769]: cluster 2024-02-13T03:37:06.026447+0000 mon.a (mon.0) 2718 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-02-13T03:37:07.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:07 smithi161 bash[19769]: audit 2024-02-13T03:37:06.027007+0000 mon.a (mon.0) 2719 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-110"}]: dispatch 2024-02-13T03:37:07.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:07 smithi161 bash[19769]: audit 2024-02-13T03:37:06.325651+0000 mgr.y (mgr.24473) 625 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:07.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:07 smithi183 bash[20824]: cluster 2024-02-13T03:37:05.513584+0000 mgr.y (mgr.24473) 624 : cluster [DBG] pgmap v980: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:07 smithi183 bash[20824]: audit 2024-02-13T03:37:06.026388+0000 mon.a (mon.0) 2717 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:07 smithi183 bash[20824]: cluster 2024-02-13T03:37:06.026447+0000 mon.a (mon.0) 2718 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-02-13T03:37:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:07 smithi183 bash[20824]: audit 2024-02-13T03:37:06.027007+0000 mon.a (mon.0) 2719 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-110"}]: dispatch 2024-02-13T03:37:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:07 smithi183 bash[20824]: audit 2024-02-13T03:37:06.325651+0000 mgr.y (mgr.24473) 625 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:08.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:08 smithi161 bash[27879]: audit 2024-02-13T03:37:07.027340+0000 mon.a (mon.0) 2720 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-110"}]': finished 2024-02-13T03:37:08.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:08 smithi161 bash[27879]: cluster 2024-02-13T03:37:07.027429+0000 mon.a (mon.0) 2721 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-02-13T03:37:08.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:08 smithi161 bash[27879]: audit 2024-02-13T03:37:07.028744+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-110", "mode": "writeback"}]: dispatch 2024-02-13T03:37:08.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:08 smithi161 bash[27879]: cluster 2024-02-13T03:37:08.022519+0000 mon.a (mon.0) 2723 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:08.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:08 smithi161 bash[19769]: audit 2024-02-13T03:37:07.027340+0000 mon.a (mon.0) 2720 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-110"}]': finished 2024-02-13T03:37:08.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:08 smithi161 bash[19769]: cluster 2024-02-13T03:37:07.027429+0000 mon.a (mon.0) 2721 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-02-13T03:37:08.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:08 smithi161 bash[19769]: audit 2024-02-13T03:37:07.028744+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-110", "mode": "writeback"}]: dispatch 2024-02-13T03:37:08.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:08 smithi161 bash[19769]: cluster 2024-02-13T03:37:08.022519+0000 mon.a (mon.0) 2723 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:08.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:08 smithi183 bash[20824]: audit 2024-02-13T03:37:07.027340+0000 mon.a (mon.0) 2720 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-110"}]': finished 2024-02-13T03:37:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:08 smithi183 bash[20824]: cluster 2024-02-13T03:37:07.027429+0000 mon.a (mon.0) 2721 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-02-13T03:37:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:08 smithi183 bash[20824]: audit 2024-02-13T03:37:07.028744+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-110", "mode": "writeback"}]: dispatch 2024-02-13T03:37:08.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:08 smithi183 bash[20824]: cluster 2024-02-13T03:37:08.022519+0000 mon.a (mon.0) 2723 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:09.332 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:09 smithi183 bash[20824]: cluster 2024-02-13T03:37:07.514448+0000 mgr.y (mgr.24473) 626 : cluster [DBG] pgmap v983: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:09.333 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:09 smithi183 bash[20824]: audit 2024-02-13T03:37:08.035657+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-110", "mode": "writeback"}]': finished 2024-02-13T03:37:09.333 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:09 smithi183 bash[20824]: cluster 2024-02-13T03:37:08.035790+0000 mon.a (mon.0) 2725 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-02-13T03:37:09.333 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:09 smithi183 bash[20824]: audit 2024-02-13T03:37:08.107850+0000 mon.a (mon.0) 2726 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:09.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:09 smithi161 bash[27879]: cluster 2024-02-13T03:37:07.514448+0000 mgr.y (mgr.24473) 626 : cluster [DBG] pgmap v983: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:09.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:09 smithi161 bash[27879]: audit 2024-02-13T03:37:08.035657+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-110", "mode": "writeback"}]': finished 2024-02-13T03:37:09.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:09 smithi161 bash[27879]: cluster 2024-02-13T03:37:08.035790+0000 mon.a (mon.0) 2725 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-02-13T03:37:09.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:09 smithi161 bash[27879]: audit 2024-02-13T03:37:08.107850+0000 mon.a (mon.0) 2726 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:09.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:09 smithi161 bash[19769]: cluster 2024-02-13T03:37:07.514448+0000 mgr.y (mgr.24473) 626 : cluster [DBG] pgmap v983: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:09.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:09 smithi161 bash[19769]: audit 2024-02-13T03:37:08.035657+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-110", "mode": "writeback"}]': finished 2024-02-13T03:37:09.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:09 smithi161 bash[19769]: cluster 2024-02-13T03:37:08.035790+0000 mon.a (mon.0) 2725 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-02-13T03:37:09.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:09 smithi161 bash[19769]: audit 2024-02-13T03:37:08.107850+0000 mon.a (mon.0) 2726 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:09.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:37:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:37:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:37:10.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:10 smithi161 bash[27879]: audit 2024-02-13T03:37:09.056566+0000 mon.a (mon.0) 2727 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:10.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:10 smithi161 bash[27879]: cluster 2024-02-13T03:37:09.056899+0000 mon.a (mon.0) 2728 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-02-13T03:37:10.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:10 smithi161 bash[27879]: audit 2024-02-13T03:37:09.060576+0000 mon.a (mon.0) 2729 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-110"}]: dispatch 2024-02-13T03:37:10.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:10 smithi161 bash[19769]: audit 2024-02-13T03:37:09.056566+0000 mon.a (mon.0) 2727 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:10.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:10 smithi161 bash[19769]: cluster 2024-02-13T03:37:09.056899+0000 mon.a (mon.0) 2728 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-02-13T03:37:10.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:10 smithi161 bash[19769]: audit 2024-02-13T03:37:09.060576+0000 mon.a (mon.0) 2729 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-110"}]: dispatch 2024-02-13T03:37:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:10 smithi183 bash[20824]: audit 2024-02-13T03:37:09.056566+0000 mon.a (mon.0) 2727 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:10 smithi183 bash[20824]: cluster 2024-02-13T03:37:09.056899+0000 mon.a (mon.0) 2728 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-02-13T03:37:10.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:10 smithi183 bash[20824]: audit 2024-02-13T03:37:09.060576+0000 mon.a (mon.0) 2729 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-110"}]: dispatch 2024-02-13T03:37:11.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:37:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:37:11] "GET /metrics HTTP/1.1" 200 189373 "" "Prometheus/2.33.4" 2024-02-13T03:37:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:11 smithi161 bash[27879]: cluster 2024-02-13T03:37:09.515300+0000 mgr.y (mgr.24473) 627 : cluster [DBG] pgmap v986: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:11 smithi161 bash[27879]: cluster 2024-02-13T03:37:10.049697+0000 mon.a (mon.0) 2730 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:11 smithi161 bash[27879]: audit 2024-02-13T03:37:10.057773+0000 mon.a (mon.0) 2731 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-110"}]': finished 2024-02-13T03:37:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:11 smithi161 bash[27879]: cluster 2024-02-13T03:37:10.057937+0000 mon.a (mon.0) 2732 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-02-13T03:37:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:11 smithi161 bash[19769]: cluster 2024-02-13T03:37:09.515300+0000 mgr.y (mgr.24473) 627 : cluster [DBG] pgmap v986: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:11 smithi161 bash[19769]: cluster 2024-02-13T03:37:10.049697+0000 mon.a (mon.0) 2730 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:11 smithi161 bash[19769]: audit 2024-02-13T03:37:10.057773+0000 mon.a (mon.0) 2731 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-110"}]': finished 2024-02-13T03:37:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:11 smithi161 bash[19769]: cluster 2024-02-13T03:37:10.057937+0000 mon.a (mon.0) 2732 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-02-13T03:37:11.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:11 smithi183 bash[20824]: cluster 2024-02-13T03:37:09.515300+0000 mgr.y (mgr.24473) 627 : cluster [DBG] pgmap v986: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:11.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:11 smithi183 bash[20824]: cluster 2024-02-13T03:37:10.049697+0000 mon.a (mon.0) 2730 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:11.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:11 smithi183 bash[20824]: audit 2024-02-13T03:37:10.057773+0000 mon.a (mon.0) 2731 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-110"}]': finished 2024-02-13T03:37:11.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:11 smithi183 bash[20824]: cluster 2024-02-13T03:37:10.057937+0000 mon.a (mon.0) 2732 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-02-13T03:37:12.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:12 smithi161 bash[27879]: cluster 2024-02-13T03:37:11.078439+0000 mon.a (mon.0) 2733 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-02-13T03:37:12.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:12 smithi161 bash[19769]: cluster 2024-02-13T03:37:11.078439+0000 mon.a (mon.0) 2733 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-02-13T03:37:12.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:12 smithi183 bash[20824]: cluster 2024-02-13T03:37:11.078439+0000 mon.a (mon.0) 2733 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-02-13T03:37:13.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:13 smithi161 bash[27879]: cluster 2024-02-13T03:37:11.516133+0000 mgr.y (mgr.24473) 628 : cluster [DBG] pgmap v989: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:13.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:13 smithi161 bash[27879]: cluster 2024-02-13T03:37:12.091471+0000 mon.a (mon.0) 2734 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-02-13T03:37:13.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:13 smithi161 bash[27879]: audit 2024-02-13T03:37:12.096053+0000 mon.a (mon.0) 2735 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:13.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:13 smithi161 bash[19769]: cluster 2024-02-13T03:37:11.516133+0000 mgr.y (mgr.24473) 628 : cluster [DBG] pgmap v989: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:13.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:13 smithi161 bash[19769]: cluster 2024-02-13T03:37:12.091471+0000 mon.a (mon.0) 2734 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-02-13T03:37:13.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:13 smithi161 bash[19769]: audit 2024-02-13T03:37:12.096053+0000 mon.a (mon.0) 2735 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:13.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:13 smithi183 bash[20824]: cluster 2024-02-13T03:37:11.516133+0000 mgr.y (mgr.24473) 628 : cluster [DBG] pgmap v989: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:13.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:13 smithi183 bash[20824]: cluster 2024-02-13T03:37:12.091471+0000 mon.a (mon.0) 2734 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-02-13T03:37:13.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:13 smithi183 bash[20824]: audit 2024-02-13T03:37:12.096053+0000 mon.a (mon.0) 2735 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:14.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:14 smithi161 bash[27879]: audit 2024-02-13T03:37:13.092637+0000 mon.a (mon.0) 2736 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:14.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:14 smithi161 bash[27879]: cluster 2024-02-13T03:37:13.092909+0000 mon.a (mon.0) 2737 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-02-13T03:37:14.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:14 smithi161 bash[27879]: audit 2024-02-13T03:37:13.109207+0000 mon.a (mon.0) 2738 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:14.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:14 smithi161 bash[19769]: audit 2024-02-13T03:37:13.092637+0000 mon.a (mon.0) 2736 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:14.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:14 smithi161 bash[19769]: cluster 2024-02-13T03:37:13.092909+0000 mon.a (mon.0) 2737 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-02-13T03:37:14.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:14 smithi161 bash[19769]: audit 2024-02-13T03:37:13.109207+0000 mon.a (mon.0) 2738 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:14.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:14 smithi183 bash[20824]: audit 2024-02-13T03:37:13.092637+0000 mon.a (mon.0) 2736 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:14 smithi183 bash[20824]: cluster 2024-02-13T03:37:13.092909+0000 mon.a (mon.0) 2737 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-02-13T03:37:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:14 smithi183 bash[20824]: audit 2024-02-13T03:37:13.109207+0000 mon.a (mon.0) 2738 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:15.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:15 smithi161 bash[27879]: cluster 2024-02-13T03:37:13.516962+0000 mgr.y (mgr.24473) 629 : cluster [DBG] pgmap v992: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:15.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:15 smithi161 bash[27879]: audit 2024-02-13T03:37:14.115470+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:15.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:15 smithi161 bash[27879]: cluster 2024-02-13T03:37:14.115582+0000 mon.a (mon.0) 2740 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-02-13T03:37:15.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:15 smithi161 bash[27879]: audit 2024-02-13T03:37:14.116500+0000 mon.a (mon.0) 2741 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-112"}]: dispatch 2024-02-13T03:37:15.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:15 smithi161 bash[19769]: cluster 2024-02-13T03:37:13.516962+0000 mgr.y (mgr.24473) 629 : cluster [DBG] pgmap v992: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:15.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:15 smithi161 bash[19769]: audit 2024-02-13T03:37:14.115470+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:15.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:15 smithi161 bash[19769]: cluster 2024-02-13T03:37:14.115582+0000 mon.a (mon.0) 2740 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-02-13T03:37:15.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:15 smithi161 bash[19769]: audit 2024-02-13T03:37:14.116500+0000 mon.a (mon.0) 2741 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-112"}]: dispatch 2024-02-13T03:37:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:15 smithi183 bash[20824]: cluster 2024-02-13T03:37:13.516962+0000 mgr.y (mgr.24473) 629 : cluster [DBG] pgmap v992: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:15.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:15 smithi183 bash[20824]: audit 2024-02-13T03:37:14.115470+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:15.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:15 smithi183 bash[20824]: cluster 2024-02-13T03:37:14.115582+0000 mon.a (mon.0) 2740 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-02-13T03:37:15.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:15 smithi183 bash[20824]: audit 2024-02-13T03:37:14.116500+0000 mon.a (mon.0) 2741 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-112"}]: dispatch 2024-02-13T03:37:16.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:16 smithi161 bash[27879]: audit 2024-02-13T03:37:15.120863+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-112"}]': finished 2024-02-13T03:37:16.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:16 smithi161 bash[27879]: cluster 2024-02-13T03:37:15.121049+0000 mon.a (mon.0) 2743 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-02-13T03:37:16.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:16 smithi161 bash[27879]: audit 2024-02-13T03:37:15.122378+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-112", "mode": "writeback"}]: dispatch 2024-02-13T03:37:16.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:16 smithi161 bash[27879]: cluster 2024-02-13T03:37:16.115472+0000 mon.a (mon.0) 2745 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:16.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:16 smithi161 bash[27879]: audit 2024-02-13T03:37:16.123475+0000 mon.a (mon.0) 2746 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-112", "mode": "writeback"}]': finished 2024-02-13T03:37:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:16 smithi161 bash[27879]: cluster 2024-02-13T03:37:16.123678+0000 mon.a (mon.0) 2747 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-02-13T03:37:16.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:16 smithi161 bash[19769]: audit 2024-02-13T03:37:15.120863+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-112"}]': finished 2024-02-13T03:37:16.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:16 smithi161 bash[19769]: cluster 2024-02-13T03:37:15.121049+0000 mon.a (mon.0) 2743 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-02-13T03:37:16.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:16 smithi161 bash[19769]: audit 2024-02-13T03:37:15.122378+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-112", "mode": "writeback"}]: dispatch 2024-02-13T03:37:16.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:16 smithi161 bash[19769]: cluster 2024-02-13T03:37:16.115472+0000 mon.a (mon.0) 2745 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:16.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:16 smithi161 bash[19769]: audit 2024-02-13T03:37:16.123475+0000 mon.a (mon.0) 2746 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-112", "mode": "writeback"}]': finished 2024-02-13T03:37:16.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:16 smithi161 bash[19769]: cluster 2024-02-13T03:37:16.123678+0000 mon.a (mon.0) 2747 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-02-13T03:37:16.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:37:16 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:37:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:16 smithi183 bash[20824]: audit 2024-02-13T03:37:15.120863+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-112"}]': finished 2024-02-13T03:37:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:16 smithi183 bash[20824]: cluster 2024-02-13T03:37:15.121049+0000 mon.a (mon.0) 2743 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-02-13T03:37:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:16 smithi183 bash[20824]: audit 2024-02-13T03:37:15.122378+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-112", "mode": "writeback"}]: dispatch 2024-02-13T03:37:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:16 smithi183 bash[20824]: cluster 2024-02-13T03:37:16.115472+0000 mon.a (mon.0) 2745 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:16 smithi183 bash[20824]: audit 2024-02-13T03:37:16.123475+0000 mon.a (mon.0) 2746 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-112", "mode": "writeback"}]': finished 2024-02-13T03:37:16.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:16 smithi183 bash[20824]: cluster 2024-02-13T03:37:16.123678+0000 mon.a (mon.0) 2747 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-02-13T03:37:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:17 smithi161 bash[27879]: cluster 2024-02-13T03:37:15.517823+0000 mgr.y (mgr.24473) 630 : cluster [DBG] pgmap v995: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:17 smithi161 bash[27879]: audit 2024-02-13T03:37:16.199272+0000 mon.a (mon.0) 2748 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:17.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:17 smithi161 bash[27879]: audit 2024-02-13T03:37:16.336887+0000 mgr.y (mgr.24473) 631 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:17.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:17 smithi161 bash[19769]: cluster 2024-02-13T03:37:15.517823+0000 mgr.y (mgr.24473) 630 : cluster [DBG] pgmap v995: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:17.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:17 smithi161 bash[19769]: audit 2024-02-13T03:37:16.199272+0000 mon.a (mon.0) 2748 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:17.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:17 smithi161 bash[19769]: audit 2024-02-13T03:37:16.336887+0000 mgr.y (mgr.24473) 631 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:17 smithi183 bash[20824]: cluster 2024-02-13T03:37:15.517823+0000 mgr.y (mgr.24473) 630 : cluster [DBG] pgmap v995: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:17 smithi183 bash[20824]: audit 2024-02-13T03:37:16.199272+0000 mon.a (mon.0) 2748 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:17.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:17 smithi183 bash[20824]: audit 2024-02-13T03:37:16.336887+0000 mgr.y (mgr.24473) 631 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:18.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:18 smithi161 bash[19769]: audit 2024-02-13T03:37:17.152980+0000 mon.a (mon.0) 2749 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:18.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:18 smithi161 bash[19769]: cluster 2024-02-13T03:37:17.153083+0000 mon.a (mon.0) 2750 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-02-13T03:37:18.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:18 smithi161 bash[19769]: audit 2024-02-13T03:37:17.153878+0000 mon.a (mon.0) 2751 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-112"}]: dispatch 2024-02-13T03:37:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:18 smithi161 bash[27879]: audit 2024-02-13T03:37:17.152980+0000 mon.a (mon.0) 2749 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:18 smithi161 bash[27879]: cluster 2024-02-13T03:37:17.153083+0000 mon.a (mon.0) 2750 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-02-13T03:37:18.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:18 smithi161 bash[27879]: audit 2024-02-13T03:37:17.153878+0000 mon.a (mon.0) 2751 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-112"}]: dispatch 2024-02-13T03:37:18.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:18 smithi183 bash[20824]: audit 2024-02-13T03:37:17.152980+0000 mon.a (mon.0) 2749 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:18.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:18 smithi183 bash[20824]: cluster 2024-02-13T03:37:17.153083+0000 mon.a (mon.0) 2750 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-02-13T03:37:18.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:18 smithi183 bash[20824]: audit 2024-02-13T03:37:17.153878+0000 mon.a (mon.0) 2751 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-112"}]: dispatch 2024-02-13T03:37:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:19 smithi161 bash[27879]: cluster 2024-02-13T03:37:17.518698+0000 mgr.y (mgr.24473) 632 : cluster [DBG] pgmap v998: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:19 smithi161 bash[27879]: cluster 2024-02-13T03:37:18.149596+0000 mon.a (mon.0) 2752 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:19 smithi161 bash[27879]: audit 2024-02-13T03:37:18.161470+0000 mon.a (mon.0) 2753 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-112"}]': finished 2024-02-13T03:37:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:19 smithi161 bash[27879]: cluster 2024-02-13T03:37:18.161692+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-02-13T03:37:19.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:19 smithi161 bash[19769]: cluster 2024-02-13T03:37:17.518698+0000 mgr.y (mgr.24473) 632 : cluster [DBG] pgmap v998: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:19.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:19 smithi161 bash[19769]: cluster 2024-02-13T03:37:18.149596+0000 mon.a (mon.0) 2752 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:19.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:19 smithi161 bash[19769]: audit 2024-02-13T03:37:18.161470+0000 mon.a (mon.0) 2753 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-112"}]': finished 2024-02-13T03:37:19.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:19 smithi161 bash[19769]: cluster 2024-02-13T03:37:18.161692+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-02-13T03:37:19.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:19 smithi183 bash[20824]: cluster 2024-02-13T03:37:17.518698+0000 mgr.y (mgr.24473) 632 : cluster [DBG] pgmap v998: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:19.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:19 smithi183 bash[20824]: cluster 2024-02-13T03:37:18.149596+0000 mon.a (mon.0) 2752 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:19.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:19 smithi183 bash[20824]: audit 2024-02-13T03:37:18.161470+0000 mon.a (mon.0) 2753 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-112"}]': finished 2024-02-13T03:37:19.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:19 smithi183 bash[20824]: cluster 2024-02-13T03:37:18.161692+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-02-13T03:37:19.488 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:37:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:37:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:37:20.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:20 smithi183 bash[20824]: cluster 2024-02-13T03:37:19.184387+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-02-13T03:37:20.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:20 smithi161 bash[27879]: cluster 2024-02-13T03:37:19.184387+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-02-13T03:37:20.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:20 smithi161 bash[19769]: cluster 2024-02-13T03:37:19.184387+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-02-13T03:37:21.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:37:21] "GET /metrics HTTP/1.1" 200 186944 "" "Prometheus/2.33.4" 2024-02-13T03:37:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[27879]: cluster 2024-02-13T03:37:19.519461+0000 mgr.y (mgr.24473) 633 : cluster [DBG] pgmap v1001: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[27879]: cluster 2024-02-13T03:37:20.196718+0000 mon.a (mon.0) 2756 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-02-13T03:37:21.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[27879]: audit 2024-02-13T03:37:20.202425+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:21.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[27879]: audit 2024-02-13T03:37:20.330203+0000 mon.c (mon.2) 567 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:37:21.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[27879]: audit 2024-02-13T03:37:20.629834+0000 mon.c (mon.2) 568 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:37:21.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[27879]: audit 2024-02-13T03:37:20.631331+0000 mon.c (mon.2) 569 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:37:21.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[27879]: audit 2024-02-13T03:37:20.640595+0000 mon.a (mon.0) 2758 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:37:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[19769]: cluster 2024-02-13T03:37:19.519461+0000 mgr.y (mgr.24473) 633 : cluster [DBG] pgmap v1001: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[19769]: cluster 2024-02-13T03:37:20.196718+0000 mon.a (mon.0) 2756 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-02-13T03:37:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[19769]: audit 2024-02-13T03:37:20.202425+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[19769]: audit 2024-02-13T03:37:20.330203+0000 mon.c (mon.2) 567 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:37:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[19769]: audit 2024-02-13T03:37:20.629834+0000 mon.c (mon.2) 568 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:37:21.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[19769]: audit 2024-02-13T03:37:20.631331+0000 mon.c (mon.2) 569 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:37:21.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:21 smithi161 bash[19769]: audit 2024-02-13T03:37:20.640595+0000 mon.a (mon.0) 2758 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:37:21.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:21 smithi183 bash[20824]: cluster 2024-02-13T03:37:19.519461+0000 mgr.y (mgr.24473) 633 : cluster [DBG] pgmap v1001: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:21.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:21 smithi183 bash[20824]: cluster 2024-02-13T03:37:20.196718+0000 mon.a (mon.0) 2756 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-02-13T03:37:21.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:21 smithi183 bash[20824]: audit 2024-02-13T03:37:20.202425+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:21.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:21 smithi183 bash[20824]: audit 2024-02-13T03:37:20.330203+0000 mon.c (mon.2) 567 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:37:21.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:21 smithi183 bash[20824]: audit 2024-02-13T03:37:20.629834+0000 mon.c (mon.2) 568 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:37:21.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:21 smithi183 bash[20824]: audit 2024-02-13T03:37:20.631331+0000 mon.c (mon.2) 569 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:37:21.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:21 smithi183 bash[20824]: audit 2024-02-13T03:37:20.640595+0000 mon.a (mon.0) 2758 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:37:22.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:22 smithi183 bash[20824]: audit 2024-02-13T03:37:21.202815+0000 mon.a (mon.0) 2759 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:22.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:22 smithi183 bash[20824]: cluster 2024-02-13T03:37:21.203816+0000 mon.a (mon.0) 2760 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-02-13T03:37:22.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:22 smithi183 bash[20824]: cluster 2024-02-13T03:37:22.200549+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-02-13T03:37:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:22 smithi161 bash[27879]: audit 2024-02-13T03:37:21.202815+0000 mon.a (mon.0) 2759 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:22 smithi161 bash[27879]: cluster 2024-02-13T03:37:21.203816+0000 mon.a (mon.0) 2760 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-02-13T03:37:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:22 smithi161 bash[27879]: cluster 2024-02-13T03:37:22.200549+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-02-13T03:37:22.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:22 smithi161 bash[19769]: audit 2024-02-13T03:37:21.202815+0000 mon.a (mon.0) 2759 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:22.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:22 smithi161 bash[19769]: cluster 2024-02-13T03:37:21.203816+0000 mon.a (mon.0) 2760 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-02-13T03:37:22.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:22 smithi161 bash[19769]: cluster 2024-02-13T03:37:22.200549+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-02-13T03:37:23.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:23 smithi183 bash[20824]: cluster 2024-02-13T03:37:21.520261+0000 mgr.y (mgr.24473) 634 : cluster [DBG] pgmap v1004: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:23 smithi183 bash[20824]: audit 2024-02-13T03:37:22.223291+0000 mon.a (mon.0) 2762 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:23 smithi161 bash[27879]: cluster 2024-02-13T03:37:21.520261+0000 mgr.y (mgr.24473) 634 : cluster [DBG] pgmap v1004: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:23 smithi161 bash[27879]: audit 2024-02-13T03:37:22.223291+0000 mon.a (mon.0) 2762 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:23.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:23 smithi161 bash[19769]: cluster 2024-02-13T03:37:21.520261+0000 mgr.y (mgr.24473) 634 : cluster [DBG] pgmap v1004: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:23.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:23 smithi161 bash[19769]: audit 2024-02-13T03:37:22.223291+0000 mon.a (mon.0) 2762 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:24 smithi161 bash[27879]: audit 2024-02-13T03:37:23.236132+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:24 smithi161 bash[27879]: cluster 2024-02-13T03:37:23.236359+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-02-13T03:37:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:24 smithi161 bash[27879]: audit 2024-02-13T03:37:23.237724+0000 mon.a (mon.0) 2765 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-114"}]: dispatch 2024-02-13T03:37:24.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:24 smithi161 bash[19769]: audit 2024-02-13T03:37:23.236132+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:24 smithi161 bash[19769]: cluster 2024-02-13T03:37:23.236359+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-02-13T03:37:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:24 smithi161 bash[19769]: audit 2024-02-13T03:37:23.237724+0000 mon.a (mon.0) 2765 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-114"}]: dispatch 2024-02-13T03:37:24.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:24 smithi183 bash[20824]: audit 2024-02-13T03:37:23.236132+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:24.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:24 smithi183 bash[20824]: cluster 2024-02-13T03:37:23.236359+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-02-13T03:37:24.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:24 smithi183 bash[20824]: audit 2024-02-13T03:37:23.237724+0000 mon.a (mon.0) 2765 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-114"}]: dispatch 2024-02-13T03:37:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:25 smithi161 bash[27879]: cluster 2024-02-13T03:37:23.521155+0000 mgr.y (mgr.24473) 635 : cluster [DBG] pgmap v1007: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:25 smithi161 bash[27879]: audit 2024-02-13T03:37:24.246546+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-114"}]': finished 2024-02-13T03:37:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:25 smithi161 bash[27879]: cluster 2024-02-13T03:37:24.246917+0000 mon.a (mon.0) 2767 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-02-13T03:37:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:25 smithi161 bash[27879]: audit 2024-02-13T03:37:24.248470+0000 mon.a (mon.0) 2768 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-114", "mode": "writeback"}]: dispatch 2024-02-13T03:37:25.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:25 smithi161 bash[19769]: cluster 2024-02-13T03:37:23.521155+0000 mgr.y (mgr.24473) 635 : cluster [DBG] pgmap v1007: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:25.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:25 smithi161 bash[19769]: audit 2024-02-13T03:37:24.246546+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-114"}]': finished 2024-02-13T03:37:25.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:25 smithi161 bash[19769]: cluster 2024-02-13T03:37:24.246917+0000 mon.a (mon.0) 2767 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-02-13T03:37:25.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:25 smithi161 bash[19769]: audit 2024-02-13T03:37:24.248470+0000 mon.a (mon.0) 2768 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-114", "mode": "writeback"}]: dispatch 2024-02-13T03:37:25.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:25 smithi183 bash[20824]: cluster 2024-02-13T03:37:23.521155+0000 mgr.y (mgr.24473) 635 : cluster [DBG] pgmap v1007: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:25.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:25 smithi183 bash[20824]: audit 2024-02-13T03:37:24.246546+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-114"}]': finished 2024-02-13T03:37:25.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:25 smithi183 bash[20824]: cluster 2024-02-13T03:37:24.246917+0000 mon.a (mon.0) 2767 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-02-13T03:37:25.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:25 smithi183 bash[20824]: audit 2024-02-13T03:37:24.248470+0000 mon.a (mon.0) 2768 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-114", "mode": "writeback"}]: dispatch 2024-02-13T03:37:26.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:26 smithi161 bash[27879]: cluster 2024-02-13T03:37:25.241113+0000 mon.a (mon.0) 2769 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:26.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:26 smithi161 bash[27879]: audit 2024-02-13T03:37:25.256114+0000 mon.a (mon.0) 2770 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-114", "mode": "writeback"}]': finished 2024-02-13T03:37:26.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:26 smithi161 bash[27879]: cluster 2024-02-13T03:37:25.256271+0000 mon.a (mon.0) 2771 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-02-13T03:37:26.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:26 smithi161 bash[19769]: cluster 2024-02-13T03:37:25.241113+0000 mon.a (mon.0) 2769 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:26.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:26 smithi161 bash[19769]: audit 2024-02-13T03:37:25.256114+0000 mon.a (mon.0) 2770 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-114", "mode": "writeback"}]': finished 2024-02-13T03:37:26.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:26 smithi161 bash[19769]: cluster 2024-02-13T03:37:25.256271+0000 mon.a (mon.0) 2771 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-02-13T03:37:26.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:37:26 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:37:26.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:26 smithi183 bash[20824]: cluster 2024-02-13T03:37:25.241113+0000 mon.a (mon.0) 2769 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:26.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:26 smithi183 bash[20824]: audit 2024-02-13T03:37:25.256114+0000 mon.a (mon.0) 2770 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-114", "mode": "writeback"}]': finished 2024-02-13T03:37:26.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:26 smithi183 bash[20824]: cluster 2024-02-13T03:37:25.256271+0000 mon.a (mon.0) 2771 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-02-13T03:37:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:27 smithi161 bash[27879]: cluster 2024-02-13T03:37:25.522032+0000 mgr.y (mgr.24473) 636 : cluster [DBG] pgmap v1010: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:27 smithi161 bash[27879]: cluster 2024-02-13T03:37:26.267222+0000 mon.a (mon.0) 2772 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-02-13T03:37:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:27 smithi161 bash[27879]: audit 2024-02-13T03:37:26.296675+0000 mon.a (mon.0) 2773 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:27 smithi161 bash[27879]: audit 2024-02-13T03:37:26.348215+0000 mgr.y (mgr.24473) 637 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:27.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:27 smithi161 bash[19769]: cluster 2024-02-13T03:37:25.522032+0000 mgr.y (mgr.24473) 636 : cluster [DBG] pgmap v1010: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:27.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:27 smithi161 bash[19769]: cluster 2024-02-13T03:37:26.267222+0000 mon.a (mon.0) 2772 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-02-13T03:37:27.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:27 smithi161 bash[19769]: audit 2024-02-13T03:37:26.296675+0000 mon.a (mon.0) 2773 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:27.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:27 smithi161 bash[19769]: audit 2024-02-13T03:37:26.348215+0000 mgr.y (mgr.24473) 637 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:27.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:27 smithi183 bash[20824]: cluster 2024-02-13T03:37:25.522032+0000 mgr.y (mgr.24473) 636 : cluster [DBG] pgmap v1010: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:27.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:27 smithi183 bash[20824]: cluster 2024-02-13T03:37:26.267222+0000 mon.a (mon.0) 2772 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-02-13T03:37:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:27 smithi183 bash[20824]: audit 2024-02-13T03:37:26.296675+0000 mon.a (mon.0) 2773 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:27 smithi183 bash[20824]: audit 2024-02-13T03:37:26.348215+0000 mgr.y (mgr.24473) 637 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:28 smithi161 bash[27879]: audit 2024-02-13T03:37:27.272182+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:28 smithi161 bash[27879]: cluster 2024-02-13T03:37:27.272358+0000 mon.a (mon.0) 2775 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-02-13T03:37:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:28 smithi161 bash[27879]: audit 2024-02-13T03:37:27.274113+0000 mon.a (mon.0) 2776 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-114"}]: dispatch 2024-02-13T03:37:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:28 smithi161 bash[27879]: cluster 2024-02-13T03:37:28.265942+0000 mon.a (mon.0) 2777 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:28.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:28 smithi161 bash[19769]: audit 2024-02-13T03:37:27.272182+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:28.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:28 smithi161 bash[19769]: cluster 2024-02-13T03:37:27.272358+0000 mon.a (mon.0) 2775 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-02-13T03:37:28.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:28 smithi161 bash[19769]: audit 2024-02-13T03:37:27.274113+0000 mon.a (mon.0) 2776 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-114"}]: dispatch 2024-02-13T03:37:28.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:28 smithi161 bash[19769]: cluster 2024-02-13T03:37:28.265942+0000 mon.a (mon.0) 2777 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:28.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:28 smithi183 bash[20824]: audit 2024-02-13T03:37:27.272182+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:28 smithi183 bash[20824]: cluster 2024-02-13T03:37:27.272358+0000 mon.a (mon.0) 2775 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-02-13T03:37:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:28 smithi183 bash[20824]: audit 2024-02-13T03:37:27.274113+0000 mon.a (mon.0) 2776 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-114"}]: dispatch 2024-02-13T03:37:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:28 smithi183 bash[20824]: cluster 2024-02-13T03:37:28.265942+0000 mon.a (mon.0) 2777 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:29.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:29 smithi161 bash[27879]: cluster 2024-02-13T03:37:27.522904+0000 mgr.y (mgr.24473) 638 : cluster [DBG] pgmap v1013: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-13T03:37:29.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:29 smithi161 bash[27879]: audit 2024-02-13T03:37:28.275367+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-114"}]': finished 2024-02-13T03:37:29.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:29 smithi161 bash[27879]: cluster 2024-02-13T03:37:28.275546+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-02-13T03:37:29.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:29 smithi161 bash[19769]: cluster 2024-02-13T03:37:27.522904+0000 mgr.y (mgr.24473) 638 : cluster [DBG] pgmap v1013: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-13T03:37:29.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:29 smithi161 bash[19769]: audit 2024-02-13T03:37:28.275367+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-114"}]': finished 2024-02-13T03:37:29.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:29 smithi161 bash[19769]: cluster 2024-02-13T03:37:28.275546+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-02-13T03:37:29.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:37:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:37:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:37:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:29 smithi183 bash[20824]: cluster 2024-02-13T03:37:27.522904+0000 mgr.y (mgr.24473) 638 : cluster [DBG] pgmap v1013: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-13T03:37:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:29 smithi183 bash[20824]: audit 2024-02-13T03:37:28.275367+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-114"}]': finished 2024-02-13T03:37:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:29 smithi183 bash[20824]: cluster 2024-02-13T03:37:28.275546+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-02-13T03:37:30.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:30 smithi161 bash[27879]: cluster 2024-02-13T03:37:29.302605+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-02-13T03:37:30.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:30 smithi161 bash[19769]: cluster 2024-02-13T03:37:29.302605+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-02-13T03:37:30.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:30 smithi183 bash[20824]: cluster 2024-02-13T03:37:29.302605+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-02-13T03:37:31.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:37:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:37:31] "GET /metrics HTTP/1.1" 200 186944 "" "Prometheus/2.33.4" 2024-02-13T03:37:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:31 smithi161 bash[27879]: cluster 2024-02-13T03:37:29.523673+0000 mgr.y (mgr.24473) 639 : cluster [DBG] pgmap v1016: 172 pgs: 172 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-13T03:37:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:31 smithi161 bash[27879]: cluster 2024-02-13T03:37:30.308663+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-02-13T03:37:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:31 smithi161 bash[27879]: audit 2024-02-13T03:37:30.311410+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:31 smithi161 bash[19769]: cluster 2024-02-13T03:37:29.523673+0000 mgr.y (mgr.24473) 639 : cluster [DBG] pgmap v1016: 172 pgs: 172 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-13T03:37:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:31 smithi161 bash[19769]: cluster 2024-02-13T03:37:30.308663+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-02-13T03:37:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:31 smithi161 bash[19769]: audit 2024-02-13T03:37:30.311410+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:31.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:31 smithi183 bash[20824]: cluster 2024-02-13T03:37:29.523673+0000 mgr.y (mgr.24473) 639 : cluster [DBG] pgmap v1016: 172 pgs: 172 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-13T03:37:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:31 smithi183 bash[20824]: cluster 2024-02-13T03:37:30.308663+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-02-13T03:37:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:31 smithi183 bash[20824]: audit 2024-02-13T03:37:30.311410+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:32.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:32 smithi161 bash[27879]: audit 2024-02-13T03:37:31.314100+0000 mon.a (mon.0) 2783 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:32.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:32 smithi161 bash[27879]: cluster 2024-02-13T03:37:31.314447+0000 mon.a (mon.0) 2784 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-02-13T03:37:32.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:32 smithi161 bash[27879]: audit 2024-02-13T03:37:31.317948+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:32.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:32 smithi161 bash[27879]: audit 2024-02-13T03:37:32.317706+0000 mon.a (mon.0) 2786 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:32.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:32 smithi161 bash[27879]: cluster 2024-02-13T03:37:32.317924+0000 mon.a (mon.0) 2787 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-02-13T03:37:32.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:32 smithi161 bash[27879]: audit 2024-02-13T03:37:32.320642+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-116"}]: dispatch 2024-02-13T03:37:32.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:32 smithi161 bash[19769]: audit 2024-02-13T03:37:31.314100+0000 mon.a (mon.0) 2783 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:32.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:32 smithi161 bash[19769]: cluster 2024-02-13T03:37:31.314447+0000 mon.a (mon.0) 2784 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-02-13T03:37:32.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:32 smithi161 bash[19769]: audit 2024-02-13T03:37:31.317948+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:32.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:32 smithi161 bash[19769]: audit 2024-02-13T03:37:32.317706+0000 mon.a (mon.0) 2786 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:32.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:32 smithi161 bash[19769]: cluster 2024-02-13T03:37:32.317924+0000 mon.a (mon.0) 2787 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-02-13T03:37:32.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:32 smithi161 bash[19769]: audit 2024-02-13T03:37:32.320642+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-116"}]: dispatch 2024-02-13T03:37:32.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:32 smithi183 bash[20824]: audit 2024-02-13T03:37:31.314100+0000 mon.a (mon.0) 2783 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:32 smithi183 bash[20824]: cluster 2024-02-13T03:37:31.314447+0000 mon.a (mon.0) 2784 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-02-13T03:37:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:32 smithi183 bash[20824]: audit 2024-02-13T03:37:31.317948+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:32 smithi183 bash[20824]: audit 2024-02-13T03:37:32.317706+0000 mon.a (mon.0) 2786 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:32 smithi183 bash[20824]: cluster 2024-02-13T03:37:32.317924+0000 mon.a (mon.0) 2787 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-02-13T03:37:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:32 smithi183 bash[20824]: audit 2024-02-13T03:37:32.320642+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-116"}]: dispatch 2024-02-13T03:37:33.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:33 smithi161 bash[27879]: cluster 2024-02-13T03:37:31.524507+0000 mgr.y (mgr.24473) 640 : cluster [DBG] pgmap v1019: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:33.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:33 smithi161 bash[27879]: audit 2024-02-13T03:37:33.318700+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-116"}]': finished 2024-02-13T03:37:33.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:33 smithi161 bash[27879]: cluster 2024-02-13T03:37:33.318846+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-02-13T03:37:33.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:33 smithi161 bash[27879]: audit 2024-02-13T03:37:33.319475+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-116", "mode": "writeback"}]: dispatch 2024-02-13T03:37:33.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:33 smithi161 bash[19769]: cluster 2024-02-13T03:37:31.524507+0000 mgr.y (mgr.24473) 640 : cluster [DBG] pgmap v1019: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:33.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:33 smithi161 bash[19769]: audit 2024-02-13T03:37:33.318700+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-116"}]': finished 2024-02-13T03:37:33.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:33 smithi161 bash[19769]: cluster 2024-02-13T03:37:33.318846+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-02-13T03:37:33.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:33 smithi161 bash[19769]: audit 2024-02-13T03:37:33.319475+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-116", "mode": "writeback"}]: dispatch 2024-02-13T03:37:33.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:33 smithi183 bash[20824]: cluster 2024-02-13T03:37:31.524507+0000 mgr.y (mgr.24473) 640 : cluster [DBG] pgmap v1019: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:33.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:33 smithi183 bash[20824]: audit 2024-02-13T03:37:33.318700+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-116"}]': finished 2024-02-13T03:37:33.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:33 smithi183 bash[20824]: cluster 2024-02-13T03:37:33.318846+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-02-13T03:37:33.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:33 smithi183 bash[20824]: audit 2024-02-13T03:37:33.319475+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-116", "mode": "writeback"}]: dispatch 2024-02-13T03:37:34.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:34 smithi161 bash[27879]: cluster 2024-02-13T03:37:34.023407+0000 mon.a (mon.0) 2792 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:34.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:34 smithi161 bash[27879]: audit 2024-02-13T03:37:34.033443+0000 mon.a (mon.0) 2793 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-116", "mode": "writeback"}]': finished 2024-02-13T03:37:34.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:34 smithi161 bash[27879]: cluster 2024-02-13T03:37:34.033672+0000 mon.a (mon.0) 2794 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-02-13T03:37:34.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:34 smithi161 bash[27879]: audit 2024-02-13T03:37:34.109229+0000 mon.a (mon.0) 2795 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:34.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:34 smithi161 bash[19769]: cluster 2024-02-13T03:37:34.023407+0000 mon.a (mon.0) 2792 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:34.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:34 smithi161 bash[19769]: audit 2024-02-13T03:37:34.033443+0000 mon.a (mon.0) 2793 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-116", "mode": "writeback"}]': finished 2024-02-13T03:37:34.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:34 smithi161 bash[19769]: cluster 2024-02-13T03:37:34.033672+0000 mon.a (mon.0) 2794 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-02-13T03:37:34.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:34 smithi161 bash[19769]: audit 2024-02-13T03:37:34.109229+0000 mon.a (mon.0) 2795 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:34.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:34 smithi183 bash[20824]: cluster 2024-02-13T03:37:34.023407+0000 mon.a (mon.0) 2792 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:34.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:34 smithi183 bash[20824]: audit 2024-02-13T03:37:34.033443+0000 mon.a (mon.0) 2793 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-116", "mode": "writeback"}]': finished 2024-02-13T03:37:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:34 smithi183 bash[20824]: cluster 2024-02-13T03:37:34.033672+0000 mon.a (mon.0) 2794 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-02-13T03:37:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:34 smithi183 bash[20824]: audit 2024-02-13T03:37:34.109229+0000 mon.a (mon.0) 2795 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:35.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:35 smithi161 bash[27879]: cluster 2024-02-13T03:37:33.525352+0000 mgr.y (mgr.24473) 641 : cluster [DBG] pgmap v1022: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:37:35.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:35 smithi161 bash[27879]: audit 2024-02-13T03:37:34.794318+0000 mon.a (mon.0) 2796 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:37:35.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:35 smithi161 bash[27879]: audit 2024-02-13T03:37:35.049440+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:35.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:35 smithi161 bash[27879]: cluster 2024-02-13T03:37:35.049621+0000 mon.a (mon.0) 2798 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-02-13T03:37:35.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:35 smithi161 bash[27879]: audit 2024-02-13T03:37:35.050321+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-116"}]: dispatch 2024-02-13T03:37:35.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:35 smithi161 bash[19769]: cluster 2024-02-13T03:37:33.525352+0000 mgr.y (mgr.24473) 641 : cluster [DBG] pgmap v1022: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:37:35.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:35 smithi161 bash[19769]: audit 2024-02-13T03:37:34.794318+0000 mon.a (mon.0) 2796 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:37:35.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:35 smithi161 bash[19769]: audit 2024-02-13T03:37:35.049440+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:35.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:35 smithi161 bash[19769]: cluster 2024-02-13T03:37:35.049621+0000 mon.a (mon.0) 2798 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-02-13T03:37:35.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:35 smithi161 bash[19769]: audit 2024-02-13T03:37:35.050321+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-116"}]: dispatch 2024-02-13T03:37:35.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:35 smithi183 bash[20824]: cluster 2024-02-13T03:37:33.525352+0000 mgr.y (mgr.24473) 641 : cluster [DBG] pgmap v1022: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:37:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:35 smithi183 bash[20824]: audit 2024-02-13T03:37:34.794318+0000 mon.a (mon.0) 2796 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:37:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:35 smithi183 bash[20824]: audit 2024-02-13T03:37:35.049440+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:35 smithi183 bash[20824]: cluster 2024-02-13T03:37:35.049621+0000 mon.a (mon.0) 2798 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-02-13T03:37:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:35 smithi183 bash[20824]: audit 2024-02-13T03:37:35.050321+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-116"}]: dispatch 2024-02-13T03:37:36.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:36 smithi161 bash[27879]: cluster 2024-02-13T03:37:36.041831+0000 mon.a (mon.0) 2800 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:36.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:36 smithi161 bash[27879]: audit 2024-02-13T03:37:36.050507+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-116"}]': finished 2024-02-13T03:37:36.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:36 smithi161 bash[27879]: cluster 2024-02-13T03:37:36.050914+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-02-13T03:37:36.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:36 smithi161 bash[19769]: cluster 2024-02-13T03:37:36.041831+0000 mon.a (mon.0) 2800 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:36.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:36 smithi161 bash[19769]: audit 2024-02-13T03:37:36.050507+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-116"}]': finished 2024-02-13T03:37:36.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:36 smithi161 bash[19769]: cluster 2024-02-13T03:37:36.050914+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-02-13T03:37:36.737 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:37:36 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:37:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:36 smithi183 bash[20824]: cluster 2024-02-13T03:37:36.041831+0000 mon.a (mon.0) 2800 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:36 smithi183 bash[20824]: audit 2024-02-13T03:37:36.050507+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-116"}]': finished 2024-02-13T03:37:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:36 smithi183 bash[20824]: cluster 2024-02-13T03:37:36.050914+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-02-13T03:37:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[27879]: cluster 2024-02-13T03:37:35.526194+0000 mgr.y (mgr.24473) 642 : cluster [DBG] pgmap v1025: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[27879]: audit 2024-02-13T03:37:36.354524+0000 mgr.y (mgr.24473) 643 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[27879]: audit 2024-02-13T03:37:36.815929+0000 mon.c (mon.2) 570 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1d", "id": [5, 0]}]: dispatch 2024-02-13T03:37:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[27879]: audit 2024-02-13T03:37:36.816608+0000 mon.c (mon.2) 571 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]: dispatch 2024-02-13T03:37:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[27879]: audit 2024-02-13T03:37:36.816905+0000 mon.a (mon.0) 2803 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1d", "id": [5, 0]}]: dispatch 2024-02-13T03:37:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[27879]: audit 2024-02-13T03:37:36.817876+0000 mon.a (mon.0) 2804 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]: dispatch 2024-02-13T03:37:37.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[27879]: audit 2024-02-13T03:37:37.055609+0000 mon.a (mon.0) 2805 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1d", "id": [5, 0]}]': finished 2024-02-13T03:37:37.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[27879]: audit 2024-02-13T03:37:37.055839+0000 mon.a (mon.0) 2806 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]': finished 2024-02-13T03:37:37.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[27879]: cluster 2024-02-13T03:37:37.056036+0000 mon.a (mon.0) 2807 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-02-13T03:37:37.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[19769]: cluster 2024-02-13T03:37:35.526194+0000 mgr.y (mgr.24473) 642 : cluster [DBG] pgmap v1025: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:37.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[19769]: audit 2024-02-13T03:37:36.354524+0000 mgr.y (mgr.24473) 643 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:37.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[19769]: audit 2024-02-13T03:37:36.815929+0000 mon.c (mon.2) 570 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1d", "id": [5, 0]}]: dispatch 2024-02-13T03:37:37.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[19769]: audit 2024-02-13T03:37:36.816608+0000 mon.c (mon.2) 571 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]: dispatch 2024-02-13T03:37:37.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[19769]: audit 2024-02-13T03:37:36.816905+0000 mon.a (mon.0) 2803 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1d", "id": [5, 0]}]: dispatch 2024-02-13T03:37:37.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[19769]: audit 2024-02-13T03:37:36.817876+0000 mon.a (mon.0) 2804 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]: dispatch 2024-02-13T03:37:37.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[19769]: audit 2024-02-13T03:37:37.055609+0000 mon.a (mon.0) 2805 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1d", "id": [5, 0]}]': finished 2024-02-13T03:37:37.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[19769]: audit 2024-02-13T03:37:37.055839+0000 mon.a (mon.0) 2806 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]': finished 2024-02-13T03:37:37.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:37 smithi161 bash[19769]: cluster 2024-02-13T03:37:37.056036+0000 mon.a (mon.0) 2807 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-02-13T03:37:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:37 smithi183 bash[20824]: cluster 2024-02-13T03:37:35.526194+0000 mgr.y (mgr.24473) 642 : cluster [DBG] pgmap v1025: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:37 smithi183 bash[20824]: audit 2024-02-13T03:37:36.354524+0000 mgr.y (mgr.24473) 643 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:37 smithi183 bash[20824]: audit 2024-02-13T03:37:36.815929+0000 mon.c (mon.2) 570 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1d", "id": [5, 0]}]: dispatch 2024-02-13T03:37:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:37 smithi183 bash[20824]: audit 2024-02-13T03:37:36.816608+0000 mon.c (mon.2) 571 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]: dispatch 2024-02-13T03:37:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:37 smithi183 bash[20824]: audit 2024-02-13T03:37:36.816905+0000 mon.a (mon.0) 2803 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1d", "id": [5, 0]}]: dispatch 2024-02-13T03:37:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:37 smithi183 bash[20824]: audit 2024-02-13T03:37:36.817876+0000 mon.a (mon.0) 2804 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]: dispatch 2024-02-13T03:37:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:37 smithi183 bash[20824]: audit 2024-02-13T03:37:37.055609+0000 mon.a (mon.0) 2805 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1d", "id": [5, 0]}]': finished 2024-02-13T03:37:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:37 smithi183 bash[20824]: audit 2024-02-13T03:37:37.055839+0000 mon.a (mon.0) 2806 : audit [INF] from='mgr.24473 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]': finished 2024-02-13T03:37:37.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:37 smithi183 bash[20824]: cluster 2024-02-13T03:37:37.056036+0000 mon.a (mon.0) 2807 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-02-13T03:37:39.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:39 smithi183 bash[20824]: cluster 2024-02-13T03:37:37.527033+0000 mgr.y (mgr.24473) 644 : cluster [DBG] pgmap v1028: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:39.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:39 smithi183 bash[20824]: cluster 2024-02-13T03:37:38.060810+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-02-13T03:37:39.331 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:39 smithi183 bash[20824]: audit 2024-02-13T03:37:38.063600+0000 mon.a (mon.0) 2809 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:39.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:39 smithi161 bash[27879]: cluster 2024-02-13T03:37:37.527033+0000 mgr.y (mgr.24473) 644 : cluster [DBG] pgmap v1028: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:39.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:39 smithi161 bash[27879]: cluster 2024-02-13T03:37:38.060810+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-02-13T03:37:39.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:39 smithi161 bash[27879]: audit 2024-02-13T03:37:38.063600+0000 mon.a (mon.0) 2809 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:39 smithi161 bash[19769]: cluster 2024-02-13T03:37:37.527033+0000 mgr.y (mgr.24473) 644 : cluster [DBG] pgmap v1028: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:39 smithi161 bash[19769]: cluster 2024-02-13T03:37:38.060810+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-02-13T03:37:39.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:39 smithi161 bash[19769]: audit 2024-02-13T03:37:38.063600+0000 mon.a (mon.0) 2809 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:39.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:37:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:37:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:37:40.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:40 smithi161 bash[27879]: audit 2024-02-13T03:37:39.063022+0000 mon.a (mon.0) 2810 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:40.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:40 smithi161 bash[27879]: cluster 2024-02-13T03:37:39.063581+0000 mon.a (mon.0) 2811 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-02-13T03:37:40.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:40 smithi161 bash[27879]: audit 2024-02-13T03:37:39.066872+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:40.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:40 smithi161 bash[27879]: audit 2024-02-13T03:37:40.064451+0000 mon.a (mon.0) 2813 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:40.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:40 smithi161 bash[27879]: cluster 2024-02-13T03:37:40.064625+0000 mon.a (mon.0) 2814 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-02-13T03:37:40.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:40 smithi161 bash[27879]: audit 2024-02-13T03:37:40.067728+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-118"}]: dispatch 2024-02-13T03:37:40.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:40 smithi161 bash[19769]: audit 2024-02-13T03:37:39.063022+0000 mon.a (mon.0) 2810 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:40.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:40 smithi161 bash[19769]: cluster 2024-02-13T03:37:39.063581+0000 mon.a (mon.0) 2811 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-02-13T03:37:40.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:40 smithi161 bash[19769]: audit 2024-02-13T03:37:39.066872+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:40.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:40 smithi161 bash[19769]: audit 2024-02-13T03:37:40.064451+0000 mon.a (mon.0) 2813 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:40.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:40 smithi161 bash[19769]: cluster 2024-02-13T03:37:40.064625+0000 mon.a (mon.0) 2814 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-02-13T03:37:40.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:40 smithi161 bash[19769]: audit 2024-02-13T03:37:40.067728+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-118"}]: dispatch 2024-02-13T03:37:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:40 smithi183 bash[20824]: audit 2024-02-13T03:37:39.063022+0000 mon.a (mon.0) 2810 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:40 smithi183 bash[20824]: cluster 2024-02-13T03:37:39.063581+0000 mon.a (mon.0) 2811 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-02-13T03:37:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:40 smithi183 bash[20824]: audit 2024-02-13T03:37:39.066872+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:40 smithi183 bash[20824]: audit 2024-02-13T03:37:40.064451+0000 mon.a (mon.0) 2813 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:40 smithi183 bash[20824]: cluster 2024-02-13T03:37:40.064625+0000 mon.a (mon.0) 2814 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-02-13T03:37:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:40 smithi183 bash[20824]: audit 2024-02-13T03:37:40.067728+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-118"}]: dispatch 2024-02-13T03:37:41.236 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:37:40 smithi183 bash[46554]: t=2024-02-13T03:37:40+0000 lvl=info msg="Database locked, sleeping then retrying" logger=sqlstore error="database is locked" retry=0 2024-02-13T03:37:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:41 smithi183 bash[20824]: cluster 2024-02-13T03:37:39.527874+0000 mgr.y (mgr.24473) 645 : cluster [DBG] pgmap v1031: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:41 smithi183 bash[20824]: audit 2024-02-13T03:37:41.067306+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-118"}]': finished 2024-02-13T03:37:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:41 smithi183 bash[20824]: cluster 2024-02-13T03:37:41.067812+0000 mon.a (mon.0) 2817 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-02-13T03:37:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:41 smithi183 bash[20824]: audit 2024-02-13T03:37:41.070597+0000 mon.a (mon.0) 2818 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-118", "mode": "writeback"}]: dispatch 2024-02-13T03:37:41.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:37:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:37:41] "GET /metrics HTTP/1.1" 200 189498 "" "Prometheus/2.33.4" 2024-02-13T03:37:41.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:41 smithi161 bash[27879]: cluster 2024-02-13T03:37:39.527874+0000 mgr.y (mgr.24473) 645 : cluster [DBG] pgmap v1031: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:41.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:41 smithi161 bash[27879]: audit 2024-02-13T03:37:41.067306+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-118"}]': finished 2024-02-13T03:37:41.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:41 smithi161 bash[27879]: cluster 2024-02-13T03:37:41.067812+0000 mon.a (mon.0) 2817 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-02-13T03:37:41.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:41 smithi161 bash[27879]: audit 2024-02-13T03:37:41.070597+0000 mon.a (mon.0) 2818 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-118", "mode": "writeback"}]: dispatch 2024-02-13T03:37:41.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:41 smithi161 bash[19769]: cluster 2024-02-13T03:37:39.527874+0000 mgr.y (mgr.24473) 645 : cluster [DBG] pgmap v1031: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:37:41.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:41 smithi161 bash[19769]: audit 2024-02-13T03:37:41.067306+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-118"}]': finished 2024-02-13T03:37:41.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:41 smithi161 bash[19769]: cluster 2024-02-13T03:37:41.067812+0000 mon.a (mon.0) 2817 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-02-13T03:37:41.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:41 smithi161 bash[19769]: audit 2024-02-13T03:37:41.070597+0000 mon.a (mon.0) 2818 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-118", "mode": "writeback"}]: dispatch 2024-02-13T03:37:42.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:42 smithi161 bash[27879]: cluster 2024-02-13T03:37:42.060274+0000 mon.a (mon.0) 2819 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:42.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:42 smithi161 bash[27879]: audit 2024-02-13T03:37:42.069329+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-118", "mode": "writeback"}]': finished 2024-02-13T03:37:42.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:42 smithi161 bash[27879]: cluster 2024-02-13T03:37:42.069849+0000 mon.a (mon.0) 2821 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-02-13T03:37:42.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:42 smithi161 bash[19769]: cluster 2024-02-13T03:37:42.060274+0000 mon.a (mon.0) 2819 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:42.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:42 smithi161 bash[19769]: audit 2024-02-13T03:37:42.069329+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-118", "mode": "writeback"}]': finished 2024-02-13T03:37:42.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:42 smithi161 bash[19769]: cluster 2024-02-13T03:37:42.069849+0000 mon.a (mon.0) 2821 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-02-13T03:37:42.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:42 smithi183 bash[20824]: cluster 2024-02-13T03:37:42.060274+0000 mon.a (mon.0) 2819 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:42 smithi183 bash[20824]: audit 2024-02-13T03:37:42.069329+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-118", "mode": "writeback"}]': finished 2024-02-13T03:37:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:42 smithi183 bash[20824]: cluster 2024-02-13T03:37:42.069849+0000 mon.a (mon.0) 2821 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-02-13T03:37:43.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:43 smithi161 bash[27879]: cluster 2024-02-13T03:37:41.528690+0000 mgr.y (mgr.24473) 646 : cluster [DBG] pgmap v1034: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:37:43.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:43 smithi161 bash[19769]: cluster 2024-02-13T03:37:41.528690+0000 mgr.y (mgr.24473) 646 : cluster [DBG] pgmap v1034: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:37:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:43 smithi183 bash[20824]: cluster 2024-02-13T03:37:41.528690+0000 mgr.y (mgr.24473) 646 : cluster [DBG] pgmap v1034: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:37:45.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:45 smithi161 bash[27879]: cluster 2024-02-13T03:37:43.529528+0000 mgr.y (mgr.24473) 647 : cluster [DBG] pgmap v1036: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-13T03:37:45.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:45 smithi161 bash[19769]: cluster 2024-02-13T03:37:43.529528+0000 mgr.y (mgr.24473) 647 : cluster [DBG] pgmap v1036: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-13T03:37:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:45 smithi183 bash[20824]: cluster 2024-02-13T03:37:43.529528+0000 mgr.y (mgr.24473) 647 : cluster [DBG] pgmap v1036: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-13T03:37:46.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:37:46 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:37:47.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:47 smithi161 bash[19769]: cluster 2024-02-13T03:37:45.530392+0000 mgr.y (mgr.24473) 648 : cluster [DBG] pgmap v1037: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:37:47.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:47 smithi161 bash[19769]: audit 2024-02-13T03:37:46.365765+0000 mgr.y (mgr.24473) 649 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:47.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:47 smithi161 bash[27879]: cluster 2024-02-13T03:37:45.530392+0000 mgr.y (mgr.24473) 648 : cluster [DBG] pgmap v1037: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:37:47.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:47 smithi161 bash[27879]: audit 2024-02-13T03:37:46.365765+0000 mgr.y (mgr.24473) 649 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:47 smithi183 bash[20824]: cluster 2024-02-13T03:37:45.530392+0000 mgr.y (mgr.24473) 648 : cluster [DBG] pgmap v1037: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:37:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:47 smithi183 bash[20824]: audit 2024-02-13T03:37:46.365765+0000 mgr.y (mgr.24473) 649 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:48.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:48 smithi161 bash[27879]: audit 2024-02-13T03:37:47.163813+0000 mon.a (mon.0) 2822 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:48.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:48 smithi161 bash[19769]: audit 2024-02-13T03:37:47.163813+0000 mon.a (mon.0) 2822 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:48 smithi183 bash[20824]: audit 2024-02-13T03:37:47.163813+0000 mon.a (mon.0) 2822 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:49.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:49 smithi161 bash[19769]: cluster 2024-02-13T03:37:47.532531+0000 mgr.y (mgr.24473) 650 : cluster [DBG] pgmap v1038: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 274 B/s wr, 1 op/s 2024-02-13T03:37:49.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:49 smithi161 bash[19769]: audit 2024-02-13T03:37:48.132233+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:49.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:49 smithi161 bash[19769]: cluster 2024-02-13T03:37:48.132315+0000 mon.a (mon.0) 2824 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-02-13T03:37:49.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:49 smithi161 bash[19769]: audit 2024-02-13T03:37:48.133390+0000 mon.a (mon.0) 2825 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-118"}]: dispatch 2024-02-13T03:37:49.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:49 smithi161 bash[27879]: cluster 2024-02-13T03:37:47.532531+0000 mgr.y (mgr.24473) 650 : cluster [DBG] pgmap v1038: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 274 B/s wr, 1 op/s 2024-02-13T03:37:49.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:49 smithi161 bash[27879]: audit 2024-02-13T03:37:48.132233+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:49.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:49 smithi161 bash[27879]: cluster 2024-02-13T03:37:48.132315+0000 mon.a (mon.0) 2824 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-02-13T03:37:49.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:49 smithi161 bash[27879]: audit 2024-02-13T03:37:48.133390+0000 mon.a (mon.0) 2825 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-118"}]: dispatch 2024-02-13T03:37:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:49 smithi183 bash[20824]: cluster 2024-02-13T03:37:47.532531+0000 mgr.y (mgr.24473) 650 : cluster [DBG] pgmap v1038: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 274 B/s wr, 1 op/s 2024-02-13T03:37:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:49 smithi183 bash[20824]: audit 2024-02-13T03:37:48.132233+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:49 smithi183 bash[20824]: cluster 2024-02-13T03:37:48.132315+0000 mon.a (mon.0) 2824 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-02-13T03:37:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:49 smithi183 bash[20824]: audit 2024-02-13T03:37:48.133390+0000 mon.a (mon.0) 2825 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-118"}]: dispatch 2024-02-13T03:37:49.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:37:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:37:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:37:50.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:50 smithi161 bash[27879]: cluster 2024-02-13T03:37:49.128175+0000 mon.a (mon.0) 2826 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:50.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:50 smithi161 bash[27879]: audit 2024-02-13T03:37:49.136114+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-118"}]': finished 2024-02-13T03:37:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:50 smithi161 bash[27879]: cluster 2024-02-13T03:37:49.136313+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-02-13T03:37:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:50 smithi161 bash[27879]: audit 2024-02-13T03:37:49.794272+0000 mon.a (mon.0) 2829 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:37:50.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:50 smithi161 bash[19769]: cluster 2024-02-13T03:37:49.128175+0000 mon.a (mon.0) 2826 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:50.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:50 smithi161 bash[19769]: audit 2024-02-13T03:37:49.136114+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-118"}]': finished 2024-02-13T03:37:50.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:50 smithi161 bash[19769]: cluster 2024-02-13T03:37:49.136313+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-02-13T03:37:50.429 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:50 smithi161 bash[19769]: audit 2024-02-13T03:37:49.794272+0000 mon.a (mon.0) 2829 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:37:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:50 smithi183 bash[20824]: cluster 2024-02-13T03:37:49.128175+0000 mon.a (mon.0) 2826 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:50.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:50 smithi183 bash[20824]: audit 2024-02-13T03:37:49.136114+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-118"}]': finished 2024-02-13T03:37:50.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:50 smithi183 bash[20824]: cluster 2024-02-13T03:37:49.136313+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-02-13T03:37:50.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:50 smithi183 bash[20824]: audit 2024-02-13T03:37:49.794272+0000 mon.a (mon.0) 2829 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:37:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:37:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:37:51] "GET /metrics HTTP/1.1" 200 189655 "" "Prometheus/2.33.4" 2024-02-13T03:37:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:51 smithi161 bash[27879]: cluster 2024-02-13T03:37:49.533326+0000 mgr.y (mgr.24473) 651 : cluster [DBG] pgmap v1041: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 822 B/s rd, 274 B/s wr, 1 op/s 2024-02-13T03:37:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:51 smithi161 bash[27879]: cluster 2024-02-13T03:37:50.140191+0000 mon.a (mon.0) 2830 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:37:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:51 smithi161 bash[27879]: cluster 2024-02-13T03:37:50.156846+0000 mon.a (mon.0) 2831 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-02-13T03:37:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:51 smithi161 bash[19769]: cluster 2024-02-13T03:37:49.533326+0000 mgr.y (mgr.24473) 651 : cluster [DBG] pgmap v1041: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 822 B/s rd, 274 B/s wr, 1 op/s 2024-02-13T03:37:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:51 smithi161 bash[19769]: cluster 2024-02-13T03:37:50.140191+0000 mon.a (mon.0) 2830 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:37:51.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:51 smithi161 bash[19769]: cluster 2024-02-13T03:37:50.156846+0000 mon.a (mon.0) 2831 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-02-13T03:37:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:51 smithi183 bash[20824]: cluster 2024-02-13T03:37:49.533326+0000 mgr.y (mgr.24473) 651 : cluster [DBG] pgmap v1041: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 822 B/s rd, 274 B/s wr, 1 op/s 2024-02-13T03:37:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:51 smithi183 bash[20824]: cluster 2024-02-13T03:37:50.140191+0000 mon.a (mon.0) 2830 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:37:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:51 smithi183 bash[20824]: cluster 2024-02-13T03:37:50.156846+0000 mon.a (mon.0) 2831 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-02-13T03:37:52.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:52 smithi161 bash[27879]: cluster 2024-02-13T03:37:51.170421+0000 mon.a (mon.0) 2832 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-02-13T03:37:52.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:52 smithi161 bash[27879]: audit 2024-02-13T03:37:51.176184+0000 mon.a (mon.0) 2833 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:52.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:52 smithi161 bash[19769]: cluster 2024-02-13T03:37:51.170421+0000 mon.a (mon.0) 2832 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-02-13T03:37:52.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:52 smithi161 bash[19769]: audit 2024-02-13T03:37:51.176184+0000 mon.a (mon.0) 2833 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:52.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:52 smithi183 bash[20824]: cluster 2024-02-13T03:37:51.170421+0000 mon.a (mon.0) 2832 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-02-13T03:37:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:52 smithi183 bash[20824]: audit 2024-02-13T03:37:51.176184+0000 mon.a (mon.0) 2833 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:37:53.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:53 smithi183 bash[20824]: cluster 2024-02-13T03:37:51.534211+0000 mgr.y (mgr.24473) 652 : cluster [DBG] pgmap v1044: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:37:53.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:53 smithi183 bash[20824]: audit 2024-02-13T03:37:52.176651+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:53.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:53 smithi183 bash[20824]: cluster 2024-02-13T03:37:52.176827+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-02-13T03:37:53.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:53 smithi183 bash[20824]: audit 2024-02-13T03:37:52.178434+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:53.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:53 smithi161 bash[27879]: cluster 2024-02-13T03:37:51.534211+0000 mgr.y (mgr.24473) 652 : cluster [DBG] pgmap v1044: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:37:53.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:53 smithi161 bash[27879]: audit 2024-02-13T03:37:52.176651+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:53.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:53 smithi161 bash[27879]: cluster 2024-02-13T03:37:52.176827+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-02-13T03:37:53.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:53 smithi161 bash[27879]: audit 2024-02-13T03:37:52.178434+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:53.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:53 smithi161 bash[19769]: cluster 2024-02-13T03:37:51.534211+0000 mgr.y (mgr.24473) 652 : cluster [DBG] pgmap v1044: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:37:53.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:53 smithi161 bash[19769]: audit 2024-02-13T03:37:52.176651+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:37:53.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:53 smithi161 bash[19769]: cluster 2024-02-13T03:37:52.176827+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-02-13T03:37:53.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:53 smithi161 bash[19769]: audit 2024-02-13T03:37:52.178434+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:37:54.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:54 smithi183 bash[20824]: audit 2024-02-13T03:37:53.182279+0000 mon.a (mon.0) 2837 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:54.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:54 smithi183 bash[20824]: cluster 2024-02-13T03:37:53.182488+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-02-13T03:37:54.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:54 smithi183 bash[20824]: audit 2024-02-13T03:37:53.183326+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-120"}]: dispatch 2024-02-13T03:37:54.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:54 smithi183 bash[20824]: audit 2024-02-13T03:37:54.183815+0000 mon.a (mon.0) 2840 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-120"}]': finished 2024-02-13T03:37:54.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:54 smithi183 bash[20824]: cluster 2024-02-13T03:37:54.183918+0000 mon.a (mon.0) 2841 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-02-13T03:37:54.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:54 smithi183 bash[20824]: audit 2024-02-13T03:37:54.185473+0000 mon.a (mon.0) 2842 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-120", "mode": "writeback"}]: dispatch 2024-02-13T03:37:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:54 smithi161 bash[27879]: audit 2024-02-13T03:37:53.182279+0000 mon.a (mon.0) 2837 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:54 smithi161 bash[27879]: cluster 2024-02-13T03:37:53.182488+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-02-13T03:37:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:54 smithi161 bash[27879]: audit 2024-02-13T03:37:53.183326+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-120"}]: dispatch 2024-02-13T03:37:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:54 smithi161 bash[27879]: audit 2024-02-13T03:37:54.183815+0000 mon.a (mon.0) 2840 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-120"}]': finished 2024-02-13T03:37:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:54 smithi161 bash[27879]: cluster 2024-02-13T03:37:54.183918+0000 mon.a (mon.0) 2841 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-02-13T03:37:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:54 smithi161 bash[27879]: audit 2024-02-13T03:37:54.185473+0000 mon.a (mon.0) 2842 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-120", "mode": "writeback"}]: dispatch 2024-02-13T03:37:54.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:54 smithi161 bash[19769]: audit 2024-02-13T03:37:53.182279+0000 mon.a (mon.0) 2837 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:37:54.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:54 smithi161 bash[19769]: cluster 2024-02-13T03:37:53.182488+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-02-13T03:37:54.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:54 smithi161 bash[19769]: audit 2024-02-13T03:37:53.183326+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-120"}]: dispatch 2024-02-13T03:37:54.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:54 smithi161 bash[19769]: audit 2024-02-13T03:37:54.183815+0000 mon.a (mon.0) 2840 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-120"}]': finished 2024-02-13T03:37:54.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:54 smithi161 bash[19769]: cluster 2024-02-13T03:37:54.183918+0000 mon.a (mon.0) 2841 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-02-13T03:37:54.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:54 smithi161 bash[19769]: audit 2024-02-13T03:37:54.185473+0000 mon.a (mon.0) 2842 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-120", "mode": "writeback"}]: dispatch 2024-02-13T03:37:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:55 smithi183 bash[20824]: cluster 2024-02-13T03:37:53.535183+0000 mgr.y (mgr.24473) 653 : cluster [DBG] pgmap v1047: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:37:55.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:55 smithi183 bash[20824]: cluster 2024-02-13T03:37:55.178918+0000 mon.a (mon.0) 2843 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:55.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:55 smithi183 bash[20824]: audit 2024-02-13T03:37:55.187296+0000 mon.a (mon.0) 2844 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-120", "mode": "writeback"}]': finished 2024-02-13T03:37:55.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:55 smithi183 bash[20824]: cluster 2024-02-13T03:37:55.187434+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-02-13T03:37:55.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:55 smithi161 bash[19769]: cluster 2024-02-13T03:37:53.535183+0000 mgr.y (mgr.24473) 653 : cluster [DBG] pgmap v1047: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:37:55.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:55 smithi161 bash[19769]: cluster 2024-02-13T03:37:55.178918+0000 mon.a (mon.0) 2843 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:55.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:55 smithi161 bash[19769]: audit 2024-02-13T03:37:55.187296+0000 mon.a (mon.0) 2844 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-120", "mode": "writeback"}]': finished 2024-02-13T03:37:55.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:55 smithi161 bash[19769]: cluster 2024-02-13T03:37:55.187434+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-02-13T03:37:55.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:55 smithi161 bash[27879]: cluster 2024-02-13T03:37:53.535183+0000 mgr.y (mgr.24473) 653 : cluster [DBG] pgmap v1047: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:37:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:55 smithi161 bash[27879]: cluster 2024-02-13T03:37:55.178918+0000 mon.a (mon.0) 2843 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:37:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:55 smithi161 bash[27879]: audit 2024-02-13T03:37:55.187296+0000 mon.a (mon.0) 2844 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-120", "mode": "writeback"}]': finished 2024-02-13T03:37:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:55 smithi161 bash[27879]: cluster 2024-02-13T03:37:55.187434+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-02-13T03:37:56.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:37:56 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:37:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:56 smithi183 bash[20824]: audit 2024-02-13T03:37:55.302693+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:56 smithi161 bash[27879]: audit 2024-02-13T03:37:55.302693+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:56.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:56 smithi161 bash[19769]: audit 2024-02-13T03:37:55.302693+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:37:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:57 smithi183 bash[20824]: cluster 2024-02-13T03:37:55.536018+0000 mgr.y (mgr.24473) 654 : cluster [DBG] pgmap v1050: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:57 smithi183 bash[20824]: audit 2024-02-13T03:37:56.215380+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:57 smithi183 bash[20824]: cluster 2024-02-13T03:37:56.215503+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-02-13T03:37:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:57 smithi183 bash[20824]: audit 2024-02-13T03:37:56.216287+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-120"}]: dispatch 2024-02-13T03:37:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:57 smithi183 bash[20824]: audit 2024-02-13T03:37:56.377146+0000 mgr.y (mgr.24473) 655 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:57.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:57 smithi161 bash[19769]: cluster 2024-02-13T03:37:55.536018+0000 mgr.y (mgr.24473) 654 : cluster [DBG] pgmap v1050: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:57.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:57 smithi161 bash[19769]: audit 2024-02-13T03:37:56.215380+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:57.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:57 smithi161 bash[19769]: cluster 2024-02-13T03:37:56.215503+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-02-13T03:37:57.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:57 smithi161 bash[19769]: audit 2024-02-13T03:37:56.216287+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-120"}]: dispatch 2024-02-13T03:37:57.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:57 smithi161 bash[19769]: audit 2024-02-13T03:37:56.377146+0000 mgr.y (mgr.24473) 655 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:57 smithi161 bash[27879]: cluster 2024-02-13T03:37:55.536018+0000 mgr.y (mgr.24473) 654 : cluster [DBG] pgmap v1050: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:37:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:57 smithi161 bash[27879]: audit 2024-02-13T03:37:56.215380+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:37:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:57 smithi161 bash[27879]: cluster 2024-02-13T03:37:56.215503+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-02-13T03:37:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:57 smithi161 bash[27879]: audit 2024-02-13T03:37:56.216287+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-120"}]: dispatch 2024-02-13T03:37:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:57 smithi161 bash[27879]: audit 2024-02-13T03:37:56.377146+0000 mgr.y (mgr.24473) 655 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:37:58.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:58 smithi183 bash[20824]: cluster 2024-02-13T03:37:57.208599+0000 mon.a (mon.0) 2850 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:58 smithi183 bash[20824]: audit 2024-02-13T03:37:57.217753+0000 mon.a (mon.0) 2851 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-120"}]': finished 2024-02-13T03:37:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:58 smithi183 bash[20824]: cluster 2024-02-13T03:37:57.218052+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-02-13T03:37:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:58 smithi161 bash[27879]: cluster 2024-02-13T03:37:57.208599+0000 mon.a (mon.0) 2850 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:58 smithi161 bash[27879]: audit 2024-02-13T03:37:57.217753+0000 mon.a (mon.0) 2851 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-120"}]': finished 2024-02-13T03:37:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:58 smithi161 bash[27879]: cluster 2024-02-13T03:37:57.218052+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-02-13T03:37:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:58 smithi161 bash[19769]: cluster 2024-02-13T03:37:57.208599+0000 mon.a (mon.0) 2850 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:37:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:58 smithi161 bash[19769]: audit 2024-02-13T03:37:57.217753+0000 mon.a (mon.0) 2851 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-120"}]': finished 2024-02-13T03:37:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:58 smithi161 bash[19769]: cluster 2024-02-13T03:37:57.218052+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-02-13T03:37:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:59 smithi161 bash[27879]: cluster 2024-02-13T03:37:57.536848+0000 mgr.y (mgr.24473) 656 : cluster [DBG] pgmap v1053: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-13T03:37:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:59 smithi161 bash[27879]: cluster 2024-02-13T03:37:58.239903+0000 mon.a (mon.0) 2853 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-13T03:37:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:37:59 smithi161 bash[27879]: cluster 2024-02-13T03:37:59.032840+0000 mon.a (mon.0) 2854 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:37:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:59 smithi161 bash[19769]: cluster 2024-02-13T03:37:57.536848+0000 mgr.y (mgr.24473) 656 : cluster [DBG] pgmap v1053: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-13T03:37:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:59 smithi161 bash[19769]: cluster 2024-02-13T03:37:58.239903+0000 mon.a (mon.0) 2853 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-13T03:37:59.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:37:59 smithi161 bash[19769]: cluster 2024-02-13T03:37:59.032840+0000 mon.a (mon.0) 2854 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:37:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:59 smithi183 bash[20824]: cluster 2024-02-13T03:37:57.536848+0000 mgr.y (mgr.24473) 656 : cluster [DBG] pgmap v1053: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-13T03:37:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:59 smithi183 bash[20824]: cluster 2024-02-13T03:37:58.239903+0000 mon.a (mon.0) 2853 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-13T03:37:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:37:59 smithi183 bash[20824]: cluster 2024-02-13T03:37:59.032840+0000 mon.a (mon.0) 2854 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:37:59.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:37:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:37:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:38:00.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:00 smithi161 bash[27879]: cluster 2024-02-13T03:37:59.248075+0000 mon.a (mon.0) 2855 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-13T03:38:00.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:00 smithi161 bash[27879]: audit 2024-02-13T03:37:59.258032+0000 mon.a (mon.0) 2856 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:00.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:00 smithi161 bash[27879]: audit 2024-02-13T03:38:00.248271+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:00.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:00 smithi161 bash[27879]: cluster 2024-02-13T03:38:00.248574+0000 mon.a (mon.0) 2858 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-13T03:38:00.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:00 smithi161 bash[27879]: audit 2024-02-13T03:38:00.252337+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:00 smithi161 bash[19769]: cluster 2024-02-13T03:37:59.248075+0000 mon.a (mon.0) 2855 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-13T03:38:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:00 smithi161 bash[19769]: audit 2024-02-13T03:37:59.258032+0000 mon.a (mon.0) 2856 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:00 smithi161 bash[19769]: audit 2024-02-13T03:38:00.248271+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:00 smithi161 bash[19769]: cluster 2024-02-13T03:38:00.248574+0000 mon.a (mon.0) 2858 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-13T03:38:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:00 smithi161 bash[19769]: audit 2024-02-13T03:38:00.252337+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:00.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:00 smithi183 bash[20824]: cluster 2024-02-13T03:37:59.248075+0000 mon.a (mon.0) 2855 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-13T03:38:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:00 smithi183 bash[20824]: audit 2024-02-13T03:37:59.258032+0000 mon.a (mon.0) 2856 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:00 smithi183 bash[20824]: audit 2024-02-13T03:38:00.248271+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:00 smithi183 bash[20824]: cluster 2024-02-13T03:38:00.248574+0000 mon.a (mon.0) 2858 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-13T03:38:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:00 smithi183 bash[20824]: audit 2024-02-13T03:38:00.252337+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:01.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:38:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:38:01] "GET /metrics HTTP/1.1" 200 189655 "" "Prometheus/2.33.4" 2024-02-13T03:38:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:01 smithi161 bash[27879]: cluster 2024-02-13T03:37:59.537670+0000 mgr.y (mgr.24473) 657 : cluster [DBG] pgmap v1056: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:38:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:01 smithi161 bash[27879]: audit 2024-02-13T03:38:01.251694+0000 mon.a (mon.0) 2860 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:01 smithi161 bash[27879]: cluster 2024-02-13T03:38:01.251862+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-13T03:38:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:01 smithi161 bash[27879]: audit 2024-02-13T03:38:01.252594+0000 mon.a (mon.0) 2862 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-122"}]: dispatch 2024-02-13T03:38:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:01 smithi161 bash[19769]: cluster 2024-02-13T03:37:59.537670+0000 mgr.y (mgr.24473) 657 : cluster [DBG] pgmap v1056: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:38:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:01 smithi161 bash[19769]: audit 2024-02-13T03:38:01.251694+0000 mon.a (mon.0) 2860 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:01 smithi161 bash[19769]: cluster 2024-02-13T03:38:01.251862+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-13T03:38:01.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:01 smithi161 bash[19769]: audit 2024-02-13T03:38:01.252594+0000 mon.a (mon.0) 2862 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-122"}]: dispatch 2024-02-13T03:38:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:01 smithi183 bash[20824]: cluster 2024-02-13T03:37:59.537670+0000 mgr.y (mgr.24473) 657 : cluster [DBG] pgmap v1056: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T03:38:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:01 smithi183 bash[20824]: audit 2024-02-13T03:38:01.251694+0000 mon.a (mon.0) 2860 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:01 smithi183 bash[20824]: cluster 2024-02-13T03:38:01.251862+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-13T03:38:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:01 smithi183 bash[20824]: audit 2024-02-13T03:38:01.252594+0000 mon.a (mon.0) 2862 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-122"}]: dispatch 2024-02-13T03:38:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:03 smithi161 bash[27879]: cluster 2024-02-13T03:38:01.538544+0000 mgr.y (mgr.24473) 658 : cluster [DBG] pgmap v1059: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:38:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:03 smithi161 bash[27879]: audit 2024-02-13T03:38:02.253254+0000 mon.a (mon.0) 2863 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-122"}]': finished 2024-02-13T03:38:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:03 smithi161 bash[27879]: cluster 2024-02-13T03:38:02.253480+0000 mon.a (mon.0) 2864 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-13T03:38:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:03 smithi161 bash[27879]: audit 2024-02-13T03:38:02.254258+0000 mon.a (mon.0) 2865 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-122", "mode": "writeback"}]: dispatch 2024-02-13T03:38:03.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:03 smithi161 bash[19769]: cluster 2024-02-13T03:38:01.538544+0000 mgr.y (mgr.24473) 658 : cluster [DBG] pgmap v1059: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:38:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:03 smithi161 bash[19769]: audit 2024-02-13T03:38:02.253254+0000 mon.a (mon.0) 2863 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-122"}]': finished 2024-02-13T03:38:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:03 smithi161 bash[19769]: cluster 2024-02-13T03:38:02.253480+0000 mon.a (mon.0) 2864 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-13T03:38:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:03 smithi161 bash[19769]: audit 2024-02-13T03:38:02.254258+0000 mon.a (mon.0) 2865 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-122", "mode": "writeback"}]: dispatch 2024-02-13T03:38:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:03 smithi183 bash[20824]: cluster 2024-02-13T03:38:01.538544+0000 mgr.y (mgr.24473) 658 : cluster [DBG] pgmap v1059: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:38:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:03 smithi183 bash[20824]: audit 2024-02-13T03:38:02.253254+0000 mon.a (mon.0) 2863 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-122"}]': finished 2024-02-13T03:38:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:03 smithi183 bash[20824]: cluster 2024-02-13T03:38:02.253480+0000 mon.a (mon.0) 2864 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-13T03:38:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:03 smithi183 bash[20824]: audit 2024-02-13T03:38:02.254258+0000 mon.a (mon.0) 2865 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-122", "mode": "writeback"}]: dispatch 2024-02-13T03:38:04.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:04 smithi161 bash[27879]: cluster 2024-02-13T03:38:03.247597+0000 mon.a (mon.0) 2866 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:38:04.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:04 smithi161 bash[27879]: audit 2024-02-13T03:38:03.259058+0000 mon.a (mon.0) 2867 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-122", "mode": "writeback"}]': finished 2024-02-13T03:38:04.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:04 smithi161 bash[27879]: cluster 2024-02-13T03:38:03.259433+0000 mon.a (mon.0) 2868 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-13T03:38:04.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:04 smithi161 bash[19769]: cluster 2024-02-13T03:38:03.247597+0000 mon.a (mon.0) 2866 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:38:04.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:04 smithi161 bash[19769]: audit 2024-02-13T03:38:03.259058+0000 mon.a (mon.0) 2867 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-122", "mode": "writeback"}]': finished 2024-02-13T03:38:04.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:04 smithi161 bash[19769]: cluster 2024-02-13T03:38:03.259433+0000 mon.a (mon.0) 2868 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-13T03:38:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:04 smithi183 bash[20824]: cluster 2024-02-13T03:38:03.247597+0000 mon.a (mon.0) 2866 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:38:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:04 smithi183 bash[20824]: audit 2024-02-13T03:38:03.259058+0000 mon.a (mon.0) 2867 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-122", "mode": "writeback"}]': finished 2024-02-13T03:38:04.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:04 smithi183 bash[20824]: cluster 2024-02-13T03:38:03.259433+0000 mon.a (mon.0) 2868 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-13T03:38:05.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:05 smithi161 bash[27879]: cluster 2024-02-13T03:38:03.539389+0000 mgr.y (mgr.24473) 659 : cluster [DBG] pgmap v1062: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:05.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:05 smithi161 bash[27879]: cluster 2024-02-13T03:38:04.276211+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-13T03:38:05.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:05 smithi161 bash[27879]: audit 2024-02-13T03:38:04.794768+0000 mon.a (mon.0) 2870 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:38:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:05 smithi161 bash[19769]: cluster 2024-02-13T03:38:03.539389+0000 mgr.y (mgr.24473) 659 : cluster [DBG] pgmap v1062: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:05 smithi161 bash[19769]: cluster 2024-02-13T03:38:04.276211+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-13T03:38:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:05 smithi161 bash[19769]: audit 2024-02-13T03:38:04.794768+0000 mon.a (mon.0) 2870 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:38:05.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:05 smithi183 bash[20824]: cluster 2024-02-13T03:38:03.539389+0000 mgr.y (mgr.24473) 659 : cluster [DBG] pgmap v1062: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:05 smithi183 bash[20824]: cluster 2024-02-13T03:38:04.276211+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-13T03:38:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:05 smithi183 bash[20824]: audit 2024-02-13T03:38:04.794768+0000 mon.a (mon.0) 2870 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:38:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:06 smithi161 bash[19769]: cluster 2024-02-13T03:38:05.281930+0000 mon.a (mon.0) 2871 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-13T03:38:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:06 smithi161 bash[19769]: audit 2024-02-13T03:38:05.329443+0000 mon.a (mon.0) 2872 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:06 smithi161 bash[27879]: cluster 2024-02-13T03:38:05.281930+0000 mon.a (mon.0) 2871 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-13T03:38:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:06 smithi161 bash[27879]: audit 2024-02-13T03:38:05.329443+0000 mon.a (mon.0) 2872 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:06.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:38:06 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:38:06.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:06 smithi183 bash[20824]: cluster 2024-02-13T03:38:05.281930+0000 mon.a (mon.0) 2871 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-13T03:38:06.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:06 smithi183 bash[20824]: audit 2024-02-13T03:38:05.329443+0000 mon.a (mon.0) 2872 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:07 smithi161 bash[27879]: cluster 2024-02-13T03:38:05.540250+0000 mgr.y (mgr.24473) 660 : cluster [DBG] pgmap v1065: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:38:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:07 smithi161 bash[27879]: audit 2024-02-13T03:38:06.299761+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:38:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:07 smithi161 bash[27879]: cluster 2024-02-13T03:38:06.299980+0000 mon.a (mon.0) 2874 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-13T03:38:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:07 smithi161 bash[27879]: audit 2024-02-13T03:38:06.311020+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-122"}]: dispatch 2024-02-13T03:38:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:07 smithi161 bash[27879]: audit 2024-02-13T03:38:06.388593+0000 mgr.y (mgr.24473) 661 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:07 smithi161 bash[19769]: cluster 2024-02-13T03:38:05.540250+0000 mgr.y (mgr.24473) 660 : cluster [DBG] pgmap v1065: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:38:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:07 smithi161 bash[19769]: audit 2024-02-13T03:38:06.299761+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:38:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:07 smithi161 bash[19769]: cluster 2024-02-13T03:38:06.299980+0000 mon.a (mon.0) 2874 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-13T03:38:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:07 smithi161 bash[19769]: audit 2024-02-13T03:38:06.311020+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-122"}]: dispatch 2024-02-13T03:38:07.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:07 smithi161 bash[19769]: audit 2024-02-13T03:38:06.388593+0000 mgr.y (mgr.24473) 661 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:07 smithi183 bash[20824]: cluster 2024-02-13T03:38:05.540250+0000 mgr.y (mgr.24473) 660 : cluster [DBG] pgmap v1065: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:38:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:07 smithi183 bash[20824]: audit 2024-02-13T03:38:06.299761+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:38:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:07 smithi183 bash[20824]: cluster 2024-02-13T03:38:06.299980+0000 mon.a (mon.0) 2874 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-13T03:38:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:07 smithi183 bash[20824]: audit 2024-02-13T03:38:06.311020+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-122"}]: dispatch 2024-02-13T03:38:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:07 smithi183 bash[20824]: audit 2024-02-13T03:38:06.388593+0000 mgr.y (mgr.24473) 661 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:08 smithi161 bash[27879]: audit 2024-02-13T03:38:07.313999+0000 mon.a (mon.0) 2876 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-122"}]': finished 2024-02-13T03:38:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:08 smithi161 bash[27879]: cluster 2024-02-13T03:38:07.314188+0000 mon.a (mon.0) 2877 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-13T03:38:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:08 smithi161 bash[27879]: cluster 2024-02-13T03:38:08.315925+0000 mon.a (mon.0) 2878 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-13T03:38:08.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:08 smithi161 bash[19769]: audit 2024-02-13T03:38:07.313999+0000 mon.a (mon.0) 2876 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-122"}]': finished 2024-02-13T03:38:08.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:08 smithi161 bash[19769]: cluster 2024-02-13T03:38:07.314188+0000 mon.a (mon.0) 2877 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-13T03:38:08.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:08 smithi161 bash[19769]: cluster 2024-02-13T03:38:08.315925+0000 mon.a (mon.0) 2878 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-13T03:38:08.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:08 smithi183 bash[20824]: audit 2024-02-13T03:38:07.313999+0000 mon.a (mon.0) 2876 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-122"}]': finished 2024-02-13T03:38:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:08 smithi183 bash[20824]: cluster 2024-02-13T03:38:07.314188+0000 mon.a (mon.0) 2877 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-13T03:38:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:08 smithi183 bash[20824]: cluster 2024-02-13T03:38:08.315925+0000 mon.a (mon.0) 2878 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-13T03:38:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:09 smithi161 bash[27879]: cluster 2024-02-13T03:38:07.541143+0000 mgr.y (mgr.24473) 662 : cluster [DBG] pgmap v1068: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:38:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:09 smithi161 bash[27879]: audit 2024-02-13T03:38:08.351791+0000 mon.a (mon.0) 2879 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:09 smithi161 bash[19769]: cluster 2024-02-13T03:38:07.541143+0000 mgr.y (mgr.24473) 662 : cluster [DBG] pgmap v1068: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:38:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:09 smithi161 bash[19769]: audit 2024-02-13T03:38:08.351791+0000 mon.a (mon.0) 2879 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:09.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:38:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:38:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:38:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:09 smithi183 bash[20824]: cluster 2024-02-13T03:38:07.541143+0000 mgr.y (mgr.24473) 662 : cluster [DBG] pgmap v1068: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:38:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:09 smithi183 bash[20824]: audit 2024-02-13T03:38:08.351791+0000 mon.a (mon.0) 2879 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:10 smithi161 bash[27879]: audit 2024-02-13T03:38:09.343359+0000 mon.a (mon.0) 2880 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:38:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:10 smithi161 bash[27879]: cluster 2024-02-13T03:38:09.343546+0000 mon.a (mon.0) 2881 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-13T03:38:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:10 smithi161 bash[27879]: audit 2024-02-13T03:38:09.348218+0000 mon.a (mon.0) 2882 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-122"}]: dispatch 2024-02-13T03:38:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:10 smithi161 bash[19769]: audit 2024-02-13T03:38:09.343359+0000 mon.a (mon.0) 2880 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:38:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:10 smithi161 bash[19769]: cluster 2024-02-13T03:38:09.343546+0000 mon.a (mon.0) 2881 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-13T03:38:10.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:10 smithi161 bash[19769]: audit 2024-02-13T03:38:09.348218+0000 mon.a (mon.0) 2882 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-122"}]: dispatch 2024-02-13T03:38:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:10 smithi183 bash[20824]: audit 2024-02-13T03:38:09.343359+0000 mon.a (mon.0) 2880 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:38:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:10 smithi183 bash[20824]: cluster 2024-02-13T03:38:09.343546+0000 mon.a (mon.0) 2881 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-13T03:38:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:10 smithi183 bash[20824]: audit 2024-02-13T03:38:09.348218+0000 mon.a (mon.0) 2882 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-122"}]: dispatch 2024-02-13T03:38:11.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:38:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:38:11] "GET /metrics HTTP/1.1" 200 189750 "" "Prometheus/2.33.4" 2024-02-13T03:38:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:11 smithi161 bash[27879]: cluster 2024-02-13T03:38:09.541973+0000 mgr.y (mgr.24473) 663 : cluster [DBG] pgmap v1071: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:38:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:11 smithi161 bash[27879]: cluster 2024-02-13T03:38:10.339621+0000 mon.a (mon.0) 2883 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:38:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:11 smithi161 bash[27879]: audit 2024-02-13T03:38:10.355259+0000 mon.a (mon.0) 2884 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-122"}]': finished 2024-02-13T03:38:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:11 smithi161 bash[27879]: cluster 2024-02-13T03:38:10.355499+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-13T03:38:11.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:11 smithi161 bash[19769]: cluster 2024-02-13T03:38:09.541973+0000 mgr.y (mgr.24473) 663 : cluster [DBG] pgmap v1071: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:38:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:11 smithi161 bash[19769]: cluster 2024-02-13T03:38:10.339621+0000 mon.a (mon.0) 2883 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:38:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:11 smithi161 bash[19769]: audit 2024-02-13T03:38:10.355259+0000 mon.a (mon.0) 2884 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-122"}]': finished 2024-02-13T03:38:11.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:11 smithi161 bash[19769]: cluster 2024-02-13T03:38:10.355499+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-13T03:38:11.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:11 smithi183 bash[20824]: cluster 2024-02-13T03:38:09.541973+0000 mgr.y (mgr.24473) 663 : cluster [DBG] pgmap v1071: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-13T03:38:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:11 smithi183 bash[20824]: cluster 2024-02-13T03:38:10.339621+0000 mon.a (mon.0) 2883 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:38:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:11 smithi183 bash[20824]: audit 2024-02-13T03:38:10.355259+0000 mon.a (mon.0) 2884 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-122"}]': finished 2024-02-13T03:38:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:11 smithi183 bash[20824]: cluster 2024-02-13T03:38:10.355499+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-13T03:38:12.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:12 smithi161 bash[27879]: cluster 2024-02-13T03:38:11.358062+0000 mon.a (mon.0) 2886 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-13T03:38:12.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:12 smithi161 bash[19769]: cluster 2024-02-13T03:38:11.358062+0000 mon.a (mon.0) 2886 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-13T03:38:12.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:12 smithi183 bash[20824]: cluster 2024-02-13T03:38:11.358062+0000 mon.a (mon.0) 2886 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-13T03:38:13.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:13 smithi161 bash[27879]: cluster 2024-02-13T03:38:11.542884+0000 mgr.y (mgr.24473) 664 : cluster [DBG] pgmap v1074: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:13.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:13 smithi161 bash[27879]: cluster 2024-02-13T03:38:12.370706+0000 mon.a (mon.0) 2887 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-13T03:38:13.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:13 smithi161 bash[27879]: audit 2024-02-13T03:38:12.371938+0000 mon.a (mon.0) 2888 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:13.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:13 smithi161 bash[19769]: cluster 2024-02-13T03:38:11.542884+0000 mgr.y (mgr.24473) 664 : cluster [DBG] pgmap v1074: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:13.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:13 smithi161 bash[19769]: cluster 2024-02-13T03:38:12.370706+0000 mon.a (mon.0) 2887 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-13T03:38:13.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:13 smithi161 bash[19769]: audit 2024-02-13T03:38:12.371938+0000 mon.a (mon.0) 2888 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:13 smithi183 bash[20824]: cluster 2024-02-13T03:38:11.542884+0000 mgr.y (mgr.24473) 664 : cluster [DBG] pgmap v1074: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:13 smithi183 bash[20824]: cluster 2024-02-13T03:38:12.370706+0000 mon.a (mon.0) 2887 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-13T03:38:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:13 smithi183 bash[20824]: audit 2024-02-13T03:38:12.371938+0000 mon.a (mon.0) 2888 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:14.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:14 smithi161 bash[27879]: audit 2024-02-13T03:38:13.379394+0000 mon.a (mon.0) 2889 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:14.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:14 smithi161 bash[27879]: cluster 2024-02-13T03:38:13.379683+0000 mon.a (mon.0) 2890 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-13T03:38:14.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:14 smithi161 bash[27879]: audit 2024-02-13T03:38:13.382539+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:14 smithi161 bash[27879]: audit 2024-02-13T03:38:14.048105+0000 mon.a (mon.0) 2892 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:14 smithi161 bash[27879]: cluster 2024-02-13T03:38:14.048302+0000 mon.a (mon.0) 2893 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-13T03:38:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:14 smithi161 bash[27879]: audit 2024-02-13T03:38:14.049128+0000 mon.a (mon.0) 2894 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-124"}]: dispatch 2024-02-13T03:38:14.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:14 smithi161 bash[19769]: audit 2024-02-13T03:38:13.379394+0000 mon.a (mon.0) 2889 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:14.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:14 smithi161 bash[19769]: cluster 2024-02-13T03:38:13.379683+0000 mon.a (mon.0) 2890 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-13T03:38:14.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:14 smithi161 bash[19769]: audit 2024-02-13T03:38:13.382539+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:14.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:14 smithi161 bash[19769]: audit 2024-02-13T03:38:14.048105+0000 mon.a (mon.0) 2892 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:14.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:14 smithi161 bash[19769]: cluster 2024-02-13T03:38:14.048302+0000 mon.a (mon.0) 2893 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-13T03:38:14.678 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:14 smithi161 bash[19769]: audit 2024-02-13T03:38:14.049128+0000 mon.a (mon.0) 2894 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-124"}]: dispatch 2024-02-13T03:38:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:14 smithi183 bash[20824]: audit 2024-02-13T03:38:13.379394+0000 mon.a (mon.0) 2889 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:14 smithi183 bash[20824]: cluster 2024-02-13T03:38:13.379683+0000 mon.a (mon.0) 2890 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-13T03:38:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:14 smithi183 bash[20824]: audit 2024-02-13T03:38:13.382539+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:14 smithi183 bash[20824]: audit 2024-02-13T03:38:14.048105+0000 mon.a (mon.0) 2892 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:14 smithi183 bash[20824]: cluster 2024-02-13T03:38:14.048302+0000 mon.a (mon.0) 2893 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-13T03:38:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:14 smithi183 bash[20824]: audit 2024-02-13T03:38:14.049128+0000 mon.a (mon.0) 2894 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-124"}]: dispatch 2024-02-13T03:38:15.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:15 smithi161 bash[27879]: cluster 2024-02-13T03:38:13.543856+0000 mgr.y (mgr.24473) 665 : cluster [DBG] pgmap v1077: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:15.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:15 smithi161 bash[27879]: audit 2024-02-13T03:38:15.051571+0000 mon.a (mon.0) 2895 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-124"}]': finished 2024-02-13T03:38:15.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:15 smithi161 bash[27879]: cluster 2024-02-13T03:38:15.051801+0000 mon.a (mon.0) 2896 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-13T03:38:15.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:15 smithi161 bash[27879]: audit 2024-02-13T03:38:15.054322+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-124", "mode": "writeback"}]: dispatch 2024-02-13T03:38:15.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:15 smithi161 bash[19769]: cluster 2024-02-13T03:38:13.543856+0000 mgr.y (mgr.24473) 665 : cluster [DBG] pgmap v1077: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:15.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:15 smithi161 bash[19769]: audit 2024-02-13T03:38:15.051571+0000 mon.a (mon.0) 2895 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-124"}]': finished 2024-02-13T03:38:15.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:15 smithi161 bash[19769]: cluster 2024-02-13T03:38:15.051801+0000 mon.a (mon.0) 2896 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-13T03:38:15.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:15 smithi161 bash[19769]: audit 2024-02-13T03:38:15.054322+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-124", "mode": "writeback"}]: dispatch 2024-02-13T03:38:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:15 smithi183 bash[20824]: cluster 2024-02-13T03:38:13.543856+0000 mgr.y (mgr.24473) 665 : cluster [DBG] pgmap v1077: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:15 smithi183 bash[20824]: audit 2024-02-13T03:38:15.051571+0000 mon.a (mon.0) 2895 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-124"}]': finished 2024-02-13T03:38:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:15 smithi183 bash[20824]: cluster 2024-02-13T03:38:15.051801+0000 mon.a (mon.0) 2896 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-13T03:38:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:15 smithi183 bash[20824]: audit 2024-02-13T03:38:15.054322+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-124", "mode": "writeback"}]: dispatch 2024-02-13T03:38:16.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:16 smithi161 bash[27879]: cluster 2024-02-13T03:38:16.044647+0000 mon.a (mon.0) 2898 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:38:16.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:16 smithi161 bash[27879]: audit 2024-02-13T03:38:16.050573+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-124", "mode": "writeback"}]': finished 2024-02-13T03:38:16.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:16 smithi161 bash[27879]: cluster 2024-02-13T03:38:16.050693+0000 mon.a (mon.0) 2900 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-13T03:38:16.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:16 smithi161 bash[27879]: audit 2024-02-13T03:38:16.099064+0000 mon.a (mon.0) 2901 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:16.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:16 smithi161 bash[19769]: cluster 2024-02-13T03:38:16.044647+0000 mon.a (mon.0) 2898 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:38:16.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:16 smithi161 bash[19769]: audit 2024-02-13T03:38:16.050573+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-124", "mode": "writeback"}]': finished 2024-02-13T03:38:16.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:16 smithi161 bash[19769]: cluster 2024-02-13T03:38:16.050693+0000 mon.a (mon.0) 2900 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-13T03:38:16.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:16 smithi161 bash[19769]: audit 2024-02-13T03:38:16.099064+0000 mon.a (mon.0) 2901 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:16.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:38:16 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:38:16.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:16 smithi183 bash[20824]: cluster 2024-02-13T03:38:16.044647+0000 mon.a (mon.0) 2898 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:38:16.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:16 smithi183 bash[20824]: audit 2024-02-13T03:38:16.050573+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-124", "mode": "writeback"}]': finished 2024-02-13T03:38:16.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:16 smithi183 bash[20824]: cluster 2024-02-13T03:38:16.050693+0000 mon.a (mon.0) 2900 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-13T03:38:16.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:16 smithi183 bash[20824]: audit 2024-02-13T03:38:16.099064+0000 mon.a (mon.0) 2901 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:17.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:17 smithi161 bash[27879]: cluster 2024-02-13T03:38:15.544692+0000 mgr.y (mgr.24473) 666 : cluster [DBG] pgmap v1080: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:38:17.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:17 smithi161 bash[27879]: audit 2024-02-13T03:38:16.399832+0000 mgr.y (mgr.24473) 667 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:17.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:17 smithi161 bash[27879]: audit 2024-02-13T03:38:17.056079+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:38:17.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:17 smithi161 bash[27879]: cluster 2024-02-13T03:38:17.056193+0000 mon.a (mon.0) 2903 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-13T03:38:17.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:17 smithi161 bash[27879]: audit 2024-02-13T03:38:17.058568+0000 mon.a (mon.0) 2904 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-124"}]: dispatch 2024-02-13T03:38:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:17 smithi161 bash[19769]: cluster 2024-02-13T03:38:15.544692+0000 mgr.y (mgr.24473) 666 : cluster [DBG] pgmap v1080: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:38:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:17 smithi161 bash[19769]: audit 2024-02-13T03:38:16.399832+0000 mgr.y (mgr.24473) 667 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:17 smithi161 bash[19769]: audit 2024-02-13T03:38:17.056079+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:38:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:17 smithi161 bash[19769]: cluster 2024-02-13T03:38:17.056193+0000 mon.a (mon.0) 2903 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-13T03:38:17.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:17 smithi161 bash[19769]: audit 2024-02-13T03:38:17.058568+0000 mon.a (mon.0) 2904 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-124"}]: dispatch 2024-02-13T03:38:17.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:17 smithi183 bash[20824]: cluster 2024-02-13T03:38:15.544692+0000 mgr.y (mgr.24473) 666 : cluster [DBG] pgmap v1080: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:38:17.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:17 smithi183 bash[20824]: audit 2024-02-13T03:38:16.399832+0000 mgr.y (mgr.24473) 667 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:17.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:17 smithi183 bash[20824]: audit 2024-02-13T03:38:17.056079+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:38:17.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:17 smithi183 bash[20824]: cluster 2024-02-13T03:38:17.056193+0000 mon.a (mon.0) 2903 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-13T03:38:17.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:17 smithi183 bash[20824]: audit 2024-02-13T03:38:17.058568+0000 mon.a (mon.0) 2904 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-124"}]: dispatch 2024-02-13T03:38:18.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:18 smithi161 bash[27879]: cluster 2024-02-13T03:38:18.049520+0000 mon.a (mon.0) 2905 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:38:18.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:18 smithi161 bash[27879]: audit 2024-02-13T03:38:18.058265+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-124"}]': finished 2024-02-13T03:38:18.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:18 smithi161 bash[27879]: cluster 2024-02-13T03:38:18.058361+0000 mon.a (mon.0) 2907 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-13T03:38:18.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:18 smithi161 bash[19769]: cluster 2024-02-13T03:38:18.049520+0000 mon.a (mon.0) 2905 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:38:18.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:18 smithi161 bash[19769]: audit 2024-02-13T03:38:18.058265+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-124"}]': finished 2024-02-13T03:38:18.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:18 smithi161 bash[19769]: cluster 2024-02-13T03:38:18.058361+0000 mon.a (mon.0) 2907 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-13T03:38:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:18 smithi183 bash[20824]: cluster 2024-02-13T03:38:18.049520+0000 mon.a (mon.0) 2905 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:38:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:18 smithi183 bash[20824]: audit 2024-02-13T03:38:18.058265+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-124"}]': finished 2024-02-13T03:38:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:18 smithi183 bash[20824]: cluster 2024-02-13T03:38:18.058361+0000 mon.a (mon.0) 2907 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-13T03:38:19.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:19 smithi183 bash[20824]: cluster 2024-02-13T03:38:17.545558+0000 mgr.y (mgr.24473) 668 : cluster [DBG] pgmap v1083: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:19.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:38:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:38:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:38:19.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:19 smithi161 bash[27879]: cluster 2024-02-13T03:38:17.545558+0000 mgr.y (mgr.24473) 668 : cluster [DBG] pgmap v1083: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:19.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:19 smithi161 bash[19769]: cluster 2024-02-13T03:38:17.545558+0000 mgr.y (mgr.24473) 668 : cluster [DBG] pgmap v1083: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:20.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:20 smithi161 bash[27879]: cluster 2024-02-13T03:38:19.060213+0000 mon.a (mon.0) 2908 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-13T03:38:20.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:20 smithi161 bash[27879]: cluster 2024-02-13T03:38:19.546359+0000 mgr.y (mgr.24473) 669 : cluster [DBG] pgmap v1086: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:20.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:20 smithi161 bash[27879]: audit 2024-02-13T03:38:19.796973+0000 mon.a (mon.0) 2909 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:38:20.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:20 smithi161 bash[27879]: cluster 2024-02-13T03:38:20.063772+0000 mon.a (mon.0) 2910 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-13T03:38:20.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:20 smithi161 bash[27879]: audit 2024-02-13T03:38:20.072149+0000 mon.a (mon.0) 2911 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:20.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:20 smithi161 bash[19769]: cluster 2024-02-13T03:38:19.060213+0000 mon.a (mon.0) 2908 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-13T03:38:20.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:20 smithi161 bash[19769]: cluster 2024-02-13T03:38:19.546359+0000 mgr.y (mgr.24473) 669 : cluster [DBG] pgmap v1086: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:20.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:20 smithi161 bash[19769]: audit 2024-02-13T03:38:19.796973+0000 mon.a (mon.0) 2909 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:38:20.929 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:20 smithi161 bash[19769]: cluster 2024-02-13T03:38:20.063772+0000 mon.a (mon.0) 2910 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-13T03:38:20.930 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:20 smithi161 bash[19769]: audit 2024-02-13T03:38:20.072149+0000 mon.a (mon.0) 2911 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:20.992 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:20 smithi183 bash[20824]: cluster 2024-02-13T03:38:19.060213+0000 mon.a (mon.0) 2908 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-13T03:38:20.992 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:20 smithi183 bash[20824]: cluster 2024-02-13T03:38:19.546359+0000 mgr.y (mgr.24473) 669 : cluster [DBG] pgmap v1086: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:20.992 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:20 smithi183 bash[20824]: audit 2024-02-13T03:38:19.796973+0000 mon.a (mon.0) 2909 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:38:20.992 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:20 smithi183 bash[20824]: cluster 2024-02-13T03:38:20.063772+0000 mon.a (mon.0) 2910 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-13T03:38:20.993 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:20 smithi183 bash[20824]: audit 2024-02-13T03:38:20.072149+0000 mon.a (mon.0) 2911 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:21.431 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:38:21] "GET /metrics HTTP/1.1" 200 187324 "" "Prometheus/2.33.4" 2024-02-13T03:38:21.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[27879]: audit 2024-02-13T03:38:20.651107+0000 mon.c (mon.2) 572 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:38:21.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[27879]: audit 2024-02-13T03:38:20.977763+0000 mon.c (mon.2) 573 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:38:21.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[27879]: audit 2024-02-13T03:38:20.979419+0000 mon.c (mon.2) 574 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:38:21.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[27879]: audit 2024-02-13T03:38:20.989858+0000 mon.a (mon.0) 2912 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:38:21.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[27879]: audit 2024-02-13T03:38:21.067266+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:21.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[27879]: cluster 2024-02-13T03:38:21.067509+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-13T03:38:21.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[27879]: audit 2024-02-13T03:38:21.070972+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[19769]: audit 2024-02-13T03:38:20.651107+0000 mon.c (mon.2) 572 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:38:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[19769]: audit 2024-02-13T03:38:20.977763+0000 mon.c (mon.2) 573 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:38:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[19769]: audit 2024-02-13T03:38:20.979419+0000 mon.c (mon.2) 574 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:38:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[19769]: audit 2024-02-13T03:38:20.989858+0000 mon.a (mon.0) 2912 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:38:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[19769]: audit 2024-02-13T03:38:21.067266+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[19769]: cluster 2024-02-13T03:38:21.067509+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-13T03:38:21.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:21 smithi161 bash[19769]: audit 2024-02-13T03:38:21.070972+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:21 smithi183 bash[20824]: audit 2024-02-13T03:38:20.651107+0000 mon.c (mon.2) 572 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:38:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:21 smithi183 bash[20824]: audit 2024-02-13T03:38:20.977763+0000 mon.c (mon.2) 573 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:38:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:21 smithi183 bash[20824]: audit 2024-02-13T03:38:20.979419+0000 mon.c (mon.2) 574 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:38:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:21 smithi183 bash[20824]: audit 2024-02-13T03:38:20.989858+0000 mon.a (mon.0) 2912 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:38:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:21 smithi183 bash[20824]: audit 2024-02-13T03:38:21.067266+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:21 smithi183 bash[20824]: cluster 2024-02-13T03:38:21.067509+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-13T03:38:21.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:21 smithi183 bash[20824]: audit 2024-02-13T03:38:21.070972+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:23.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:23 smithi161 bash[27879]: cluster 2024-02-13T03:38:21.547117+0000 mgr.y (mgr.24473) 670 : cluster [DBG] pgmap v1089: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:38:23.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:23 smithi161 bash[27879]: audit 2024-02-13T03:38:22.068636+0000 mon.a (mon.0) 2916 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:23.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:23 smithi161 bash[27879]: cluster 2024-02-13T03:38:22.068856+0000 mon.a (mon.0) 2917 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-13T03:38:23.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:23 smithi161 bash[27879]: audit 2024-02-13T03:38:22.070437+0000 mon.a (mon.0) 2918 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-126"}]: dispatch 2024-02-13T03:38:23.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:23 smithi161 bash[19769]: cluster 2024-02-13T03:38:21.547117+0000 mgr.y (mgr.24473) 670 : cluster [DBG] pgmap v1089: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:38:23.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:23 smithi161 bash[19769]: audit 2024-02-13T03:38:22.068636+0000 mon.a (mon.0) 2916 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:23 smithi161 bash[19769]: cluster 2024-02-13T03:38:22.068856+0000 mon.a (mon.0) 2917 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-13T03:38:23.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:23 smithi161 bash[19769]: audit 2024-02-13T03:38:22.070437+0000 mon.a (mon.0) 2918 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-126"}]: dispatch 2024-02-13T03:38:23.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:23 smithi183 bash[20824]: cluster 2024-02-13T03:38:21.547117+0000 mgr.y (mgr.24473) 670 : cluster [DBG] pgmap v1089: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:38:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:23 smithi183 bash[20824]: audit 2024-02-13T03:38:22.068636+0000 mon.a (mon.0) 2916 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:23 smithi183 bash[20824]: cluster 2024-02-13T03:38:22.068856+0000 mon.a (mon.0) 2917 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-13T03:38:23.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:23 smithi183 bash[20824]: audit 2024-02-13T03:38:22.070437+0000 mon.a (mon.0) 2918 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-126"}]: dispatch 2024-02-13T03:38:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:24 smithi161 bash[27879]: audit 2024-02-13T03:38:23.071819+0000 mon.a (mon.0) 2919 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-126"}]': finished 2024-02-13T03:38:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:24 smithi161 bash[27879]: cluster 2024-02-13T03:38:23.072079+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-13T03:38:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:24 smithi161 bash[27879]: audit 2024-02-13T03:38:23.073251+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-126", "mode": "writeback"}]: dispatch 2024-02-13T03:38:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:24 smithi161 bash[27879]: cluster 2024-02-13T03:38:24.065766+0000 mon.a (mon.0) 2922 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:38:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:24 smithi161 bash[27879]: audit 2024-02-13T03:38:24.074345+0000 mon.a (mon.0) 2923 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-126", "mode": "writeback"}]': finished 2024-02-13T03:38:24.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:24 smithi161 bash[27879]: cluster 2024-02-13T03:38:24.074616+0000 mon.a (mon.0) 2924 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-13T03:38:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:24 smithi161 bash[19769]: audit 2024-02-13T03:38:23.071819+0000 mon.a (mon.0) 2919 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-126"}]': finished 2024-02-13T03:38:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:24 smithi161 bash[19769]: cluster 2024-02-13T03:38:23.072079+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-13T03:38:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:24 smithi161 bash[19769]: audit 2024-02-13T03:38:23.073251+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-126", "mode": "writeback"}]: dispatch 2024-02-13T03:38:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:24 smithi161 bash[19769]: cluster 2024-02-13T03:38:24.065766+0000 mon.a (mon.0) 2922 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:38:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:24 smithi161 bash[19769]: audit 2024-02-13T03:38:24.074345+0000 mon.a (mon.0) 2923 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-126", "mode": "writeback"}]': finished 2024-02-13T03:38:24.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:24 smithi161 bash[19769]: cluster 2024-02-13T03:38:24.074616+0000 mon.a (mon.0) 2924 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-13T03:38:24.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:24 smithi183 bash[20824]: audit 2024-02-13T03:38:23.071819+0000 mon.a (mon.0) 2919 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-126"}]': finished 2024-02-13T03:38:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:24 smithi183 bash[20824]: cluster 2024-02-13T03:38:23.072079+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-13T03:38:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:24 smithi183 bash[20824]: audit 2024-02-13T03:38:23.073251+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-126", "mode": "writeback"}]: dispatch 2024-02-13T03:38:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:24 smithi183 bash[20824]: cluster 2024-02-13T03:38:24.065766+0000 mon.a (mon.0) 2922 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:38:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:24 smithi183 bash[20824]: audit 2024-02-13T03:38:24.074345+0000 mon.a (mon.0) 2923 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-126", "mode": "writeback"}]': finished 2024-02-13T03:38:24.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:24 smithi183 bash[20824]: cluster 2024-02-13T03:38:24.074616+0000 mon.a (mon.0) 2924 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-13T03:38:25.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:25 smithi161 bash[27879]: cluster 2024-02-13T03:38:23.547891+0000 mgr.y (mgr.24473) 671 : cluster [DBG] pgmap v1092: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:38:25.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:25 smithi161 bash[27879]: audit 2024-02-13T03:38:24.163127+0000 mon.a (mon.0) 2925 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:25.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:25 smithi161 bash[19769]: cluster 2024-02-13T03:38:23.547891+0000 mgr.y (mgr.24473) 671 : cluster [DBG] pgmap v1092: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:38:25.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:25 smithi161 bash[19769]: audit 2024-02-13T03:38:24.163127+0000 mon.a (mon.0) 2925 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:25 smithi183 bash[20824]: cluster 2024-02-13T03:38:23.547891+0000 mgr.y (mgr.24473) 671 : cluster [DBG] pgmap v1092: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T03:38:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:25 smithi183 bash[20824]: audit 2024-02-13T03:38:24.163127+0000 mon.a (mon.0) 2925 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:26.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:26 smithi183 bash[20824]: audit 2024-02-13T03:38:25.104118+0000 mon.a (mon.0) 2926 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:38:26.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:26 smithi183 bash[20824]: cluster 2024-02-13T03:38:25.104350+0000 mon.a (mon.0) 2927 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-13T03:38:26.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:26 smithi183 bash[20824]: audit 2024-02-13T03:38:25.104882+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-126"}]: dispatch 2024-02-13T03:38:26.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:26 smithi161 bash[27879]: audit 2024-02-13T03:38:25.104118+0000 mon.a (mon.0) 2926 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:38:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:26 smithi161 bash[27879]: cluster 2024-02-13T03:38:25.104350+0000 mon.a (mon.0) 2927 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-13T03:38:26.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:26 smithi161 bash[27879]: audit 2024-02-13T03:38:25.104882+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-126"}]: dispatch 2024-02-13T03:38:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:26 smithi161 bash[19769]: audit 2024-02-13T03:38:25.104118+0000 mon.a (mon.0) 2926 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:38:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:26 smithi161 bash[19769]: cluster 2024-02-13T03:38:25.104350+0000 mon.a (mon.0) 2927 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-13T03:38:26.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:26 smithi161 bash[19769]: audit 2024-02-13T03:38:25.104882+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-126"}]: dispatch 2024-02-13T03:38:26.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:38:26 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:38:27.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:27 smithi161 bash[27879]: cluster 2024-02-13T03:38:25.548715+0000 mgr.y (mgr.24473) 672 : cluster [DBG] pgmap v1095: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:38:27.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:27 smithi161 bash[27879]: cluster 2024-02-13T03:38:26.098898+0000 mon.a (mon.0) 2929 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:38:27.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:27 smithi161 bash[27879]: audit 2024-02-13T03:38:26.114452+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-126"}]': finished 2024-02-13T03:38:27.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:27 smithi161 bash[27879]: cluster 2024-02-13T03:38:26.114613+0000 mon.a (mon.0) 2931 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-13T03:38:27.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:27 smithi161 bash[27879]: audit 2024-02-13T03:38:26.408713+0000 mgr.y (mgr.24473) 673 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:27.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:27 smithi161 bash[19769]: cluster 2024-02-13T03:38:25.548715+0000 mgr.y (mgr.24473) 672 : cluster [DBG] pgmap v1095: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:38:27.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:27 smithi161 bash[19769]: cluster 2024-02-13T03:38:26.098898+0000 mon.a (mon.0) 2929 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:38:27.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:27 smithi161 bash[19769]: audit 2024-02-13T03:38:26.114452+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-126"}]': finished 2024-02-13T03:38:27.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:27 smithi161 bash[19769]: cluster 2024-02-13T03:38:26.114613+0000 mon.a (mon.0) 2931 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-13T03:38:27.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:27 smithi161 bash[19769]: audit 2024-02-13T03:38:26.408713+0000 mgr.y (mgr.24473) 673 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:27.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:27 smithi183 bash[20824]: cluster 2024-02-13T03:38:25.548715+0000 mgr.y (mgr.24473) 672 : cluster [DBG] pgmap v1095: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:38:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:27 smithi183 bash[20824]: cluster 2024-02-13T03:38:26.098898+0000 mon.a (mon.0) 2929 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:38:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:27 smithi183 bash[20824]: audit 2024-02-13T03:38:26.114452+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-126"}]': finished 2024-02-13T03:38:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:27 smithi183 bash[20824]: cluster 2024-02-13T03:38:26.114613+0000 mon.a (mon.0) 2931 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-13T03:38:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:27 smithi183 bash[20824]: audit 2024-02-13T03:38:26.408713+0000 mgr.y (mgr.24473) 673 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:28.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:28 smithi161 bash[27879]: cluster 2024-02-13T03:38:27.114932+0000 mon.a (mon.0) 2932 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-13T03:38:28.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:28 smithi161 bash[19769]: cluster 2024-02-13T03:38:27.114932+0000 mon.a (mon.0) 2932 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-13T03:38:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:28 smithi183 bash[20824]: cluster 2024-02-13T03:38:27.114932+0000 mon.a (mon.0) 2932 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-13T03:38:29.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:29 smithi161 bash[27879]: cluster 2024-02-13T03:38:27.549501+0000 mgr.y (mgr.24473) 674 : cluster [DBG] pgmap v1098: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-13T03:38:29.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:29 smithi161 bash[27879]: cluster 2024-02-13T03:38:28.128868+0000 mon.a (mon.0) 2933 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-13T03:38:29.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:29 smithi161 bash[27879]: audit 2024-02-13T03:38:28.130614+0000 mon.a (mon.0) 2934 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:29.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:29 smithi161 bash[19769]: cluster 2024-02-13T03:38:27.549501+0000 mgr.y (mgr.24473) 674 : cluster [DBG] pgmap v1098: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-13T03:38:29.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:29 smithi161 bash[19769]: cluster 2024-02-13T03:38:28.128868+0000 mon.a (mon.0) 2933 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-13T03:38:29.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:29 smithi161 bash[19769]: audit 2024-02-13T03:38:28.130614+0000 mon.a (mon.0) 2934 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:29 smithi183 bash[20824]: cluster 2024-02-13T03:38:27.549501+0000 mgr.y (mgr.24473) 674 : cluster [DBG] pgmap v1098: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-13T03:38:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:29 smithi183 bash[20824]: cluster 2024-02-13T03:38:28.128868+0000 mon.a (mon.0) 2933 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-13T03:38:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:29 smithi183 bash[20824]: audit 2024-02-13T03:38:28.130614+0000 mon.a (mon.0) 2934 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:29.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:38:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:38:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:38:30.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:30 smithi161 bash[27879]: audit 2024-02-13T03:38:29.134053+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:30.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:30 smithi161 bash[27879]: cluster 2024-02-13T03:38:29.134405+0000 mon.a (mon.0) 2936 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-13T03:38:30.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:30 smithi161 bash[27879]: audit 2024-02-13T03:38:29.138108+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:30.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:30 smithi161 bash[19769]: audit 2024-02-13T03:38:29.134053+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:30.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:30 smithi161 bash[19769]: cluster 2024-02-13T03:38:29.134405+0000 mon.a (mon.0) 2936 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-13T03:38:30.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:30 smithi161 bash[19769]: audit 2024-02-13T03:38:29.138108+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:30 smithi183 bash[20824]: audit 2024-02-13T03:38:29.134053+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:30 smithi183 bash[20824]: cluster 2024-02-13T03:38:29.134405+0000 mon.a (mon.0) 2936 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-13T03:38:30.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:30 smithi183 bash[20824]: audit 2024-02-13T03:38:29.138108+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:31.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:38:31] "GET /metrics HTTP/1.1" 200 187324 "" "Prometheus/2.33.4" 2024-02-13T03:38:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[27879]: cluster 2024-02-13T03:38:29.550498+0000 mgr.y (mgr.24473) 675 : cluster [DBG] pgmap v1101: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-13T03:38:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[27879]: audit 2024-02-13T03:38:30.139827+0000 mon.a (mon.0) 2938 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[27879]: cluster 2024-02-13T03:38:30.140169+0000 mon.a (mon.0) 2939 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-13T03:38:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[27879]: audit 2024-02-13T03:38:30.141940+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:38:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[27879]: audit 2024-02-13T03:38:31.140290+0000 mon.a (mon.0) 2941 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:38:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[27879]: cluster 2024-02-13T03:38:31.140415+0000 mon.a (mon.0) 2942 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-13T03:38:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[27879]: audit 2024-02-13T03:38:31.142194+0000 mon.a (mon.0) 2943 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:38:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[19769]: cluster 2024-02-13T03:38:29.550498+0000 mgr.y (mgr.24473) 675 : cluster [DBG] pgmap v1101: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-13T03:38:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[19769]: audit 2024-02-13T03:38:30.139827+0000 mon.a (mon.0) 2938 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[19769]: cluster 2024-02-13T03:38:30.140169+0000 mon.a (mon.0) 2939 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-13T03:38:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[19769]: audit 2024-02-13T03:38:30.141940+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:38:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[19769]: audit 2024-02-13T03:38:31.140290+0000 mon.a (mon.0) 2941 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:38:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[19769]: cluster 2024-02-13T03:38:31.140415+0000 mon.a (mon.0) 2942 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-13T03:38:31.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:31 smithi161 bash[19769]: audit 2024-02-13T03:38:31.142194+0000 mon.a (mon.0) 2943 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:38:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:31 smithi183 bash[20824]: cluster 2024-02-13T03:38:29.550498+0000 mgr.y (mgr.24473) 675 : cluster [DBG] pgmap v1101: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-13T03:38:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:31 smithi183 bash[20824]: audit 2024-02-13T03:38:30.139827+0000 mon.a (mon.0) 2938 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:31 smithi183 bash[20824]: cluster 2024-02-13T03:38:30.140169+0000 mon.a (mon.0) 2939 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-13T03:38:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:31 smithi183 bash[20824]: audit 2024-02-13T03:38:30.141940+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:38:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:31 smithi183 bash[20824]: audit 2024-02-13T03:38:31.140290+0000 mon.a (mon.0) 2941 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:38:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:31 smithi183 bash[20824]: cluster 2024-02-13T03:38:31.140415+0000 mon.a (mon.0) 2942 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-13T03:38:31.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:31 smithi183 bash[20824]: audit 2024-02-13T03:38:31.142194+0000 mon.a (mon.0) 2943 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:38:33.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:33 smithi161 bash[27879]: cluster 2024-02-13T03:38:31.551476+0000 mgr.y (mgr.24473) 676 : cluster [DBG] pgmap v1104: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:33.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:33 smithi161 bash[27879]: audit 2024-02-13T03:38:32.142090+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:38:33.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:33 smithi161 bash[27879]: cluster 2024-02-13T03:38:32.142187+0000 mon.a (mon.0) 2945 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-13T03:38:33.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:33 smithi161 bash[27879]: audit 2024-02-13T03:38:32.143370+0000 mon.a (mon.0) 2946 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:38:33.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:33 smithi161 bash[19769]: cluster 2024-02-13T03:38:31.551476+0000 mgr.y (mgr.24473) 676 : cluster [DBG] pgmap v1104: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:33.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:33 smithi161 bash[19769]: audit 2024-02-13T03:38:32.142090+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:38:33.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:33 smithi161 bash[19769]: cluster 2024-02-13T03:38:32.142187+0000 mon.a (mon.0) 2945 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-13T03:38:33.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:33 smithi161 bash[19769]: audit 2024-02-13T03:38:32.143370+0000 mon.a (mon.0) 2946 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:38:33.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:33 smithi183 bash[20824]: cluster 2024-02-13T03:38:31.551476+0000 mgr.y (mgr.24473) 676 : cluster [DBG] pgmap v1104: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:33 smithi183 bash[20824]: audit 2024-02-13T03:38:32.142090+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:38:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:33 smithi183 bash[20824]: cluster 2024-02-13T03:38:32.142187+0000 mon.a (mon.0) 2945 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-13T03:38:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:33 smithi183 bash[20824]: audit 2024-02-13T03:38:32.143370+0000 mon.a (mon.0) 2946 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-13T03:38:34.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:34 smithi161 bash[19769]: audit 2024-02-13T03:38:33.144941+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-13T03:38:34.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:34 smithi161 bash[19769]: cluster 2024-02-13T03:38:33.145031+0000 mon.a (mon.0) 2948 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-13T03:38:34.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:34 smithi161 bash[27879]: audit 2024-02-13T03:38:33.144941+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-13T03:38:34.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:34 smithi161 bash[27879]: cluster 2024-02-13T03:38:33.145031+0000 mon.a (mon.0) 2948 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-13T03:38:34.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:34 smithi183 bash[20824]: audit 2024-02-13T03:38:33.144941+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-13T03:38:34.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:34 smithi183 bash[20824]: cluster 2024-02-13T03:38:33.145031+0000 mon.a (mon.0) 2948 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-13T03:38:35.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:35 smithi161 bash[19769]: cluster 2024-02-13T03:38:33.552341+0000 mgr.y (mgr.24473) 677 : cluster [DBG] pgmap v1107: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:35.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:35 smithi161 bash[19769]: audit 2024-02-13T03:38:34.173455+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:35.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:35 smithi161 bash[19769]: audit 2024-02-13T03:38:34.174148+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-128"}]: dispatch 2024-02-13T03:38:35.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:35 smithi161 bash[27879]: cluster 2024-02-13T03:38:33.552341+0000 mgr.y (mgr.24473) 677 : cluster [DBG] pgmap v1107: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:35.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:35 smithi161 bash[27879]: audit 2024-02-13T03:38:34.173455+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:35.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:35 smithi161 bash[27879]: audit 2024-02-13T03:38:34.174148+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-128"}]: dispatch 2024-02-13T03:38:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:35 smithi183 bash[20824]: cluster 2024-02-13T03:38:33.552341+0000 mgr.y (mgr.24473) 677 : cluster [DBG] pgmap v1107: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:35 smithi183 bash[20824]: audit 2024-02-13T03:38:34.173455+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:35 smithi183 bash[20824]: audit 2024-02-13T03:38:34.174148+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-128"}]: dispatch 2024-02-13T03:38:36.187 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-02-13T03:38:36.188 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12129 ms) 2024-02-13T03:38:36.188 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-02-13T03:38:36.191 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-02-13T03:38:36.191 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: 00000010 69 73 |is| 2024-02-13T03:38:36.191 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: 00000012 2024-02-13T03:38:36.192 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12102 ms) 2024-02-13T03:38:36.192 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-02-13T03:38:36.192 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: require_osd_release = pacific 2024-02-13T03:38:36.192 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6037 ms) 2024-02-13T03:38:36.192 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [----------] 43 tests from LibRadosTwoPoolsPP (497521 ms total) 2024-02-13T03:38:36.192 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: 2024-02-13T03:38:36.192 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-02-13T03:38:36.192 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-02-13T03:38:36.192 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1354 ms) 2024-02-13T03:38:36.192 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-02-13T03:38:36.192 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (10769 ms) 2024-02-13T03:38:36.192 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-02-13T03:38:36.193 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18150 ms) 2024-02-13T03:38:36.193 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-02-13T03:38:36.193 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-02-13T03:38:36.193 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30276 ms total) 2024-02-13T03:38:36.193 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: 2024-02-13T03:38:36.193 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-02-13T03:38:36.193 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-02-13T03:38:36.193 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7063 ms) 2024-02-13T03:38:36.193 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-02-13T03:38:36.193 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8026 ms) 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: waiting for scrub... 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: done waiting 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24558 ms) 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (10000 ms) 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8002 ms) 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8106 ms) 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10122 ms) 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-02-13T03:38:36.194 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7750 ms) 2024-02-13T03:38:36.195 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-02-13T03:38:36.195 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (13101 ms) 2024-02-13T03:38:36.195 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-02-13T03:38:36.195 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8082 ms) 2024-02-13T03:38:36.195 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-02-13T03:38:36.195 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13119 ms) 2024-02-13T03:38:36.195 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-02-13T03:38:36.195 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7701 ms) 2024-02-13T03:38:36.195 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-02-13T03:38:36.195 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8057 ms) 2024-02-13T03:38:36.195 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-02-13T03:38:36.195 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-13T03:38:36.195 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: ok, hit_set contains 248:602f83fe:::foo:head 2024-02-13T03:38:36.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:36 smithi161 bash[27879]: audit 2024-02-13T03:38:35.165816+0000 mon.a (mon.0) 2951 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-128"}]': finished 2024-02-13T03:38:36.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:36 smithi161 bash[27879]: cluster 2024-02-13T03:38:35.165909+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-13T03:38:36.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:36 smithi161 bash[19769]: audit 2024-02-13T03:38:35.165816+0000 mon.a (mon.0) 2951 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-128"}]': finished 2024-02-13T03:38:36.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:36 smithi161 bash[19769]: cluster 2024-02-13T03:38:35.165909+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-13T03:38:36.486 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:38:36 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:38:36.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:36 smithi183 bash[20824]: audit 2024-02-13T03:38:35.165816+0000 mon.a (mon.0) 2951 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-128"}]': finished 2024-02-13T03:38:36.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:36 smithi183 bash[20824]: cluster 2024-02-13T03:38:35.165909+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-13T03:38:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:37 smithi183 bash[20824]: cluster 2024-02-13T03:38:35.553262+0000 mgr.y (mgr.24473) 678 : cluster [DBG] pgmap v1109: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:38:37.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:37 smithi183 bash[20824]: cluster 2024-02-13T03:38:36.186517+0000 mon.a (mon.0) 2953 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-13T03:38:37.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:37 smithi183 bash[20824]: audit 2024-02-13T03:38:36.416710+0000 mgr.y (mgr.24473) 679 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:37.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:37 smithi161 bash[27879]: cluster 2024-02-13T03:38:35.553262+0000 mgr.y (mgr.24473) 678 : cluster [DBG] pgmap v1109: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:38:37.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:37 smithi161 bash[27879]: cluster 2024-02-13T03:38:36.186517+0000 mon.a (mon.0) 2953 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-13T03:38:37.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:37 smithi161 bash[27879]: audit 2024-02-13T03:38:36.416710+0000 mgr.y (mgr.24473) 679 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:37.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:37 smithi161 bash[19769]: cluster 2024-02-13T03:38:35.553262+0000 mgr.y (mgr.24473) 678 : cluster [DBG] pgmap v1109: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:38:37.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:37 smithi161 bash[19769]: cluster 2024-02-13T03:38:36.186517+0000 mon.a (mon.0) 2953 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-13T03:38:37.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:37 smithi161 bash[19769]: audit 2024-02-13T03:38:36.416710+0000 mgr.y (mgr.24473) 679 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:38.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:38 smithi183 bash[20824]: cluster 2024-02-13T03:38:37.197180+0000 mon.a (mon.0) 2954 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-13T03:38:38.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:38 smithi183 bash[20824]: audit 2024-02-13T03:38:37.204719+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:38.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:38 smithi161 bash[27879]: cluster 2024-02-13T03:38:37.197180+0000 mon.a (mon.0) 2954 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-13T03:38:38.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:38 smithi161 bash[27879]: audit 2024-02-13T03:38:37.204719+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:38.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:38 smithi161 bash[19769]: cluster 2024-02-13T03:38:37.197180+0000 mon.a (mon.0) 2954 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-13T03:38:38.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:38 smithi161 bash[19769]: audit 2024-02-13T03:38:37.204719+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:39 smithi183 bash[20824]: cluster 2024-02-13T03:38:37.554253+0000 mgr.y (mgr.24473) 680 : cluster [DBG] pgmap v1112: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-13T03:38:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:39 smithi183 bash[20824]: audit 2024-02-13T03:38:38.201462+0000 mon.a (mon.0) 2956 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:39 smithi183 bash[20824]: cluster 2024-02-13T03:38:38.201687+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-13T03:38:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:39 smithi183 bash[20824]: audit 2024-02-13T03:38:38.204235+0000 mon.a (mon.0) 2958 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:39.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:38:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:38:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:38:39.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:39 smithi161 bash[27879]: cluster 2024-02-13T03:38:37.554253+0000 mgr.y (mgr.24473) 680 : cluster [DBG] pgmap v1112: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-13T03:38:39.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:39 smithi161 bash[27879]: audit 2024-02-13T03:38:38.201462+0000 mon.a (mon.0) 2956 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:39.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:39 smithi161 bash[27879]: cluster 2024-02-13T03:38:38.201687+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-13T03:38:39.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:39 smithi161 bash[27879]: audit 2024-02-13T03:38:38.204235+0000 mon.a (mon.0) 2958 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:39.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:39 smithi161 bash[19769]: cluster 2024-02-13T03:38:37.554253+0000 mgr.y (mgr.24473) 680 : cluster [DBG] pgmap v1112: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-13T03:38:39.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:39 smithi161 bash[19769]: audit 2024-02-13T03:38:38.201462+0000 mon.a (mon.0) 2956 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:38:39.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:39 smithi161 bash[19769]: cluster 2024-02-13T03:38:38.201687+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-13T03:38:39.680 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:39 smithi161 bash[19769]: audit 2024-02-13T03:38:38.204235+0000 mon.a (mon.0) 2958 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:38:40.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:40 smithi183 bash[20824]: audit 2024-02-13T03:38:39.205541+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:40 smithi183 bash[20824]: cluster 2024-02-13T03:38:39.205649+0000 mon.a (mon.0) 2960 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-13T03:38:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:40 smithi183 bash[20824]: audit 2024-02-13T03:38:39.208955+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-13T03:38:40.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:40 smithi183 bash[20824]: audit 2024-02-13T03:38:40.207843+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_count","val": "3"}]': finished 2024-02-13T03:38:40.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:40 smithi161 bash[27879]: audit 2024-02-13T03:38:39.205541+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:40.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:40 smithi161 bash[27879]: cluster 2024-02-13T03:38:39.205649+0000 mon.a (mon.0) 2960 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-13T03:38:40.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:40 smithi161 bash[27879]: audit 2024-02-13T03:38:39.208955+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-13T03:38:40.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:40 smithi161 bash[27879]: audit 2024-02-13T03:38:40.207843+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_count","val": "3"}]': finished 2024-02-13T03:38:40.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:40 smithi161 bash[19769]: audit 2024-02-13T03:38:39.205541+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:38:40.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:40 smithi161 bash[19769]: cluster 2024-02-13T03:38:39.205649+0000 mon.a (mon.0) 2960 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-13T03:38:40.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:40 smithi161 bash[19769]: audit 2024-02-13T03:38:39.208955+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-13T03:38:40.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:40 smithi161 bash[19769]: audit 2024-02-13T03:38:40.207843+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_count","val": "3"}]': finished 2024-02-13T03:38:41.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:38:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:38:41] "GET /metrics HTTP/1.1" 200 189888 "" "Prometheus/2.33.4" 2024-02-13T03:38:41.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:41 smithi161 bash[27879]: cluster 2024-02-13T03:38:39.555165+0000 mgr.y (mgr.24473) 681 : cluster [DBG] pgmap v1115: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:41.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:41 smithi161 bash[27879]: cluster 2024-02-13T03:38:40.207974+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-13T03:38:41.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:41 smithi161 bash[27879]: audit 2024-02-13T03:38:40.210771+0000 mon.a (mon.0) 2964 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-13T03:38:41.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:41 smithi161 bash[19769]: cluster 2024-02-13T03:38:39.555165+0000 mgr.y (mgr.24473) 681 : cluster [DBG] pgmap v1115: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:41.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:41 smithi161 bash[19769]: cluster 2024-02-13T03:38:40.207974+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-13T03:38:41.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:41 smithi161 bash[19769]: audit 2024-02-13T03:38:40.210771+0000 mon.a (mon.0) 2964 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-13T03:38:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:41 smithi183 bash[20824]: cluster 2024-02-13T03:38:39.555165+0000 mgr.y (mgr.24473) 681 : cluster [DBG] pgmap v1115: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:41 smithi183 bash[20824]: cluster 2024-02-13T03:38:40.207974+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-13T03:38:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:41 smithi183 bash[20824]: audit 2024-02-13T03:38:40.210771+0000 mon.a (mon.0) 2964 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-13T03:38:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:42 smithi183 bash[20824]: audit 2024-02-13T03:38:41.210838+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_period","val": "3"}]': finished 2024-02-13T03:38:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:42 smithi183 bash[20824]: cluster 2024-02-13T03:38:41.211336+0000 mon.a (mon.0) 2966 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-13T03:38:42.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:42 smithi183 bash[20824]: audit 2024-02-13T03:38:41.213442+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:38:42.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:42 smithi183 bash[20824]: audit 2024-02-13T03:38:42.212012+0000 mon.a (mon.0) 2968 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:38:42.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:42 smithi183 bash[20824]: cluster 2024-02-13T03:38:42.212103+0000 mon.a (mon.0) 2969 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-13T03:38:42.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:42 smithi183 bash[20824]: audit 2024-02-13T03:38:42.212987+0000 mon.a (mon.0) 2970 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-13T03:38:42.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:42 smithi161 bash[27879]: audit 2024-02-13T03:38:41.210838+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_period","val": "3"}]': finished 2024-02-13T03:38:42.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:42 smithi161 bash[27879]: cluster 2024-02-13T03:38:41.211336+0000 mon.a (mon.0) 2966 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-13T03:38:42.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:42 smithi161 bash[27879]: audit 2024-02-13T03:38:41.213442+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:38:42.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:42 smithi161 bash[27879]: audit 2024-02-13T03:38:42.212012+0000 mon.a (mon.0) 2968 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:38:42.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:42 smithi161 bash[27879]: cluster 2024-02-13T03:38:42.212103+0000 mon.a (mon.0) 2969 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-13T03:38:42.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:42 smithi161 bash[27879]: audit 2024-02-13T03:38:42.212987+0000 mon.a (mon.0) 2970 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-13T03:38:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:42 smithi161 bash[19769]: audit 2024-02-13T03:38:41.210838+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_period","val": "3"}]': finished 2024-02-13T03:38:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:42 smithi161 bash[19769]: cluster 2024-02-13T03:38:41.211336+0000 mon.a (mon.0) 2966 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-13T03:38:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:42 smithi161 bash[19769]: audit 2024-02-13T03:38:41.213442+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:38:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:42 smithi161 bash[19769]: audit 2024-02-13T03:38:42.212012+0000 mon.a (mon.0) 2968 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:38:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:42 smithi161 bash[19769]: cluster 2024-02-13T03:38:42.212103+0000 mon.a (mon.0) 2969 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-13T03:38:42.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:42 smithi161 bash[19769]: audit 2024-02-13T03:38:42.212987+0000 mon.a (mon.0) 2970 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-13T03:38:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:43 smithi183 bash[20824]: cluster 2024-02-13T03:38:41.556005+0000 mgr.y (mgr.24473) 682 : cluster [DBG] pgmap v1118: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:43.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:43 smithi183 bash[20824]: audit 2024-02-13T03:38:43.216390+0000 mon.a (mon.0) 2971 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-13T03:38:43.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:43 smithi183 bash[20824]: cluster 2024-02-13T03:38:43.216672+0000 mon.a (mon.0) 2972 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-13T03:38:43.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:43 smithi161 bash[27879]: cluster 2024-02-13T03:38:41.556005+0000 mgr.y (mgr.24473) 682 : cluster [DBG] pgmap v1118: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:43.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:43 smithi161 bash[27879]: audit 2024-02-13T03:38:43.216390+0000 mon.a (mon.0) 2971 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-13T03:38:43.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:43 smithi161 bash[27879]: cluster 2024-02-13T03:38:43.216672+0000 mon.a (mon.0) 2972 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-13T03:38:43.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:43 smithi161 bash[19769]: cluster 2024-02-13T03:38:41.556005+0000 mgr.y (mgr.24473) 682 : cluster [DBG] pgmap v1118: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:43.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:43 smithi161 bash[19769]: audit 2024-02-13T03:38:43.216390+0000 mon.a (mon.0) 2971 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-13T03:38:43.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:43 smithi161 bash[19769]: cluster 2024-02-13T03:38:43.216672+0000 mon.a (mon.0) 2972 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-13T03:38:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:45 smithi183 bash[20824]: cluster 2024-02-13T03:38:43.556852+0000 mgr.y (mgr.24473) 683 : cluster [DBG] pgmap v1121: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:45.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:45 smithi161 bash[27879]: cluster 2024-02-13T03:38:43.556852+0000 mgr.y (mgr.24473) 683 : cluster [DBG] pgmap v1121: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:45.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:45 smithi161 bash[19769]: cluster 2024-02-13T03:38:43.556852+0000 mgr.y (mgr.24473) 683 : cluster [DBG] pgmap v1121: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:46.736 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:38:46 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:38:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:47 smithi183 bash[20824]: cluster 2024-02-13T03:38:45.557829+0000 mgr.y (mgr.24473) 684 : cluster [DBG] pgmap v1122: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 956 B/s rd, 0 op/s 2024-02-13T03:38:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:47 smithi161 bash[27879]: cluster 2024-02-13T03:38:45.557829+0000 mgr.y (mgr.24473) 684 : cluster [DBG] pgmap v1122: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 956 B/s rd, 0 op/s 2024-02-13T03:38:47.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:47 smithi161 bash[19769]: cluster 2024-02-13T03:38:45.557829+0000 mgr.y (mgr.24473) 684 : cluster [DBG] pgmap v1122: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 956 B/s rd, 0 op/s 2024-02-13T03:38:48.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:48 smithi161 bash[27879]: audit 2024-02-13T03:38:46.428103+0000 mgr.y (mgr.24473) 685 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:48.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:48 smithi161 bash[19769]: audit 2024-02-13T03:38:46.428103+0000 mgr.y (mgr.24473) 685 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:48.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:48 smithi183 bash[20824]: audit 2024-02-13T03:38:46.428103+0000 mgr.y (mgr.24473) 685 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:49.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:49 smithi161 bash[27879]: cluster 2024-02-13T03:38:47.559941+0000 mgr.y (mgr.24473) 686 : cluster [DBG] pgmap v1123: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 806 B/s rd, 3.8 KiB/s wr, 1 op/s 2024-02-13T03:38:49.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:49 smithi161 bash[19769]: cluster 2024-02-13T03:38:47.559941+0000 mgr.y (mgr.24473) 686 : cluster [DBG] pgmap v1123: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 806 B/s rd, 3.8 KiB/s wr, 1 op/s 2024-02-13T03:38:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:49 smithi183 bash[20824]: cluster 2024-02-13T03:38:47.559941+0000 mgr.y (mgr.24473) 686 : cluster [DBG] pgmap v1123: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 806 B/s rd, 3.8 KiB/s wr, 1 op/s 2024-02-13T03:38:49.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:38:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:38:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:38:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:51 smithi161 bash[27879]: cluster 2024-02-13T03:38:49.560809+0000 mgr.y (mgr.24473) 687 : cluster [DBG] pgmap v1124: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 3.0 KiB/s wr, 0 op/s 2024-02-13T03:38:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:38:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:38:51] "GET /metrics HTTP/1.1" 200 190043 "" "Prometheus/2.33.4" 2024-02-13T03:38:51.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:51 smithi161 bash[19769]: cluster 2024-02-13T03:38:49.560809+0000 mgr.y (mgr.24473) 687 : cluster [DBG] pgmap v1124: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 3.0 KiB/s wr, 0 op/s 2024-02-13T03:38:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:51 smithi183 bash[20824]: cluster 2024-02-13T03:38:49.560809+0000 mgr.y (mgr.24473) 687 : cluster [DBG] pgmap v1124: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 3.0 KiB/s wr, 0 op/s 2024-02-13T03:38:53.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:53 smithi161 bash[19769]: cluster 2024-02-13T03:38:51.562937+0000 mgr.y (mgr.24473) 688 : cluster [DBG] pgmap v1125: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 7.1 KiB/s wr, 2 op/s 2024-02-13T03:38:53.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:53 smithi161 bash[27879]: cluster 2024-02-13T03:38:51.562937+0000 mgr.y (mgr.24473) 688 : cluster [DBG] pgmap v1125: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 7.1 KiB/s wr, 2 op/s 2024-02-13T03:38:53.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:53 smithi183 bash[20824]: cluster 2024-02-13T03:38:51.562937+0000 mgr.y (mgr.24473) 688 : cluster [DBG] pgmap v1125: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 7.1 KiB/s wr, 2 op/s 2024-02-13T03:38:55.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:55 smithi161 bash[27879]: cluster 2024-02-13T03:38:53.563893+0000 mgr.y (mgr.24473) 689 : cluster [DBG] pgmap v1126: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 989 B/s rd, 6.4 KiB/s wr, 1 op/s 2024-02-13T03:38:55.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:55 smithi161 bash[19769]: cluster 2024-02-13T03:38:53.563893+0000 mgr.y (mgr.24473) 689 : cluster [DBG] pgmap v1126: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 989 B/s rd, 6.4 KiB/s wr, 1 op/s 2024-02-13T03:38:55.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:55 smithi183 bash[20824]: cluster 2024-02-13T03:38:53.563893+0000 mgr.y (mgr.24473) 689 : cluster [DBG] pgmap v1126: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 989 B/s rd, 6.4 KiB/s wr, 1 op/s 2024-02-13T03:38:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:56 smithi161 bash[27879]: audit 2024-02-13T03:38:55.290316+0000 mon.a (mon.0) 2973 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:56.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:56 smithi161 bash[27879]: audit 2024-02-13T03:38:55.291117+0000 mon.a (mon.0) 2974 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-130"}]: dispatch 2024-02-13T03:38:56.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:56 smithi161 bash[19769]: audit 2024-02-13T03:38:55.290316+0000 mon.a (mon.0) 2973 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:56.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:56 smithi161 bash[19769]: audit 2024-02-13T03:38:55.291117+0000 mon.a (mon.0) 2974 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-130"}]: dispatch 2024-02-13T03:38:56.737 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:38:56 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:38:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:56 smithi183 bash[20824]: audit 2024-02-13T03:38:55.290316+0000 mon.a (mon.0) 2973 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:38:56.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:56 smithi183 bash[20824]: audit 2024-02-13T03:38:55.291117+0000 mon.a (mon.0) 2974 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-130"}]: dispatch 2024-02-13T03:38:57.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:57 smithi161 bash[27879]: cluster 2024-02-13T03:38:55.564939+0000 mgr.y (mgr.24473) 690 : cluster [DBG] pgmap v1127: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 5.5 KiB/s wr, 1 op/s 2024-02-13T03:38:57.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:57 smithi161 bash[27879]: audit 2024-02-13T03:38:56.270930+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-130"}]': finished 2024-02-13T03:38:57.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:57 smithi161 bash[27879]: cluster 2024-02-13T03:38:56.271058+0000 mon.a (mon.0) 2976 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-13T03:38:57.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:57 smithi161 bash[19769]: cluster 2024-02-13T03:38:55.564939+0000 mgr.y (mgr.24473) 690 : cluster [DBG] pgmap v1127: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 5.5 KiB/s wr, 1 op/s 2024-02-13T03:38:57.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:57 smithi161 bash[19769]: audit 2024-02-13T03:38:56.270930+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-130"}]': finished 2024-02-13T03:38:57.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:57 smithi161 bash[19769]: cluster 2024-02-13T03:38:56.271058+0000 mon.a (mon.0) 2976 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-13T03:38:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:57 smithi183 bash[20824]: cluster 2024-02-13T03:38:55.564939+0000 mgr.y (mgr.24473) 690 : cluster [DBG] pgmap v1127: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 5.5 KiB/s wr, 1 op/s 2024-02-13T03:38:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:57 smithi183 bash[20824]: audit 2024-02-13T03:38:56.270930+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-130"}]': finished 2024-02-13T03:38:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:57 smithi183 bash[20824]: cluster 2024-02-13T03:38:56.271058+0000 mon.a (mon.0) 2976 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-13T03:38:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:58 smithi161 bash[27879]: audit 2024-02-13T03:38:56.438046+0000 mgr.y (mgr.24473) 691 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:58.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:58 smithi161 bash[27879]: cluster 2024-02-13T03:38:57.292735+0000 mon.a (mon.0) 2977 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-13T03:38:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:58 smithi161 bash[19769]: audit 2024-02-13T03:38:56.438046+0000 mgr.y (mgr.24473) 691 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:58 smithi161 bash[19769]: cluster 2024-02-13T03:38:57.292735+0000 mon.a (mon.0) 2977 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-13T03:38:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:58 smithi183 bash[20824]: audit 2024-02-13T03:38:56.438046+0000 mgr.y (mgr.24473) 691 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:38:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:58 smithi183 bash[20824]: cluster 2024-02-13T03:38:57.292735+0000 mon.a (mon.0) 2977 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-13T03:38:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:59 smithi161 bash[27879]: cluster 2024-02-13T03:38:57.565653+0000 mgr.y (mgr.24473) 692 : cluster [DBG] pgmap v1130: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:59 smithi161 bash[27879]: cluster 2024-02-13T03:38:58.311002+0000 mon.a (mon.0) 2978 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-13T03:38:59.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:38:59 smithi161 bash[27879]: audit 2024-02-13T03:38:58.312540+0000 mon.a (mon.0) 2979 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:59.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:59 smithi161 bash[19769]: cluster 2024-02-13T03:38:57.565653+0000 mgr.y (mgr.24473) 692 : cluster [DBG] pgmap v1130: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:59.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:59 smithi161 bash[19769]: cluster 2024-02-13T03:38:58.311002+0000 mon.a (mon.0) 2978 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-13T03:38:59.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:38:59 smithi161 bash[19769]: audit 2024-02-13T03:38:58.312540+0000 mon.a (mon.0) 2979 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:59 smithi183 bash[20824]: cluster 2024-02-13T03:38:57.565653+0000 mgr.y (mgr.24473) 692 : cluster [DBG] pgmap v1130: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:38:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:59 smithi183 bash[20824]: cluster 2024-02-13T03:38:58.311002+0000 mon.a (mon.0) 2978 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-13T03:38:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:38:59 smithi183 bash[20824]: audit 2024-02-13T03:38:58.312540+0000 mon.a (mon.0) 2979 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:38:59.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:38:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:38:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:39:00.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:00 smithi161 bash[27879]: audit 2024-02-13T03:38:59.313153+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:00.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:00 smithi161 bash[27879]: cluster 2024-02-13T03:38:59.314073+0000 mon.a (mon.0) 2981 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-13T03:39:00.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:00 smithi161 bash[27879]: audit 2024-02-13T03:38:59.365139+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:39:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:00 smithi161 bash[19769]: audit 2024-02-13T03:38:59.313153+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:00 smithi161 bash[19769]: cluster 2024-02-13T03:38:59.314073+0000 mon.a (mon.0) 2981 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-13T03:39:00.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:00 smithi161 bash[19769]: audit 2024-02-13T03:38:59.365139+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:39:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:00 smithi183 bash[20824]: audit 2024-02-13T03:38:59.313153+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:00 smithi183 bash[20824]: cluster 2024-02-13T03:38:59.314073+0000 mon.a (mon.0) 2981 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-13T03:39:00.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:00 smithi183 bash[20824]: audit 2024-02-13T03:38:59.365139+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:39:01.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:39:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:39:01] "GET /metrics HTTP/1.1" 200 190043 "" "Prometheus/2.33.4" 2024-02-13T03:39:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:01 smithi161 bash[19769]: cluster 2024-02-13T03:38:59.566460+0000 mgr.y (mgr.24473) 693 : cluster [DBG] pgmap v1133: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:01 smithi161 bash[19769]: audit 2024-02-13T03:39:00.337824+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:39:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:01 smithi161 bash[19769]: cluster 2024-02-13T03:39:00.337956+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-13T03:39:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:01 smithi161 bash[19769]: audit 2024-02-13T03:39:00.339759+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-132"}]: dispatch 2024-02-13T03:39:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:01 smithi161 bash[27879]: cluster 2024-02-13T03:38:59.566460+0000 mgr.y (mgr.24473) 693 : cluster [DBG] pgmap v1133: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:01 smithi161 bash[27879]: audit 2024-02-13T03:39:00.337824+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:39:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:01 smithi161 bash[27879]: cluster 2024-02-13T03:39:00.337956+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-13T03:39:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:01 smithi161 bash[27879]: audit 2024-02-13T03:39:00.339759+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-132"}]: dispatch 2024-02-13T03:39:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:01 smithi183 bash[20824]: cluster 2024-02-13T03:38:59.566460+0000 mgr.y (mgr.24473) 693 : cluster [DBG] pgmap v1133: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:01 smithi183 bash[20824]: audit 2024-02-13T03:39:00.337824+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:39:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:01 smithi183 bash[20824]: cluster 2024-02-13T03:39:00.337956+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-13T03:39:01.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:01 smithi183 bash[20824]: audit 2024-02-13T03:39:00.339759+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-132"}]: dispatch 2024-02-13T03:39:02.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:02 smithi161 bash[27879]: audit 2024-02-13T03:39:01.339839+0000 mon.a (mon.0) 2986 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-132"}]': finished 2024-02-13T03:39:02.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:02 smithi161 bash[27879]: cluster 2024-02-13T03:39:01.339924+0000 mon.a (mon.0) 2987 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-13T03:39:02.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:02 smithi161 bash[27879]: audit 2024-02-13T03:39:01.341306+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-132", "mode": "writeback"}]: dispatch 2024-02-13T03:39:02.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:02 smithi161 bash[27879]: cluster 2024-02-13T03:39:02.334896+0000 mon.a (mon.0) 2989 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:39:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:02 smithi161 bash[19769]: audit 2024-02-13T03:39:01.339839+0000 mon.a (mon.0) 2986 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-132"}]': finished 2024-02-13T03:39:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:02 smithi161 bash[19769]: cluster 2024-02-13T03:39:01.339924+0000 mon.a (mon.0) 2987 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-13T03:39:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:02 smithi161 bash[19769]: audit 2024-02-13T03:39:01.341306+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-132", "mode": "writeback"}]: dispatch 2024-02-13T03:39:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:02 smithi161 bash[19769]: cluster 2024-02-13T03:39:02.334896+0000 mon.a (mon.0) 2989 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:39:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:02 smithi183 bash[20824]: audit 2024-02-13T03:39:01.339839+0000 mon.a (mon.0) 2986 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-132"}]': finished 2024-02-13T03:39:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:02 smithi183 bash[20824]: cluster 2024-02-13T03:39:01.339924+0000 mon.a (mon.0) 2987 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-13T03:39:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:02 smithi183 bash[20824]: audit 2024-02-13T03:39:01.341306+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-132", "mode": "writeback"}]: dispatch 2024-02-13T03:39:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:02 smithi183 bash[20824]: cluster 2024-02-13T03:39:02.334896+0000 mon.a (mon.0) 2989 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:39:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[27879]: cluster 2024-02-13T03:39:01.567338+0000 mgr.y (mgr.24473) 694 : cluster [DBG] pgmap v1136: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-13T03:39:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[27879]: audit 2024-02-13T03:39:02.345932+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-132", "mode": "writeback"}]': finished 2024-02-13T03:39:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[27879]: cluster 2024-02-13T03:39:02.346207+0000 mon.a (mon.0) 2991 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-02-13T03:39:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[27879]: audit 2024-02-13T03:39:02.349745+0000 mon.a (mon.0) 2992 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:39:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[27879]: audit 2024-02-13T03:39:03.347657+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:39:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[27879]: cluster 2024-02-13T03:39:03.347805+0000 mon.a (mon.0) 2994 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-02-13T03:39:03.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[27879]: audit 2024-02-13T03:39:03.348702+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:39:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[19769]: cluster 2024-02-13T03:39:01.567338+0000 mgr.y (mgr.24473) 694 : cluster [DBG] pgmap v1136: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-13T03:39:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[19769]: audit 2024-02-13T03:39:02.345932+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-132", "mode": "writeback"}]': finished 2024-02-13T03:39:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[19769]: cluster 2024-02-13T03:39:02.346207+0000 mon.a (mon.0) 2991 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-02-13T03:39:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[19769]: audit 2024-02-13T03:39:02.349745+0000 mon.a (mon.0) 2992 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:39:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[19769]: audit 2024-02-13T03:39:03.347657+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:39:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[19769]: cluster 2024-02-13T03:39:03.347805+0000 mon.a (mon.0) 2994 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-02-13T03:39:03.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:03 smithi161 bash[19769]: audit 2024-02-13T03:39:03.348702+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:39:03.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:03 smithi183 bash[20824]: cluster 2024-02-13T03:39:01.567338+0000 mgr.y (mgr.24473) 694 : cluster [DBG] pgmap v1136: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-13T03:39:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:03 smithi183 bash[20824]: audit 2024-02-13T03:39:02.345932+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-132", "mode": "writeback"}]': finished 2024-02-13T03:39:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:03 smithi183 bash[20824]: cluster 2024-02-13T03:39:02.346207+0000 mon.a (mon.0) 2991 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-02-13T03:39:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:03 smithi183 bash[20824]: audit 2024-02-13T03:39:02.349745+0000 mon.a (mon.0) 2992 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:39:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:03 smithi183 bash[20824]: audit 2024-02-13T03:39:03.347657+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:39:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:03 smithi183 bash[20824]: cluster 2024-02-13T03:39:03.347805+0000 mon.a (mon.0) 2994 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-02-13T03:39:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:03 smithi183 bash[20824]: audit 2024-02-13T03:39:03.348702+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:39:05.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:05 smithi161 bash[27879]: cluster 2024-02-13T03:39:03.568301+0000 mgr.y (mgr.24473) 695 : cluster [DBG] pgmap v1139: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-13T03:39:05.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:05 smithi161 bash[27879]: audit 2024-02-13T03:39:04.349326+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:39:05.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:05 smithi161 bash[27879]: cluster 2024-02-13T03:39:04.349544+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-02-13T03:39:05.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:05 smithi161 bash[27879]: audit 2024-02-13T03:39:04.351159+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:39:05.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:05 smithi161 bash[27879]: audit 2024-02-13T03:39:04.796577+0000 mon.a (mon.0) 2999 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:39:05.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:05 smithi161 bash[19769]: cluster 2024-02-13T03:39:03.568301+0000 mgr.y (mgr.24473) 695 : cluster [DBG] pgmap v1139: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-13T03:39:05.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:05 smithi161 bash[19769]: audit 2024-02-13T03:39:04.349326+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:39:05.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:05 smithi161 bash[19769]: cluster 2024-02-13T03:39:04.349544+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-02-13T03:39:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:05 smithi161 bash[19769]: audit 2024-02-13T03:39:04.351159+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:39:05.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:05 smithi161 bash[19769]: audit 2024-02-13T03:39:04.796577+0000 mon.a (mon.0) 2999 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:39:05.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:05 smithi183 bash[20824]: cluster 2024-02-13T03:39:03.568301+0000 mgr.y (mgr.24473) 695 : cluster [DBG] pgmap v1139: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-13T03:39:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:05 smithi183 bash[20824]: audit 2024-02-13T03:39:04.349326+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:39:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:05 smithi183 bash[20824]: cluster 2024-02-13T03:39:04.349544+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-02-13T03:39:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:05 smithi183 bash[20824]: audit 2024-02-13T03:39:04.351159+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:39:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:05 smithi183 bash[20824]: audit 2024-02-13T03:39:04.796577+0000 mon.a (mon.0) 2999 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:39:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:06 smithi161 bash[27879]: cluster 2024-02-13T03:39:05.342702+0000 mon.a (mon.0) 3000 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:39:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:06 smithi161 bash[27879]: audit 2024-02-13T03:39:05.351299+0000 mon.a (mon.0) 3001 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:39:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:06 smithi161 bash[27879]: cluster 2024-02-13T03:39:05.351519+0000 mon.a (mon.0) 3002 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-02-13T03:39:06.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:06 smithi161 bash[27879]: audit 2024-02-13T03:39:05.353729+0000 mon.a (mon.0) 3003 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:39:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:06 smithi161 bash[19769]: cluster 2024-02-13T03:39:05.342702+0000 mon.a (mon.0) 3000 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:39:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:06 smithi161 bash[19769]: audit 2024-02-13T03:39:05.351299+0000 mon.a (mon.0) 3001 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:39:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:06 smithi161 bash[19769]: cluster 2024-02-13T03:39:05.351519+0000 mon.a (mon.0) 3002 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-02-13T03:39:06.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:06 smithi161 bash[19769]: audit 2024-02-13T03:39:05.353729+0000 mon.a (mon.0) 3003 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:39:06.737 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:39:06 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:39:06.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:06 smithi183 bash[20824]: cluster 2024-02-13T03:39:05.342702+0000 mon.a (mon.0) 3000 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:39:06.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:06 smithi183 bash[20824]: audit 2024-02-13T03:39:05.351299+0000 mon.a (mon.0) 3001 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:39:06.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:06 smithi183 bash[20824]: cluster 2024-02-13T03:39:05.351519+0000 mon.a (mon.0) 3002 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-02-13T03:39:06.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:06 smithi183 bash[20824]: audit 2024-02-13T03:39:05.353729+0000 mon.a (mon.0) 3003 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:39:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:07 smithi161 bash[19769]: cluster 2024-02-13T03:39:05.569425+0000 mgr.y (mgr.24473) 696 : cluster [DBG] pgmap v1142: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:39:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:07 smithi161 bash[19769]: audit 2024-02-13T03:39:06.354732+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-13T03:39:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:07 smithi161 bash[19769]: cluster 2024-02-13T03:39:06.354915+0000 mon.a (mon.0) 3005 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-02-13T03:39:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:07 smithi161 bash[19769]: audit 2024-02-13T03:39:06.356600+0000 mon.a (mon.0) 3006 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-13T03:39:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:07 smithi161 bash[19769]: audit 2024-02-13T03:39:07.356530+0000 mon.a (mon.0) 3007 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-13T03:39:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:07 smithi161 bash[19769]: cluster 2024-02-13T03:39:07.356705+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-02-13T03:39:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:07 smithi161 bash[27879]: cluster 2024-02-13T03:39:05.569425+0000 mgr.y (mgr.24473) 696 : cluster [DBG] pgmap v1142: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:39:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:07 smithi161 bash[27879]: audit 2024-02-13T03:39:06.354732+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-13T03:39:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:07 smithi161 bash[27879]: cluster 2024-02-13T03:39:06.354915+0000 mon.a (mon.0) 3005 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-02-13T03:39:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:07 smithi161 bash[27879]: audit 2024-02-13T03:39:06.356600+0000 mon.a (mon.0) 3006 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-13T03:39:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:07 smithi161 bash[27879]: audit 2024-02-13T03:39:07.356530+0000 mon.a (mon.0) 3007 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-13T03:39:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:07 smithi161 bash[27879]: cluster 2024-02-13T03:39:07.356705+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-02-13T03:39:07.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:07 smithi183 bash[20824]: cluster 2024-02-13T03:39:05.569425+0000 mgr.y (mgr.24473) 696 : cluster [DBG] pgmap v1142: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:39:07.764 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:07 smithi183 bash[20824]: audit 2024-02-13T03:39:06.354732+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-13T03:39:07.764 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:07 smithi183 bash[20824]: cluster 2024-02-13T03:39:06.354915+0000 mon.a (mon.0) 3005 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-02-13T03:39:07.764 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:07 smithi183 bash[20824]: audit 2024-02-13T03:39:06.356600+0000 mon.a (mon.0) 3006 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-13T03:39:07.764 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:07 smithi183 bash[20824]: audit 2024-02-13T03:39:07.356530+0000 mon.a (mon.0) 3007 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-13T03:39:07.765 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:07 smithi183 bash[20824]: cluster 2024-02-13T03:39:07.356705+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-02-13T03:39:08.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:08 smithi161 bash[27879]: audit 2024-02-13T03:39:06.449521+0000 mgr.y (mgr.24473) 697 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:08 smithi161 bash[27879]: audit 2024-02-13T03:39:07.357656+0000 mon.a (mon.0) 3009 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-13T03:39:08.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:08 smithi161 bash[19769]: audit 2024-02-13T03:39:06.449521+0000 mgr.y (mgr.24473) 697 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:08.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:08 smithi161 bash[19769]: audit 2024-02-13T03:39:07.357656+0000 mon.a (mon.0) 3009 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-13T03:39:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:08 smithi183 bash[20824]: audit 2024-02-13T03:39:06.449521+0000 mgr.y (mgr.24473) 697 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:08 smithi183 bash[20824]: audit 2024-02-13T03:39:07.357656+0000 mon.a (mon.0) 3009 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-13T03:39:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:09 smithi161 bash[27879]: cluster 2024-02-13T03:39:07.570301+0000 mgr.y (mgr.24473) 698 : cluster [DBG] pgmap v1145: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:09 smithi161 bash[27879]: audit 2024-02-13T03:39:08.361136+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-13T03:39:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:09 smithi161 bash[27879]: cluster 2024-02-13T03:39:08.361373+0000 mon.a (mon.0) 3011 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-02-13T03:39:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:09 smithi161 bash[27879]: audit 2024-02-13T03:39:08.412220+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:09 smithi161 bash[19769]: cluster 2024-02-13T03:39:07.570301+0000 mgr.y (mgr.24473) 698 : cluster [DBG] pgmap v1145: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:09 smithi161 bash[19769]: audit 2024-02-13T03:39:08.361136+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-13T03:39:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:09 smithi161 bash[19769]: cluster 2024-02-13T03:39:08.361373+0000 mon.a (mon.0) 3011 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-02-13T03:39:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:09 smithi161 bash[19769]: audit 2024-02-13T03:39:08.412220+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:09 smithi183 bash[20824]: cluster 2024-02-13T03:39:07.570301+0000 mgr.y (mgr.24473) 698 : cluster [DBG] pgmap v1145: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:09 smithi183 bash[20824]: audit 2024-02-13T03:39:08.361136+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-13T03:39:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:09 smithi183 bash[20824]: cluster 2024-02-13T03:39:08.361373+0000 mon.a (mon.0) 3011 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-02-13T03:39:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:09 smithi183 bash[20824]: audit 2024-02-13T03:39:08.412220+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:09.738 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:39:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:39:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:39:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:10 smithi161 bash[27879]: audit 2024-02-13T03:39:09.383900+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:39:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:10 smithi161 bash[27879]: cluster 2024-02-13T03:39:09.384051+0000 mon.a (mon.0) 3014 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-02-13T03:39:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:10 smithi161 bash[27879]: audit 2024-02-13T03:39:09.387950+0000 mon.a (mon.0) 3015 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132"}]: dispatch 2024-02-13T03:39:10.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:10 smithi161 bash[19769]: audit 2024-02-13T03:39:09.383900+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:39:10.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:10 smithi161 bash[19769]: cluster 2024-02-13T03:39:09.384051+0000 mon.a (mon.0) 3014 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-02-13T03:39:10.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:10 smithi161 bash[19769]: audit 2024-02-13T03:39:09.387950+0000 mon.a (mon.0) 3015 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132"}]: dispatch 2024-02-13T03:39:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:10 smithi183 bash[20824]: audit 2024-02-13T03:39:09.383900+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:39:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:10 smithi183 bash[20824]: cluster 2024-02-13T03:39:09.384051+0000 mon.a (mon.0) 3014 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-02-13T03:39:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:10 smithi183 bash[20824]: audit 2024-02-13T03:39:09.387950+0000 mon.a (mon.0) 3015 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132"}]: dispatch 2024-02-13T03:39:11.396 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:39:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:39:11] "GET /metrics HTTP/1.1" 200 190121 "" "Prometheus/2.33.4" 2024-02-13T03:39:11.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:11 smithi161 bash[27879]: cluster 2024-02-13T03:39:09.571137+0000 mgr.y (mgr.24473) 699 : cluster [DBG] pgmap v1148: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:11.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:11 smithi161 bash[27879]: audit 2024-02-13T03:39:10.387079+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132"}]': finished 2024-02-13T03:39:11.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:11 smithi161 bash[27879]: cluster 2024-02-13T03:39:10.387218+0000 mon.a (mon.0) 3017 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-02-13T03:39:11.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:11 smithi161 bash[27879]: audit 2024-02-13T03:39:10.414095+0000 mon.a (mon.0) 3018 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:11.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:11 smithi161 bash[27879]: audit 2024-02-13T03:39:10.414554+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132"}]: dispatch 2024-02-13T03:39:11.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:11 smithi161 bash[19769]: cluster 2024-02-13T03:39:09.571137+0000 mgr.y (mgr.24473) 699 : cluster [DBG] pgmap v1148: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:11.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:11 smithi161 bash[19769]: audit 2024-02-13T03:39:10.387079+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132"}]': finished 2024-02-13T03:39:11.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:11 smithi161 bash[19769]: cluster 2024-02-13T03:39:10.387218+0000 mon.a (mon.0) 3017 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-02-13T03:39:11.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:11 smithi161 bash[19769]: audit 2024-02-13T03:39:10.414095+0000 mon.a (mon.0) 3018 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:11.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:11 smithi161 bash[19769]: audit 2024-02-13T03:39:10.414554+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132"}]: dispatch 2024-02-13T03:39:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:11 smithi183 bash[20824]: cluster 2024-02-13T03:39:09.571137+0000 mgr.y (mgr.24473) 699 : cluster [DBG] pgmap v1148: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:11 smithi183 bash[20824]: audit 2024-02-13T03:39:10.387079+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132"}]': finished 2024-02-13T03:39:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:11 smithi183 bash[20824]: cluster 2024-02-13T03:39:10.387218+0000 mon.a (mon.0) 3017 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-02-13T03:39:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:11 smithi183 bash[20824]: audit 2024-02-13T03:39:10.414095+0000 mon.a (mon.0) 3018 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:11 smithi183 bash[20824]: audit 2024-02-13T03:39:10.414554+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-132"}]: dispatch 2024-02-13T03:39:12.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:12 smithi161 bash[27879]: cluster 2024-02-13T03:39:11.407069+0000 mon.a (mon.0) 3020 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-02-13T03:39:12.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:12 smithi161 bash[19769]: cluster 2024-02-13T03:39:11.407069+0000 mon.a (mon.0) 3020 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-02-13T03:39:12.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:12 smithi183 bash[20824]: cluster 2024-02-13T03:39:11.407069+0000 mon.a (mon.0) 3020 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-02-13T03:39:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:13 smithi183 bash[20824]: cluster 2024-02-13T03:39:11.571894+0000 mgr.y (mgr.24473) 700 : cluster [DBG] pgmap v1151: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-13T03:39:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:13 smithi183 bash[20824]: cluster 2024-02-13T03:39:12.425149+0000 mon.a (mon.0) 3021 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-02-13T03:39:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:13 smithi183 bash[20824]: audit 2024-02-13T03:39:12.428804+0000 mon.a (mon.0) 3022 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:39:13.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:13 smithi161 bash[27879]: cluster 2024-02-13T03:39:11.571894+0000 mgr.y (mgr.24473) 700 : cluster [DBG] pgmap v1151: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-13T03:39:13.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:13 smithi161 bash[27879]: cluster 2024-02-13T03:39:12.425149+0000 mon.a (mon.0) 3021 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-02-13T03:39:13.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:13 smithi161 bash[27879]: audit 2024-02-13T03:39:12.428804+0000 mon.a (mon.0) 3022 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:39:13.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:13 smithi161 bash[19769]: cluster 2024-02-13T03:39:11.571894+0000 mgr.y (mgr.24473) 700 : cluster [DBG] pgmap v1151: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-13T03:39:13.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:13 smithi161 bash[19769]: cluster 2024-02-13T03:39:12.425149+0000 mon.a (mon.0) 3021 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-02-13T03:39:13.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:13 smithi161 bash[19769]: audit 2024-02-13T03:39:12.428804+0000 mon.a (mon.0) 3022 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:39:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:14 smithi183 bash[20824]: audit 2024-02-13T03:39:13.427504+0000 mon.a (mon.0) 3023 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:14 smithi183 bash[20824]: cluster 2024-02-13T03:39:13.427733+0000 mon.a (mon.0) 3024 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-02-13T03:39:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:14 smithi183 bash[20824]: audit 2024-02-13T03:39:13.447215+0000 mon.a (mon.0) 3025 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:39:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:14 smithi183 bash[20824]: cluster 2024-02-13T03:39:13.572740+0000 mgr.y (mgr.24473) 701 : cluster [DBG] pgmap v1154: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-13T03:39:14.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:14 smithi161 bash[27879]: audit 2024-02-13T03:39:13.427504+0000 mon.a (mon.0) 3023 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:14.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:14 smithi161 bash[27879]: cluster 2024-02-13T03:39:13.427733+0000 mon.a (mon.0) 3024 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-02-13T03:39:14.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:14 smithi161 bash[27879]: audit 2024-02-13T03:39:13.447215+0000 mon.a (mon.0) 3025 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:39:14.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:14 smithi161 bash[27879]: cluster 2024-02-13T03:39:13.572740+0000 mgr.y (mgr.24473) 701 : cluster [DBG] pgmap v1154: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-13T03:39:14.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:14 smithi161 bash[19769]: audit 2024-02-13T03:39:13.427504+0000 mon.a (mon.0) 3023 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:14.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:14 smithi161 bash[19769]: cluster 2024-02-13T03:39:13.427733+0000 mon.a (mon.0) 3024 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-02-13T03:39:14.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:14 smithi161 bash[19769]: audit 2024-02-13T03:39:13.447215+0000 mon.a (mon.0) 3025 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:39:14.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:14 smithi161 bash[19769]: cluster 2024-02-13T03:39:13.572740+0000 mgr.y (mgr.24473) 701 : cluster [DBG] pgmap v1154: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-13T03:39:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:15 smithi183 bash[20824]: audit 2024-02-13T03:39:14.441228+0000 mon.a (mon.0) 3026 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:39:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:15 smithi183 bash[20824]: cluster 2024-02-13T03:39:14.441511+0000 mon.a (mon.0) 3027 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-02-13T03:39:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:15 smithi183 bash[20824]: audit 2024-02-13T03:39:14.443557+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-134"}]: dispatch 2024-02-13T03:39:15.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:15 smithi161 bash[27879]: audit 2024-02-13T03:39:14.441228+0000 mon.a (mon.0) 3026 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:39:15.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:15 smithi161 bash[27879]: cluster 2024-02-13T03:39:14.441511+0000 mon.a (mon.0) 3027 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-02-13T03:39:15.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:15 smithi161 bash[27879]: audit 2024-02-13T03:39:14.443557+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-134"}]: dispatch 2024-02-13T03:39:15.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:15 smithi161 bash[19769]: audit 2024-02-13T03:39:14.441228+0000 mon.a (mon.0) 3026 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:39:15.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:15 smithi161 bash[19769]: cluster 2024-02-13T03:39:14.441511+0000 mon.a (mon.0) 3027 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-02-13T03:39:15.928 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:15 smithi161 bash[19769]: audit 2024-02-13T03:39:14.443557+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-134"}]: dispatch 2024-02-13T03:39:16.739 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:39:16 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:39:16.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:16 smithi183 bash[20824]: audit 2024-02-13T03:39:15.449317+0000 mon.a (mon.0) 3029 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-134"}]': finished 2024-02-13T03:39:16.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:16 smithi183 bash[20824]: cluster 2024-02-13T03:39:15.449519+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-02-13T03:39:16.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:16 smithi183 bash[20824]: audit 2024-02-13T03:39:15.450824+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-134", "mode": "readproxy"}]: dispatch 2024-02-13T03:39:16.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:16 smithi183 bash[20824]: cluster 2024-02-13T03:39:15.573741+0000 mgr.y (mgr.24473) 702 : cluster [DBG] pgmap v1157: 204 pgs: 27 unknown, 177 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:39:16.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:16 smithi183 bash[20824]: cluster 2024-02-13T03:39:16.438624+0000 mon.a (mon.0) 3032 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:39:16.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:16 smithi183 bash[20824]: audit 2024-02-13T03:39:16.447466+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-134", "mode": "readproxy"}]': finished 2024-02-13T03:39:16.740 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:16 smithi183 bash[20824]: cluster 2024-02-13T03:39:16.447711+0000 mon.a (mon.0) 3034 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-02-13T03:39:16.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[27879]: audit 2024-02-13T03:39:15.449317+0000 mon.a (mon.0) 3029 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-134"}]': finished 2024-02-13T03:39:16.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[27879]: cluster 2024-02-13T03:39:15.449519+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-02-13T03:39:16.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[27879]: audit 2024-02-13T03:39:15.450824+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-134", "mode": "readproxy"}]: dispatch 2024-02-13T03:39:16.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[27879]: cluster 2024-02-13T03:39:15.573741+0000 mgr.y (mgr.24473) 702 : cluster [DBG] pgmap v1157: 204 pgs: 27 unknown, 177 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:39:16.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[27879]: cluster 2024-02-13T03:39:16.438624+0000 mon.a (mon.0) 3032 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:39:16.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[27879]: audit 2024-02-13T03:39:16.447466+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-134", "mode": "readproxy"}]': finished 2024-02-13T03:39:16.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[27879]: cluster 2024-02-13T03:39:16.447711+0000 mon.a (mon.0) 3034 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-02-13T03:39:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[19769]: audit 2024-02-13T03:39:15.449317+0000 mon.a (mon.0) 3029 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-134"}]': finished 2024-02-13T03:39:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[19769]: cluster 2024-02-13T03:39:15.449519+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-02-13T03:39:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[19769]: audit 2024-02-13T03:39:15.450824+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-134", "mode": "readproxy"}]: dispatch 2024-02-13T03:39:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[19769]: cluster 2024-02-13T03:39:15.573741+0000 mgr.y (mgr.24473) 702 : cluster [DBG] pgmap v1157: 204 pgs: 27 unknown, 177 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:39:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[19769]: cluster 2024-02-13T03:39:16.438624+0000 mon.a (mon.0) 3032 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:39:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[19769]: audit 2024-02-13T03:39:16.447466+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-134", "mode": "readproxy"}]': finished 2024-02-13T03:39:16.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:16 smithi161 bash[19769]: cluster 2024-02-13T03:39:16.447711+0000 mon.a (mon.0) 3034 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-02-13T03:39:17.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:17 smithi183 bash[20824]: audit 2024-02-13T03:39:16.460607+0000 mgr.y (mgr.24473) 703 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:17.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:17 smithi161 bash[19769]: audit 2024-02-13T03:39:16.460607+0000 mgr.y (mgr.24473) 703 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:17.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:17 smithi161 bash[27879]: audit 2024-02-13T03:39:16.460607+0000 mgr.y (mgr.24473) 703 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:18 smithi183 bash[20824]: cluster 2024-02-13T03:39:17.574600+0000 mgr.y (mgr.24473) 704 : cluster [DBG] pgmap v1159: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 246 B/s wr, 1 op/s 2024-02-13T03:39:18.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:18 smithi161 bash[27879]: cluster 2024-02-13T03:39:17.574600+0000 mgr.y (mgr.24473) 704 : cluster [DBG] pgmap v1159: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 246 B/s wr, 1 op/s 2024-02-13T03:39:18.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:18 smithi161 bash[19769]: cluster 2024-02-13T03:39:17.574600+0000 mgr.y (mgr.24473) 704 : cluster [DBG] pgmap v1159: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 246 B/s wr, 1 op/s 2024-02-13T03:39:19.736 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:39:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:39:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:39:21.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:21 smithi161 bash[19769]: cluster 2024-02-13T03:39:19.575461+0000 mgr.y (mgr.24473) 705 : cluster [DBG] pgmap v1160: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 0 op/s 2024-02-13T03:39:21.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:21 smithi161 bash[19769]: audit 2024-02-13T03:39:21.000338+0000 mon.c (mon.2) 575 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:39:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:21 smithi161 bash[27879]: cluster 2024-02-13T03:39:19.575461+0000 mgr.y (mgr.24473) 705 : cluster [DBG] pgmap v1160: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 0 op/s 2024-02-13T03:39:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:21 smithi161 bash[27879]: audit 2024-02-13T03:39:21.000338+0000 mon.c (mon.2) 575 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:39:21.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:39:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:39:21] "GET /metrics HTTP/1.1" 200 190229 "" "Prometheus/2.33.4" 2024-02-13T03:39:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:21 smithi183 bash[20824]: cluster 2024-02-13T03:39:19.575461+0000 mgr.y (mgr.24473) 705 : cluster [DBG] pgmap v1160: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 0 op/s 2024-02-13T03:39:21.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:21 smithi183 bash[20824]: audit 2024-02-13T03:39:21.000338+0000 mon.c (mon.2) 575 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:39:22.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:22 smithi161 bash[19769]: audit 2024-02-13T03:39:21.321311+0000 mon.c (mon.2) 576 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:39:22.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:22 smithi161 bash[19769]: audit 2024-02-13T03:39:21.323154+0000 mon.c (mon.2) 577 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:39:22.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:22 smithi161 bash[19769]: audit 2024-02-13T03:39:21.332877+0000 mon.a (mon.0) 3035 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:39:22.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:22 smithi161 bash[27879]: audit 2024-02-13T03:39:21.321311+0000 mon.c (mon.2) 576 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:39:22.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:22 smithi161 bash[27879]: audit 2024-02-13T03:39:21.323154+0000 mon.c (mon.2) 577 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:39:22.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:22 smithi161 bash[27879]: audit 2024-02-13T03:39:21.332877+0000 mon.a (mon.0) 3035 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:39:22.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:22 smithi183 bash[20824]: audit 2024-02-13T03:39:21.321311+0000 mon.c (mon.2) 576 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:39:22.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:22 smithi183 bash[20824]: audit 2024-02-13T03:39:21.323154+0000 mon.c (mon.2) 577 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:39:22.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:22 smithi183 bash[20824]: audit 2024-02-13T03:39:21.332877+0000 mon.a (mon.0) 3035 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:39:23.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:23 smithi161 bash[19769]: cluster 2024-02-13T03:39:21.577579+0000 mgr.y (mgr.24473) 706 : cluster [DBG] pgmap v1161: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 143 B/s wr, 1 op/s 2024-02-13T03:39:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:23 smithi161 bash[27879]: cluster 2024-02-13T03:39:21.577579+0000 mgr.y (mgr.24473) 706 : cluster [DBG] pgmap v1161: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 143 B/s wr, 1 op/s 2024-02-13T03:39:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:23 smithi183 bash[20824]: cluster 2024-02-13T03:39:21.577579+0000 mgr.y (mgr.24473) 706 : cluster [DBG] pgmap v1161: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 143 B/s wr, 1 op/s 2024-02-13T03:39:25.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:25 smithi161 bash[19769]: cluster 2024-02-13T03:39:23.578462+0000 mgr.y (mgr.24473) 707 : cluster [DBG] pgmap v1162: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 125 B/s wr, 1 op/s 2024-02-13T03:39:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:25 smithi161 bash[27879]: cluster 2024-02-13T03:39:23.578462+0000 mgr.y (mgr.24473) 707 : cluster [DBG] pgmap v1162: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 125 B/s wr, 1 op/s 2024-02-13T03:39:25.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:25 smithi183 bash[20824]: cluster 2024-02-13T03:39:23.578462+0000 mgr.y (mgr.24473) 707 : cluster [DBG] pgmap v1162: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 125 B/s wr, 1 op/s 2024-02-13T03:39:26.737 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:39:26 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:39:27.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:27 smithi161 bash[19769]: cluster 2024-02-13T03:39:25.579481+0000 mgr.y (mgr.24473) 708 : cluster [DBG] pgmap v1163: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-02-13T03:39:27.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:27 smithi161 bash[19769]: audit 2024-02-13T03:39:26.709460+0000 mon.a (mon.0) 3036 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:27 smithi161 bash[27879]: cluster 2024-02-13T03:39:25.579481+0000 mgr.y (mgr.24473) 708 : cluster [DBG] pgmap v1163: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-02-13T03:39:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:27 smithi161 bash[27879]: audit 2024-02-13T03:39:26.709460+0000 mon.a (mon.0) 3036 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:27.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:27 smithi183 bash[20824]: cluster 2024-02-13T03:39:25.579481+0000 mgr.y (mgr.24473) 708 : cluster [DBG] pgmap v1163: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-02-13T03:39:27.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:27 smithi183 bash[20824]: audit 2024-02-13T03:39:26.709460+0000 mon.a (mon.0) 3036 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[27879]: audit 2024-02-13T03:39:26.471587+0000 mgr.y (mgr.24473) 709 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[27879]: audit 2024-02-13T03:39:27.351416+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:39:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[27879]: cluster 2024-02-13T03:39:27.351489+0000 mon.a (mon.0) 3038 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-02-13T03:39:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[27879]: audit 2024-02-13T03:39:27.354953+0000 mon.a (mon.0) 3039 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134"}]: dispatch 2024-02-13T03:39:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[27879]: cluster 2024-02-13T03:39:28.344693+0000 mon.a (mon.0) 3040 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:39:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[27879]: audit 2024-02-13T03:39:28.352147+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134"}]': finished 2024-02-13T03:39:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[27879]: cluster 2024-02-13T03:39:28.352252+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-02-13T03:39:28.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[19769]: audit 2024-02-13T03:39:26.471587+0000 mgr.y (mgr.24473) 709 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:28.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[19769]: audit 2024-02-13T03:39:27.351416+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:39:28.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[19769]: cluster 2024-02-13T03:39:27.351489+0000 mon.a (mon.0) 3038 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-02-13T03:39:28.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[19769]: audit 2024-02-13T03:39:27.354953+0000 mon.a (mon.0) 3039 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134"}]: dispatch 2024-02-13T03:39:28.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[19769]: cluster 2024-02-13T03:39:28.344693+0000 mon.a (mon.0) 3040 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:39:28.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[19769]: audit 2024-02-13T03:39:28.352147+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134"}]': finished 2024-02-13T03:39:28.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:28 smithi161 bash[19769]: cluster 2024-02-13T03:39:28.352252+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-02-13T03:39:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:28 smithi183 bash[20824]: audit 2024-02-13T03:39:26.471587+0000 mgr.y (mgr.24473) 709 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:28 smithi183 bash[20824]: audit 2024-02-13T03:39:27.351416+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:39:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:28 smithi183 bash[20824]: cluster 2024-02-13T03:39:27.351489+0000 mon.a (mon.0) 3038 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-02-13T03:39:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:28 smithi183 bash[20824]: audit 2024-02-13T03:39:27.354953+0000 mon.a (mon.0) 3039 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134"}]: dispatch 2024-02-13T03:39:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:28 smithi183 bash[20824]: cluster 2024-02-13T03:39:28.344693+0000 mon.a (mon.0) 3040 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:39:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:28 smithi183 bash[20824]: audit 2024-02-13T03:39:28.352147+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134"}]': finished 2024-02-13T03:39:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:28 smithi183 bash[20824]: cluster 2024-02-13T03:39:28.352252+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-02-13T03:39:29.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:39:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:39:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:39:30.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:29 smithi161 bash[27879]: cluster 2024-02-13T03:39:27.580304+0000 mgr.y (mgr.24473) 710 : cluster [DBG] pgmap v1165: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-13T03:39:30.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:29 smithi161 bash[27879]: audit 2024-02-13T03:39:28.389311+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:30.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:29 smithi161 bash[27879]: audit 2024-02-13T03:39:28.389897+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134"}]: dispatch 2024-02-13T03:39:30.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:29 smithi161 bash[19769]: cluster 2024-02-13T03:39:27.580304+0000 mgr.y (mgr.24473) 710 : cluster [DBG] pgmap v1165: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-13T03:39:30.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:29 smithi161 bash[19769]: audit 2024-02-13T03:39:28.389311+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:30.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:29 smithi161 bash[19769]: audit 2024-02-13T03:39:28.389897+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134"}]: dispatch 2024-02-13T03:39:30.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:29 smithi183 bash[20824]: cluster 2024-02-13T03:39:27.580304+0000 mgr.y (mgr.24473) 710 : cluster [DBG] pgmap v1165: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-13T03:39:30.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:29 smithi183 bash[20824]: audit 2024-02-13T03:39:28.389311+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:30.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:29 smithi183 bash[20824]: audit 2024-02-13T03:39:28.389897+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-134"}]: dispatch 2024-02-13T03:39:31.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:30 smithi161 bash[27879]: cluster 2024-02-13T03:39:29.371383+0000 mon.a (mon.0) 3045 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-02-13T03:39:31.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:30 smithi161 bash[27879]: cluster 2024-02-13T03:39:29.581050+0000 mgr.y (mgr.24473) 711 : cluster [DBG] pgmap v1168: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:39:31.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:30 smithi161 bash[27879]: cluster 2024-02-13T03:39:30.373982+0000 mon.a (mon.0) 3046 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-02-13T03:39:31.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:30 smithi161 bash[27879]: audit 2024-02-13T03:39:30.375755+0000 mon.a (mon.0) 3047 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:39:31.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:30 smithi161 bash[19769]: cluster 2024-02-13T03:39:29.371383+0000 mon.a (mon.0) 3045 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-02-13T03:39:31.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:30 smithi161 bash[19769]: cluster 2024-02-13T03:39:29.581050+0000 mgr.y (mgr.24473) 711 : cluster [DBG] pgmap v1168: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:39:31.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:30 smithi161 bash[19769]: cluster 2024-02-13T03:39:30.373982+0000 mon.a (mon.0) 3046 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-02-13T03:39:31.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:30 smithi161 bash[19769]: audit 2024-02-13T03:39:30.375755+0000 mon.a (mon.0) 3047 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:39:31.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:30 smithi183 bash[20824]: cluster 2024-02-13T03:39:29.371383+0000 mon.a (mon.0) 3045 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-02-13T03:39:31.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:30 smithi183 bash[20824]: cluster 2024-02-13T03:39:29.581050+0000 mgr.y (mgr.24473) 711 : cluster [DBG] pgmap v1168: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:39:31.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:30 smithi183 bash[20824]: cluster 2024-02-13T03:39:30.373982+0000 mon.a (mon.0) 3046 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-02-13T03:39:31.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:30 smithi183 bash[20824]: audit 2024-02-13T03:39:30.375755+0000 mon.a (mon.0) 3047 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:39:31.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:39:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:39:31] "GET /metrics HTTP/1.1" 200 190229 "" "Prometheus/2.33.4" 2024-02-13T03:39:32.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:32 smithi161 bash[27879]: audit 2024-02-13T03:39:31.376950+0000 mon.a (mon.0) 3048 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:32.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:32 smithi161 bash[27879]: cluster 2024-02-13T03:39:31.377225+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-02-13T03:39:32.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:32 smithi161 bash[27879]: audit 2024-02-13T03:39:31.402363+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:39:32.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:32 smithi161 bash[19769]: audit 2024-02-13T03:39:31.376950+0000 mon.a (mon.0) 3048 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:32.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:32 smithi161 bash[19769]: cluster 2024-02-13T03:39:31.377225+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-02-13T03:39:32.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:32 smithi161 bash[19769]: audit 2024-02-13T03:39:31.402363+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:39:32.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:32 smithi183 bash[20824]: audit 2024-02-13T03:39:31.376950+0000 mon.a (mon.0) 3048 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:32 smithi183 bash[20824]: cluster 2024-02-13T03:39:31.377225+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-02-13T03:39:32.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:32 smithi183 bash[20824]: audit 2024-02-13T03:39:31.402363+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:39:33.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:33 smithi161 bash[27879]: cluster 2024-02-13T03:39:31.581889+0000 mgr.y (mgr.24473) 712 : cluster [DBG] pgmap v1171: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:33.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:33 smithi161 bash[27879]: audit 2024-02-13T03:39:32.386532+0000 mon.a (mon.0) 3051 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:39:33.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:33 smithi161 bash[27879]: cluster 2024-02-13T03:39:32.386618+0000 mon.a (mon.0) 3052 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-02-13T03:39:33.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:33 smithi161 bash[27879]: audit 2024-02-13T03:39:32.387311+0000 mon.a (mon.0) 3053 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-136"}]: dispatch 2024-02-13T03:39:33.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:33 smithi161 bash[19769]: cluster 2024-02-13T03:39:31.581889+0000 mgr.y (mgr.24473) 712 : cluster [DBG] pgmap v1171: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:33.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:33 smithi161 bash[19769]: audit 2024-02-13T03:39:32.386532+0000 mon.a (mon.0) 3051 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:39:33.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:33 smithi161 bash[19769]: cluster 2024-02-13T03:39:32.386618+0000 mon.a (mon.0) 3052 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-02-13T03:39:33.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:33 smithi161 bash[19769]: audit 2024-02-13T03:39:32.387311+0000 mon.a (mon.0) 3053 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-136"}]: dispatch 2024-02-13T03:39:33.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:33 smithi183 bash[20824]: cluster 2024-02-13T03:39:31.581889+0000 mgr.y (mgr.24473) 712 : cluster [DBG] pgmap v1171: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:33.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:33 smithi183 bash[20824]: audit 2024-02-13T03:39:32.386532+0000 mon.a (mon.0) 3051 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:39:33.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:33 smithi183 bash[20824]: cluster 2024-02-13T03:39:32.386618+0000 mon.a (mon.0) 3052 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-02-13T03:39:33.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:33 smithi183 bash[20824]: audit 2024-02-13T03:39:32.387311+0000 mon.a (mon.0) 3053 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-136"}]: dispatch 2024-02-13T03:39:34.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:34 smithi161 bash[27879]: audit 2024-02-13T03:39:33.390965+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-136"}]': finished 2024-02-13T03:39:34.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:34 smithi161 bash[27879]: cluster 2024-02-13T03:39:33.391132+0000 mon.a (mon.0) 3055 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-02-13T03:39:34.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:34 smithi161 bash[27879]: audit 2024-02-13T03:39:33.392138+0000 mon.a (mon.0) 3056 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-136", "mode": "writeback"}]: dispatch 2024-02-13T03:39:34.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:34 smithi161 bash[27879]: cluster 2024-02-13T03:39:34.384462+0000 mon.a (mon.0) 3057 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:39:34.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:34 smithi161 bash[19769]: audit 2024-02-13T03:39:33.390965+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-136"}]': finished 2024-02-13T03:39:34.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:34 smithi161 bash[19769]: cluster 2024-02-13T03:39:33.391132+0000 mon.a (mon.0) 3055 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-02-13T03:39:34.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:34 smithi161 bash[19769]: audit 2024-02-13T03:39:33.392138+0000 mon.a (mon.0) 3056 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-136", "mode": "writeback"}]: dispatch 2024-02-13T03:39:34.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:34 smithi161 bash[19769]: cluster 2024-02-13T03:39:34.384462+0000 mon.a (mon.0) 3057 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:39:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:34 smithi183 bash[20824]: audit 2024-02-13T03:39:33.390965+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi161-61906-101", "overlaypool": "test-rados-api-smithi161-61906-136"}]': finished 2024-02-13T03:39:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:34 smithi183 bash[20824]: cluster 2024-02-13T03:39:33.391132+0000 mon.a (mon.0) 3055 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-02-13T03:39:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:34 smithi183 bash[20824]: audit 2024-02-13T03:39:33.392138+0000 mon.a (mon.0) 3056 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-136", "mode": "writeback"}]: dispatch 2024-02-13T03:39:34.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:34 smithi183 bash[20824]: cluster 2024-02-13T03:39:34.384462+0000 mon.a (mon.0) 3057 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-13T03:39:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:35 smithi183 bash[20824]: cluster 2024-02-13T03:39:33.582830+0000 mgr.y (mgr.24473) 713 : cluster [DBG] pgmap v1174: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:35 smithi183 bash[20824]: audit 2024-02-13T03:39:34.394366+0000 mon.a (mon.0) 3058 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-136", "mode": "writeback"}]': finished 2024-02-13T03:39:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:35 smithi183 bash[20824]: cluster 2024-02-13T03:39:34.394542+0000 mon.a (mon.0) 3059 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-02-13T03:39:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:35 smithi183 bash[20824]: audit 2024-02-13T03:39:34.551785+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:39:35.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:35 smithi161 bash[27879]: cluster 2024-02-13T03:39:33.582830+0000 mgr.y (mgr.24473) 713 : cluster [DBG] pgmap v1174: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:35 smithi161 bash[27879]: audit 2024-02-13T03:39:34.394366+0000 mon.a (mon.0) 3058 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-136", "mode": "writeback"}]': finished 2024-02-13T03:39:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:35 smithi161 bash[27879]: cluster 2024-02-13T03:39:34.394542+0000 mon.a (mon.0) 3059 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-02-13T03:39:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:35 smithi161 bash[27879]: audit 2024-02-13T03:39:34.551785+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:39:35.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:35 smithi161 bash[19769]: cluster 2024-02-13T03:39:33.582830+0000 mgr.y (mgr.24473) 713 : cluster [DBG] pgmap v1174: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:39:35.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:35 smithi161 bash[19769]: audit 2024-02-13T03:39:34.394366+0000 mon.a (mon.0) 3058 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi161-61906-136", "mode": "writeback"}]': finished 2024-02-13T03:39:35.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:35 smithi161 bash[19769]: cluster 2024-02-13T03:39:34.394542+0000 mon.a (mon.0) 3059 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-02-13T03:39:35.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:35 smithi161 bash[19769]: audit 2024-02-13T03:39:34.551785+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-13T03:39:36.737 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:39:36 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:39:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:36 smithi183 bash[20824]: audit 2024-02-13T03:39:35.440626+0000 mon.a (mon.0) 3061 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:39:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:36 smithi183 bash[20824]: cluster 2024-02-13T03:39:35.440803+0000 mon.a (mon.0) 3062 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-02-13T03:39:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:36 smithi183 bash[20824]: audit 2024-02-13T03:39:35.442818+0000 mon.a (mon.0) 3063 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:39:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:36 smithi183 bash[20824]: cluster 2024-02-13T03:39:35.583859+0000 mgr.y (mgr.24473) 714 : cluster [DBG] pgmap v1177: 204 pgs: 31 creating+peering, 173 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 0 op/s 2024-02-13T03:39:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:36 smithi161 bash[27879]: audit 2024-02-13T03:39:35.440626+0000 mon.a (mon.0) 3061 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:39:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:36 smithi161 bash[27879]: cluster 2024-02-13T03:39:35.440803+0000 mon.a (mon.0) 3062 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-02-13T03:39:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:36 smithi161 bash[27879]: audit 2024-02-13T03:39:35.442818+0000 mon.a (mon.0) 3063 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:39:36.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:36 smithi161 bash[27879]: cluster 2024-02-13T03:39:35.583859+0000 mgr.y (mgr.24473) 714 : cluster [DBG] pgmap v1177: 204 pgs: 31 creating+peering, 173 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 0 op/s 2024-02-13T03:39:36.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:36 smithi161 bash[19769]: audit 2024-02-13T03:39:35.440626+0000 mon.a (mon.0) 3061 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_count","val": "2"}]': finished 2024-02-13T03:39:36.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:36 smithi161 bash[19769]: cluster 2024-02-13T03:39:35.440803+0000 mon.a (mon.0) 3062 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-02-13T03:39:36.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:36 smithi161 bash[19769]: audit 2024-02-13T03:39:35.442818+0000 mon.a (mon.0) 3063 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-13T03:39:36.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:36 smithi161 bash[19769]: cluster 2024-02-13T03:39:35.583859+0000 mgr.y (mgr.24473) 714 : cluster [DBG] pgmap v1177: 204 pgs: 31 creating+peering, 173 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 0 op/s 2024-02-13T03:39:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:37 smithi183 bash[20824]: audit 2024-02-13T03:39:36.447732+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:39:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:37 smithi183 bash[20824]: cluster 2024-02-13T03:39:36.447855+0000 mon.a (mon.0) 3065 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-02-13T03:39:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:37 smithi183 bash[20824]: audit 2024-02-13T03:39:36.449564+0000 mon.a (mon.0) 3066 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:39:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:37 smithi183 bash[20824]: audit 2024-02-13T03:39:36.480617+0000 mgr.y (mgr.24473) 715 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:37.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:37 smithi161 bash[27879]: audit 2024-02-13T03:39:36.447732+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:39:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:37 smithi161 bash[27879]: cluster 2024-02-13T03:39:36.447855+0000 mon.a (mon.0) 3065 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-02-13T03:39:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:37 smithi161 bash[27879]: audit 2024-02-13T03:39:36.449564+0000 mon.a (mon.0) 3066 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:39:37.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:37 smithi161 bash[27879]: audit 2024-02-13T03:39:36.480617+0000 mgr.y (mgr.24473) 715 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:37.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:37 smithi161 bash[19769]: audit 2024-02-13T03:39:36.447732+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_period","val": "600"}]': finished 2024-02-13T03:39:37.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:37 smithi161 bash[19769]: cluster 2024-02-13T03:39:36.447855+0000 mon.a (mon.0) 3065 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-02-13T03:39:37.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:37 smithi161 bash[19769]: audit 2024-02-13T03:39:36.449564+0000 mon.a (mon.0) 3066 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-13T03:39:37.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:37 smithi161 bash[19769]: audit 2024-02-13T03:39:36.480617+0000 mgr.y (mgr.24473) 715 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:38.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:38 smithi183 bash[20824]: cluster 2024-02-13T03:39:37.442858+0000 mon.a (mon.0) 3067 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:39:38.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:38 smithi183 bash[20824]: audit 2024-02-13T03:39:37.450605+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:39:38.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:38 smithi183 bash[20824]: cluster 2024-02-13T03:39:37.450779+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-02-13T03:39:38.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:38 smithi183 bash[20824]: audit 2024-02-13T03:39:37.451793+0000 mon.a (mon.0) 3070 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:39:38.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:38 smithi183 bash[20824]: cluster 2024-02-13T03:39:37.584703+0000 mgr.y (mgr.24473) 716 : cluster [DBG] pgmap v1180: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-13T03:39:38.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:38 smithi161 bash[19769]: cluster 2024-02-13T03:39:37.442858+0000 mon.a (mon.0) 3067 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:39:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:38 smithi161 bash[19769]: audit 2024-02-13T03:39:37.450605+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:39:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:38 smithi161 bash[19769]: cluster 2024-02-13T03:39:37.450779+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-02-13T03:39:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:38 smithi161 bash[19769]: audit 2024-02-13T03:39:37.451793+0000 mon.a (mon.0) 3070 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:39:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:38 smithi161 bash[19769]: cluster 2024-02-13T03:39:37.584703+0000 mgr.y (mgr.24473) 716 : cluster [DBG] pgmap v1180: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-13T03:39:38.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:38 smithi161 bash[27879]: cluster 2024-02-13T03:39:37.442858+0000 mon.a (mon.0) 3067 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-13T03:39:38.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:38 smithi161 bash[27879]: audit 2024-02-13T03:39:37.450605+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-13T03:39:38.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:38 smithi161 bash[27879]: cluster 2024-02-13T03:39:37.450779+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-02-13T03:39:38.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:38 smithi161 bash[27879]: audit 2024-02-13T03:39:37.451793+0000 mon.a (mon.0) 3070 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-13T03:39:38.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:38 smithi161 bash[27879]: cluster 2024-02-13T03:39:37.584703+0000 mgr.y (mgr.24473) 716 : cluster [DBG] pgmap v1180: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-13T03:39:39.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:39 smithi183 bash[20824]: audit 2024-02-13T03:39:38.454594+0000 mon.a (mon.0) 3071 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-13T03:39:39.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:39 smithi183 bash[20824]: cluster 2024-02-13T03:39:38.455243+0000 mon.a (mon.0) 3072 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-02-13T03:39:39.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:39 smithi183 bash[20824]: audit 2024-02-13T03:39:38.456998+0000 mon.a (mon.0) 3073 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-13T03:39:39.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:39:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:39:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:39:39.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:39 smithi161 bash[27879]: audit 2024-02-13T03:39:38.454594+0000 mon.a (mon.0) 3071 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-13T03:39:39.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:39 smithi161 bash[27879]: cluster 2024-02-13T03:39:38.455243+0000 mon.a (mon.0) 3072 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-02-13T03:39:39.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:39 smithi161 bash[27879]: audit 2024-02-13T03:39:38.456998+0000 mon.a (mon.0) 3073 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-13T03:39:39.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:39 smithi161 bash[19769]: audit 2024-02-13T03:39:38.454594+0000 mon.a (mon.0) 3071 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-13T03:39:39.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:39 smithi161 bash[19769]: cluster 2024-02-13T03:39:38.455243+0000 mon.a (mon.0) 3072 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-02-13T03:39:39.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:39 smithi161 bash[19769]: audit 2024-02-13T03:39:38.456998+0000 mon.a (mon.0) 3073 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-13T03:39:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:40 smithi183 bash[20824]: audit 2024-02-13T03:39:39.459458+0000 mon.a (mon.0) 3074 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "target_max_objects","val": "1"}]': finished 2024-02-13T03:39:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:40 smithi183 bash[20824]: cluster 2024-02-13T03:39:39.459542+0000 mon.a (mon.0) 3075 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-02-13T03:39:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:40 smithi183 bash[20824]: cluster 2024-02-13T03:39:39.585539+0000 mgr.y (mgr.24473) 717 : cluster [DBG] pgmap v1183: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-13T03:39:40.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:40 smithi161 bash[27879]: audit 2024-02-13T03:39:39.459458+0000 mon.a (mon.0) 3074 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "target_max_objects","val": "1"}]': finished 2024-02-13T03:39:40.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:40 smithi161 bash[27879]: cluster 2024-02-13T03:39:39.459542+0000 mon.a (mon.0) 3075 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-02-13T03:39:40.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:40 smithi161 bash[27879]: cluster 2024-02-13T03:39:39.585539+0000 mgr.y (mgr.24473) 717 : cluster [DBG] pgmap v1183: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-13T03:39:40.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:40 smithi161 bash[19769]: audit 2024-02-13T03:39:39.459458+0000 mon.a (mon.0) 3074 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-136","var": "target_max_objects","val": "1"}]': finished 2024-02-13T03:39:40.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:40 smithi161 bash[19769]: cluster 2024-02-13T03:39:39.459542+0000 mon.a (mon.0) 3075 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-02-13T03:39:40.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:40 smithi161 bash[19769]: cluster 2024-02-13T03:39:39.585539+0000 mgr.y (mgr.24473) 717 : cluster [DBG] pgmap v1183: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-13T03:39:41.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:39:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:39:41] "GET /metrics HTTP/1.1" 200 190361 "" "Prometheus/2.33.4" 2024-02-13T03:39:41.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:41 smithi183 bash[20824]: cluster 2024-02-13T03:39:40.464765+0000 mon.a (mon.0) 3076 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-13T03:39:41.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:41 smithi161 bash[27879]: cluster 2024-02-13T03:39:40.464765+0000 mon.a (mon.0) 3076 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-13T03:39:41.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:41 smithi161 bash[19769]: cluster 2024-02-13T03:39:40.464765+0000 mon.a (mon.0) 3076 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-13T03:39:42.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:42 smithi183 bash[20824]: cluster 2024-02-13T03:39:41.587745+0000 mgr.y (mgr.24473) 718 : cluster [DBG] pgmap v1184: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 996 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:39:42.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:42 smithi161 bash[27879]: cluster 2024-02-13T03:39:41.587745+0000 mgr.y (mgr.24473) 718 : cluster [DBG] pgmap v1184: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 996 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:39:42.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:42 smithi161 bash[19769]: cluster 2024-02-13T03:39:41.587745+0000 mgr.y (mgr.24473) 718 : cluster [DBG] pgmap v1184: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 996 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:39:45.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:45 smithi161 bash[27879]: cluster 2024-02-13T03:39:43.588601+0000 mgr.y (mgr.24473) 719 : cluster [DBG] pgmap v1185: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 833 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:39:45.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:45 smithi161 bash[19769]: cluster 2024-02-13T03:39:43.588601+0000 mgr.y (mgr.24473) 719 : cluster [DBG] pgmap v1185: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 833 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:39:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:45 smithi183 bash[20824]: cluster 2024-02-13T03:39:43.588601+0000 mgr.y (mgr.24473) 719 : cluster [DBG] pgmap v1185: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 833 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:39:46.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:39:46 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:39:47.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:47 smithi161 bash[27879]: cluster 2024-02-13T03:39:45.589583+0000 mgr.y (mgr.24473) 720 : cluster [DBG] pgmap v1186: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-02-13T03:39:47.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:47 smithi161 bash[19769]: cluster 2024-02-13T03:39:45.589583+0000 mgr.y (mgr.24473) 720 : cluster [DBG] pgmap v1186: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-02-13T03:39:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:47 smithi183 bash[20824]: cluster 2024-02-13T03:39:45.589583+0000 mgr.y (mgr.24473) 720 : cluster [DBG] pgmap v1186: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-02-13T03:39:48.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:48 smithi161 bash[27879]: audit 2024-02-13T03:39:46.485391+0000 mgr.y (mgr.24473) 721 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:48.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:48 smithi161 bash[19769]: audit 2024-02-13T03:39:46.485391+0000 mgr.y (mgr.24473) 721 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:48 smithi183 bash[20824]: audit 2024-02-13T03:39:46.485391+0000 mgr.y (mgr.24473) 721 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:49.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:49 smithi161 bash[27879]: cluster 2024-02-13T03:39:47.591650+0000 mgr.y (mgr.24473) 722 : cluster [DBG] pgmap v1187: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:39:49.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:49 smithi161 bash[19769]: cluster 2024-02-13T03:39:47.591650+0000 mgr.y (mgr.24473) 722 : cluster [DBG] pgmap v1187: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:39:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:49 smithi183 bash[20824]: cluster 2024-02-13T03:39:47.591650+0000 mgr.y (mgr.24473) 722 : cluster [DBG] pgmap v1187: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:39:49.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:39:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:39:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:39:50.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:50 smithi161 bash[27879]: audit 2024-02-13T03:39:49.487785+0000 mon.a (mon.0) 3077 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:50.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:50 smithi161 bash[27879]: audit 2024-02-13T03:39:49.798045+0000 mon.a (mon.0) 3078 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:39:50.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:50 smithi161 bash[19769]: audit 2024-02-13T03:39:49.487785+0000 mon.a (mon.0) 3077 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:50.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:50 smithi161 bash[19769]: audit 2024-02-13T03:39:49.798045+0000 mon.a (mon.0) 3078 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:39:50.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:50 smithi183 bash[20824]: audit 2024-02-13T03:39:49.487785+0000 mon.a (mon.0) 3077 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:50 smithi183 bash[20824]: audit 2024-02-13T03:39:49.798045+0000 mon.a (mon.0) 3078 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:39:51.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:51 smithi161 bash[19769]: cluster 2024-02-13T03:39:49.592542+0000 mgr.y (mgr.24473) 723 : cluster [DBG] pgmap v1188: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:39:51.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:51 smithi161 bash[19769]: audit 2024-02-13T03:39:50.112729+0000 mon.a (mon.0) 3079 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:39:51.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:51 smithi161 bash[19769]: cluster 2024-02-13T03:39:50.113069+0000 mon.a (mon.0) 3080 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-02-13T03:39:51.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:51 smithi161 bash[19769]: audit 2024-02-13T03:39:50.113738+0000 mon.a (mon.0) 3081 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136"}]: dispatch 2024-02-13T03:39:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:39:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:39:51] "GET /metrics HTTP/1.1" 200 190395 "" "Prometheus/2.33.4" 2024-02-13T03:39:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:51 smithi161 bash[27879]: cluster 2024-02-13T03:39:49.592542+0000 mgr.y (mgr.24473) 723 : cluster [DBG] pgmap v1188: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:39:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:51 smithi161 bash[27879]: audit 2024-02-13T03:39:50.112729+0000 mon.a (mon.0) 3079 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:39:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:51 smithi161 bash[27879]: cluster 2024-02-13T03:39:50.113069+0000 mon.a (mon.0) 3080 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-02-13T03:39:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:51 smithi161 bash[27879]: audit 2024-02-13T03:39:50.113738+0000 mon.a (mon.0) 3081 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136"}]: dispatch 2024-02-13T03:39:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:51 smithi183 bash[20824]: cluster 2024-02-13T03:39:49.592542+0000 mgr.y (mgr.24473) 723 : cluster [DBG] pgmap v1188: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 B/s wr, 1 op/s 2024-02-13T03:39:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:51 smithi183 bash[20824]: audit 2024-02-13T03:39:50.112729+0000 mon.a (mon.0) 3079 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:39:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:51 smithi183 bash[20824]: cluster 2024-02-13T03:39:50.113069+0000 mon.a (mon.0) 3080 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-02-13T03:39:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:51 smithi183 bash[20824]: audit 2024-02-13T03:39:50.113738+0000 mon.a (mon.0) 3081 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136"}]: dispatch 2024-02-13T03:39:52.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:52 smithi161 bash[27879]: audit 2024-02-13T03:39:51.126678+0000 mon.a (mon.0) 3082 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136"}]': finished 2024-02-13T03:39:52.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:52 smithi161 bash[27879]: cluster 2024-02-13T03:39:51.126780+0000 mon.a (mon.0) 3083 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-02-13T03:39:52.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:52 smithi161 bash[27879]: audit 2024-02-13T03:39:51.163309+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:52.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:52 smithi161 bash[27879]: audit 2024-02-13T03:39:51.163890+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136"}]: dispatch 2024-02-13T03:39:52.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:52 smithi161 bash[19769]: audit 2024-02-13T03:39:51.126678+0000 mon.a (mon.0) 3082 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136"}]': finished 2024-02-13T03:39:52.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:52 smithi161 bash[19769]: cluster 2024-02-13T03:39:51.126780+0000 mon.a (mon.0) 3083 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-02-13T03:39:52.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:52 smithi161 bash[19769]: audit 2024-02-13T03:39:51.163309+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:52.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:52 smithi161 bash[19769]: audit 2024-02-13T03:39:51.163890+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136"}]: dispatch 2024-02-13T03:39:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:52 smithi183 bash[20824]: audit 2024-02-13T03:39:51.126678+0000 mon.a (mon.0) 3082 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136"}]': finished 2024-02-13T03:39:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:52 smithi183 bash[20824]: cluster 2024-02-13T03:39:51.126780+0000 mon.a (mon.0) 3083 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-02-13T03:39:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:52 smithi183 bash[20824]: audit 2024-02-13T03:39:51.163309+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:52 smithi183 bash[20824]: audit 2024-02-13T03:39:51.163890+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-136"}]: dispatch 2024-02-13T03:39:53.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:53 smithi161 bash[27879]: cluster 2024-02-13T03:39:51.593348+0000 mgr.y (mgr.24473) 724 : cluster [DBG] pgmap v1191: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-13T03:39:53.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:53 smithi161 bash[27879]: cluster 2024-02-13T03:39:52.122139+0000 mon.a (mon.0) 3086 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:39:53.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:53 smithi161 bash[27879]: cluster 2024-02-13T03:39:52.122322+0000 mon.a (mon.0) 3087 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-13T03:39:53.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:53 smithi161 bash[27879]: cluster 2024-02-13T03:39:52.136399+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-02-13T03:39:53.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:53 smithi161 bash[19769]: cluster 2024-02-13T03:39:51.593348+0000 mgr.y (mgr.24473) 724 : cluster [DBG] pgmap v1191: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-13T03:39:53.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:53 smithi161 bash[19769]: cluster 2024-02-13T03:39:52.122139+0000 mon.a (mon.0) 3086 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:39:53.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:53 smithi161 bash[19769]: cluster 2024-02-13T03:39:52.122322+0000 mon.a (mon.0) 3087 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-13T03:39:53.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:53 smithi161 bash[19769]: cluster 2024-02-13T03:39:52.136399+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-02-13T03:39:53.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:53 smithi183 bash[20824]: cluster 2024-02-13T03:39:51.593348+0000 mgr.y (mgr.24473) 724 : cluster [DBG] pgmap v1191: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-13T03:39:53.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:53 smithi183 bash[20824]: cluster 2024-02-13T03:39:52.122139+0000 mon.a (mon.0) 3086 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:39:53.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:53 smithi183 bash[20824]: cluster 2024-02-13T03:39:52.122322+0000 mon.a (mon.0) 3087 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-13T03:39:53.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:53 smithi183 bash[20824]: cluster 2024-02-13T03:39:52.136399+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-02-13T03:39:54.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:54 smithi161 bash[27879]: cluster 2024-02-13T03:39:53.142859+0000 mon.a (mon.0) 3089 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-02-13T03:39:54.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:54 smithi161 bash[27879]: audit 2024-02-13T03:39:53.144414+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:39:54.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:54 smithi161 bash[19769]: cluster 2024-02-13T03:39:53.142859+0000 mon.a (mon.0) 3089 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-02-13T03:39:54.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:54 smithi161 bash[19769]: audit 2024-02-13T03:39:53.144414+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:39:54.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:54 smithi183 bash[20824]: cluster 2024-02-13T03:39:53.142859+0000 mon.a (mon.0) 3089 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-02-13T03:39:54.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:54 smithi183 bash[20824]: audit 2024-02-13T03:39:53.144414+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:39:55.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:55 smithi161 bash[27879]: cluster 2024-02-13T03:39:53.594223+0000 mgr.y (mgr.24473) 725 : cluster [DBG] pgmap v1194: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-13T03:39:55.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:55 smithi161 bash[27879]: audit 2024-02-13T03:39:54.147944+0000 mon.a (mon.0) 3091 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:55.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:55 smithi161 bash[27879]: cluster 2024-02-13T03:39:54.148719+0000 mon.a (mon.0) 3092 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-02-13T03:39:55.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:55 smithi161 bash[27879]: audit 2024-02-13T03:39:54.175947+0000 mon.a (mon.0) 3093 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:39:55.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:55 smithi161 bash[27879]: audit 2024-02-13T03:39:55.150072+0000 mon.a (mon.0) 3094 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:39:55.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:55 smithi161 bash[27879]: cluster 2024-02-13T03:39:55.150167+0000 mon.a (mon.0) 3095 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-02-13T03:39:55.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:55 smithi161 bash[19769]: cluster 2024-02-13T03:39:53.594223+0000 mgr.y (mgr.24473) 725 : cluster [DBG] pgmap v1194: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-13T03:39:55.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:55 smithi161 bash[19769]: audit 2024-02-13T03:39:54.147944+0000 mon.a (mon.0) 3091 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:55.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:55 smithi161 bash[19769]: cluster 2024-02-13T03:39:54.148719+0000 mon.a (mon.0) 3092 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-02-13T03:39:55.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:55 smithi161 bash[19769]: audit 2024-02-13T03:39:54.175947+0000 mon.a (mon.0) 3093 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:39:55.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:55 smithi161 bash[19769]: audit 2024-02-13T03:39:55.150072+0000 mon.a (mon.0) 3094 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:39:55.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:55 smithi161 bash[19769]: cluster 2024-02-13T03:39:55.150167+0000 mon.a (mon.0) 3095 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-02-13T03:39:55.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:55 smithi183 bash[20824]: cluster 2024-02-13T03:39:53.594223+0000 mgr.y (mgr.24473) 725 : cluster [DBG] pgmap v1194: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-13T03:39:55.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:55 smithi183 bash[20824]: audit 2024-02-13T03:39:54.147944+0000 mon.a (mon.0) 3091 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:55.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:55 smithi183 bash[20824]: cluster 2024-02-13T03:39:54.148719+0000 mon.a (mon.0) 3092 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-02-13T03:39:55.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:55 smithi183 bash[20824]: audit 2024-02-13T03:39:54.175947+0000 mon.a (mon.0) 3093 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-13T03:39:55.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:55 smithi183 bash[20824]: audit 2024-02-13T03:39:55.150072+0000 mon.a (mon.0) 3094 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-13T03:39:55.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:55 smithi183 bash[20824]: cluster 2024-02-13T03:39:55.150167+0000 mon.a (mon.0) 3095 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-02-13T03:39:56.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:56 smithi161 bash[27879]: audit 2024-02-13T03:39:55.167558+0000 mon.a (mon.0) 3096 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138"}]: dispatch 2024-02-13T03:39:56.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:56 smithi161 bash[19769]: audit 2024-02-13T03:39:55.167558+0000 mon.a (mon.0) 3096 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138"}]: dispatch 2024-02-13T03:39:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:56 smithi183 bash[20824]: audit 2024-02-13T03:39:55.167558+0000 mon.a (mon.0) 3096 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138"}]: dispatch 2024-02-13T03:39:56.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:39:56 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:39:57.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:57 smithi161 bash[27879]: cluster 2024-02-13T03:39:55.595234+0000 mgr.y (mgr.24473) 726 : cluster [DBG] pgmap v1197: 204 pgs: 27 unknown, 177 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:39:57.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:57 smithi161 bash[27879]: audit 2024-02-13T03:39:56.176181+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138"}]': finished 2024-02-13T03:39:57.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:57 smithi161 bash[27879]: cluster 2024-02-13T03:39:56.176275+0000 mon.a (mon.0) 3098 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-02-13T03:39:57.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:57 smithi161 bash[27879]: audit 2024-02-13T03:39:56.212024+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:57.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:57 smithi161 bash[27879]: audit 2024-02-13T03:39:56.212588+0000 mon.a (mon.0) 3100 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138"}]: dispatch 2024-02-13T03:39:57.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:57 smithi161 bash[19769]: cluster 2024-02-13T03:39:55.595234+0000 mgr.y (mgr.24473) 726 : cluster [DBG] pgmap v1197: 204 pgs: 27 unknown, 177 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:39:57.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:57 smithi161 bash[19769]: audit 2024-02-13T03:39:56.176181+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138"}]': finished 2024-02-13T03:39:57.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:57 smithi161 bash[19769]: cluster 2024-02-13T03:39:56.176275+0000 mon.a (mon.0) 3098 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-02-13T03:39:57.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:57 smithi161 bash[19769]: audit 2024-02-13T03:39:56.212024+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:57.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:57 smithi161 bash[19769]: audit 2024-02-13T03:39:56.212588+0000 mon.a (mon.0) 3100 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138"}]: dispatch 2024-02-13T03:39:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:57 smithi183 bash[20824]: cluster 2024-02-13T03:39:55.595234+0000 mgr.y (mgr.24473) 726 : cluster [DBG] pgmap v1197: 204 pgs: 27 unknown, 177 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:39:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:57 smithi183 bash[20824]: audit 2024-02-13T03:39:56.176181+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138"}]': finished 2024-02-13T03:39:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:57 smithi183 bash[20824]: cluster 2024-02-13T03:39:56.176275+0000 mon.a (mon.0) 3098 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-02-13T03:39:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:57 smithi183 bash[20824]: audit 2024-02-13T03:39:56.212024+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:57.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:57 smithi183 bash[20824]: audit 2024-02-13T03:39:56.212588+0000 mon.a (mon.0) 3100 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-138"}]: dispatch 2024-02-13T03:39:58.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:58 smithi161 bash[27879]: audit 2024-02-13T03:39:56.493093+0000 mgr.y (mgr.24473) 727 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:58.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:58 smithi161 bash[27879]: cluster 2024-02-13T03:39:57.181233+0000 mon.a (mon.0) 3101 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-02-13T03:39:58.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:58 smithi161 bash[27879]: cluster 2024-02-13T03:39:57.595971+0000 mgr.y (mgr.24473) 728 : cluster [DBG] pgmap v1200: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:39:58.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:58 smithi161 bash[27879]: cluster 2024-02-13T03:39:58.185067+0000 mon.a (mon.0) 3102 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-02-13T03:39:58.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:58 smithi161 bash[27879]: audit 2024-02-13T03:39:58.190215+0000 mon.a (mon.0) 3103 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:39:58.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:58 smithi161 bash[19769]: audit 2024-02-13T03:39:56.493093+0000 mgr.y (mgr.24473) 727 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:58.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:58 smithi161 bash[19769]: cluster 2024-02-13T03:39:57.181233+0000 mon.a (mon.0) 3101 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-02-13T03:39:58.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:58 smithi161 bash[19769]: cluster 2024-02-13T03:39:57.595971+0000 mgr.y (mgr.24473) 728 : cluster [DBG] pgmap v1200: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:39:58.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:58 smithi161 bash[19769]: cluster 2024-02-13T03:39:58.185067+0000 mon.a (mon.0) 3102 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-02-13T03:39:58.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:58 smithi161 bash[19769]: audit 2024-02-13T03:39:58.190215+0000 mon.a (mon.0) 3103 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:39:58.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:58 smithi183 bash[20824]: audit 2024-02-13T03:39:56.493093+0000 mgr.y (mgr.24473) 727 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:39:58.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:58 smithi183 bash[20824]: cluster 2024-02-13T03:39:57.181233+0000 mon.a (mon.0) 3101 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-02-13T03:39:58.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:58 smithi183 bash[20824]: cluster 2024-02-13T03:39:57.595971+0000 mgr.y (mgr.24473) 728 : cluster [DBG] pgmap v1200: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:39:58.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:58 smithi183 bash[20824]: cluster 2024-02-13T03:39:58.185067+0000 mon.a (mon.0) 3102 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-02-13T03:39:58.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:58 smithi183 bash[20824]: audit 2024-02-13T03:39:58.190215+0000 mon.a (mon.0) 3103 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:39:59.643 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:39:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:39:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:39:59.925 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:59 smithi161 bash[19769]: cluster 2024-02-13T03:39:59.084353+0000 mon.a (mon.0) 3104 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:39:59.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:59 smithi161 bash[19769]: audit 2024-02-13T03:39:59.189623+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:59.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:59 smithi161 bash[19769]: cluster 2024-02-13T03:39:59.189922+0000 mon.a (mon.0) 3106 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-02-13T03:39:59.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:59 smithi161 bash[19769]: audit 2024-02-13T03:39:59.232704+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:59.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:39:59 smithi161 bash[19769]: audit 2024-02-13T03:39:59.233182+0000 mon.a (mon.0) 3108 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-140"}]: dispatch 2024-02-13T03:39:59.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:59 smithi161 bash[27879]: cluster 2024-02-13T03:39:59.084353+0000 mon.a (mon.0) 3104 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:39:59.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:59 smithi161 bash[27879]: audit 2024-02-13T03:39:59.189623+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:59.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:59 smithi161 bash[27879]: cluster 2024-02-13T03:39:59.189922+0000 mon.a (mon.0) 3106 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-02-13T03:39:59.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:59 smithi161 bash[27879]: audit 2024-02-13T03:39:59.232704+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:59.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:39:59 smithi161 bash[27879]: audit 2024-02-13T03:39:59.233182+0000 mon.a (mon.0) 3108 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-140"}]: dispatch 2024-02-13T03:39:59.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:59 smithi183 bash[20824]: cluster 2024-02-13T03:39:59.084353+0000 mon.a (mon.0) 3104 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:39:59.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:59 smithi183 bash[20824]: audit 2024-02-13T03:39:59.189623+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:39:59.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:59 smithi183 bash[20824]: cluster 2024-02-13T03:39:59.189922+0000 mon.a (mon.0) 3106 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-02-13T03:39:59.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:59 smithi183 bash[20824]: audit 2024-02-13T03:39:59.232704+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:39:59.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:39:59 smithi183 bash[20824]: audit 2024-02-13T03:39:59.233182+0000 mon.a (mon.0) 3108 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-140"}]: dispatch 2024-02-13T03:40:00.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[27879]: cluster 2024-02-13T03:39:59.596765+0000 mgr.y (mgr.24473) 729 : cluster [DBG] pgmap v1203: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:40:00.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[27879]: cluster 2024-02-13T03:40:00.000122+0000 mon.a (mon.0) 3109 : cluster [WRN] Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-02-13T03:40:00.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[27879]: cluster 2024-02-13T03:40:00.000166+0000 mon.a (mon.0) 3110 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-13T03:40:00.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[27879]: cluster 2024-02-13T03:40:00.000189+0000 mon.a (mon.0) 3111 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-13T03:40:00.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[27879]: cluster 2024-02-13T03:40:00.000208+0000 mon.a (mon.0) 3112 : cluster [WRN] application not enabled on pool 'test-rados-api-smithi161-61906-101' 2024-02-13T03:40:00.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[27879]: cluster 2024-02-13T03:40:00.000252+0000 mon.a (mon.0) 3113 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-13T03:40:00.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[27879]: cluster 2024-02-13T03:40:00.197940+0000 mon.a (mon.0) 3114 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-02-13T03:40:00.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[19769]: cluster 2024-02-13T03:39:59.596765+0000 mgr.y (mgr.24473) 729 : cluster [DBG] pgmap v1203: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:40:00.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[19769]: cluster 2024-02-13T03:40:00.000122+0000 mon.a (mon.0) 3109 : cluster [WRN] Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-02-13T03:40:00.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[19769]: cluster 2024-02-13T03:40:00.000166+0000 mon.a (mon.0) 3110 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-13T03:40:00.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[19769]: cluster 2024-02-13T03:40:00.000189+0000 mon.a (mon.0) 3111 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-13T03:40:00.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[19769]: cluster 2024-02-13T03:40:00.000208+0000 mon.a (mon.0) 3112 : cluster [WRN] application not enabled on pool 'test-rados-api-smithi161-61906-101' 2024-02-13T03:40:00.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[19769]: cluster 2024-02-13T03:40:00.000252+0000 mon.a (mon.0) 3113 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-13T03:40:00.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:00 smithi161 bash[19769]: cluster 2024-02-13T03:40:00.197940+0000 mon.a (mon.0) 3114 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-02-13T03:40:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:00 smithi183 bash[20824]: cluster 2024-02-13T03:39:59.596765+0000 mgr.y (mgr.24473) 729 : cluster [DBG] pgmap v1203: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-13T03:40:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:00 smithi183 bash[20824]: cluster 2024-02-13T03:40:00.000122+0000 mon.a (mon.0) 3109 : cluster [WRN] Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-02-13T03:40:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:00 smithi183 bash[20824]: cluster 2024-02-13T03:40:00.000166+0000 mon.a (mon.0) 3110 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-13T03:40:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:00 smithi183 bash[20824]: cluster 2024-02-13T03:40:00.000189+0000 mon.a (mon.0) 3111 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-13T03:40:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:00 smithi183 bash[20824]: cluster 2024-02-13T03:40:00.000208+0000 mon.a (mon.0) 3112 : cluster [WRN] application not enabled on pool 'test-rados-api-smithi161-61906-101' 2024-02-13T03:40:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:00 smithi183 bash[20824]: cluster 2024-02-13T03:40:00.000252+0000 mon.a (mon.0) 3113 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-13T03:40:00.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:00 smithi183 bash[20824]: cluster 2024-02-13T03:40:00.197940+0000 mon.a (mon.0) 3114 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-02-13T03:40:01.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:40:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:40:01] "GET /metrics HTTP/1.1" 200 190395 "" "Prometheus/2.33.4" 2024-02-13T03:40:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:02 smithi183 bash[20824]: cluster 2024-02-13T03:40:01.200820+0000 mon.a (mon.0) 3115 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-02-13T03:40:02.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:02 smithi183 bash[20824]: audit 2024-02-13T03:40:01.206214+0000 mon.a (mon.0) 3116 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:40:02.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:02 smithi161 bash[27879]: cluster 2024-02-13T03:40:01.200820+0000 mon.a (mon.0) 3115 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-02-13T03:40:02.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:02 smithi161 bash[27879]: audit 2024-02-13T03:40:01.206214+0000 mon.a (mon.0) 3116 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:40:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:02 smithi161 bash[19769]: cluster 2024-02-13T03:40:01.200820+0000 mon.a (mon.0) 3115 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-02-13T03:40:02.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:02 smithi161 bash[19769]: audit 2024-02-13T03:40:01.206214+0000 mon.a (mon.0) 3116 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:40:03.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:03 smithi161 bash[27879]: cluster 2024-02-13T03:40:01.597625+0000 mgr.y (mgr.24473) 730 : cluster [DBG] pgmap v1206: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:40:03.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:03 smithi161 bash[27879]: audit 2024-02-13T03:40:02.204624+0000 mon.a (mon.0) 3117 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:40:03.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:03 smithi161 bash[27879]: cluster 2024-02-13T03:40:02.205234+0000 mon.a (mon.0) 3118 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-02-13T03:40:03.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:03 smithi161 bash[27879]: audit 2024-02-13T03:40:02.256228+0000 mon.a (mon.0) 3119 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:40:03.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:03 smithi161 bash[27879]: audit 2024-02-13T03:40:02.256814+0000 mon.a (mon.0) 3120 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-142"}]: dispatch 2024-02-13T03:40:03.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:03 smithi161 bash[19769]: cluster 2024-02-13T03:40:01.597625+0000 mgr.y (mgr.24473) 730 : cluster [DBG] pgmap v1206: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:40:03.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:03 smithi161 bash[19769]: audit 2024-02-13T03:40:02.204624+0000 mon.a (mon.0) 3117 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:40:03.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:03 smithi161 bash[19769]: cluster 2024-02-13T03:40:02.205234+0000 mon.a (mon.0) 3118 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-02-13T03:40:03.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:03 smithi161 bash[19769]: audit 2024-02-13T03:40:02.256228+0000 mon.a (mon.0) 3119 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:40:03.927 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:03 smithi161 bash[19769]: audit 2024-02-13T03:40:02.256814+0000 mon.a (mon.0) 3120 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-142"}]: dispatch 2024-02-13T03:40:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:03 smithi183 bash[20824]: cluster 2024-02-13T03:40:01.597625+0000 mgr.y (mgr.24473) 730 : cluster [DBG] pgmap v1206: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:40:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:03 smithi183 bash[20824]: audit 2024-02-13T03:40:02.204624+0000 mon.a (mon.0) 3117 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:40:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:03 smithi183 bash[20824]: cluster 2024-02-13T03:40:02.205234+0000 mon.a (mon.0) 3118 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-02-13T03:40:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:03 smithi183 bash[20824]: audit 2024-02-13T03:40:02.256228+0000 mon.a (mon.0) 3119 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:40:03.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:03 smithi183 bash[20824]: audit 2024-02-13T03:40:02.256814+0000 mon.a (mon.0) 3120 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-142"}]: dispatch 2024-02-13T03:40:04.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:04 smithi183 bash[20824]: cluster 2024-02-13T03:40:03.222557+0000 mon.a (mon.0) 3121 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-02-13T03:40:04.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:04 smithi183 bash[20824]: cluster 2024-02-13T03:40:03.598380+0000 mgr.y (mgr.24473) 731 : cluster [DBG] pgmap v1209: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:40:04.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:04 smithi183 bash[20824]: cluster 2024-02-13T03:40:04.226996+0000 mon.a (mon.0) 3122 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-02-13T03:40:04.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:04 smithi183 bash[20824]: audit 2024-02-13T03:40:04.231613+0000 mon.a (mon.0) 3123 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:40:05.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:04 smithi161 bash[27879]: cluster 2024-02-13T03:40:03.222557+0000 mon.a (mon.0) 3121 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-02-13T03:40:05.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:04 smithi161 bash[27879]: cluster 2024-02-13T03:40:03.598380+0000 mgr.y (mgr.24473) 731 : cluster [DBG] pgmap v1209: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:40:05.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:04 smithi161 bash[27879]: cluster 2024-02-13T03:40:04.226996+0000 mon.a (mon.0) 3122 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-02-13T03:40:05.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:04 smithi161 bash[27879]: audit 2024-02-13T03:40:04.231613+0000 mon.a (mon.0) 3123 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:40:05.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:04 smithi161 bash[19769]: cluster 2024-02-13T03:40:03.222557+0000 mon.a (mon.0) 3121 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-02-13T03:40:05.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:04 smithi161 bash[19769]: cluster 2024-02-13T03:40:03.598380+0000 mgr.y (mgr.24473) 731 : cluster [DBG] pgmap v1209: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T03:40:05.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:04 smithi161 bash[19769]: cluster 2024-02-13T03:40:04.226996+0000 mon.a (mon.0) 3122 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-02-13T03:40:05.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:04 smithi161 bash[19769]: audit 2024-02-13T03:40:04.231613+0000 mon.a (mon.0) 3123 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-13T03:40:06.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:05 smithi161 bash[19769]: audit 2024-02-13T03:40:04.798070+0000 mon.a (mon.0) 3124 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:40:06.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:05 smithi161 bash[19769]: audit 2024-02-13T03:40:05.229621+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:40:06.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:05 smithi161 bash[19769]: cluster 2024-02-13T03:40:05.230112+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-02-13T03:40:06.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:05 smithi161 bash[19769]: audit 2024-02-13T03:40:05.233670+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-101","var": "dedup_tier","val": "test-rados-api-smithi161-61906-144"}]: dispatch 2024-02-13T03:40:06.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:05 smithi161 bash[19769]: audit 2024-02-13T03:40:05.262983+0000 mon.a (mon.0) 3128 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:40:06.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:05 smithi161 bash[19769]: audit 2024-02-13T03:40:05.263328+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-144"}]: dispatch 2024-02-13T03:40:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:05 smithi161 bash[27879]: audit 2024-02-13T03:40:04.798070+0000 mon.a (mon.0) 3124 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:40:06.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:05 smithi161 bash[27879]: audit 2024-02-13T03:40:05.229621+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:40:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:05 smithi161 bash[27879]: cluster 2024-02-13T03:40:05.230112+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-02-13T03:40:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:05 smithi161 bash[27879]: audit 2024-02-13T03:40:05.233670+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-101","var": "dedup_tier","val": "test-rados-api-smithi161-61906-144"}]: dispatch 2024-02-13T03:40:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:05 smithi161 bash[27879]: audit 2024-02-13T03:40:05.262983+0000 mon.a (mon.0) 3128 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:40:06.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:05 smithi161 bash[27879]: audit 2024-02-13T03:40:05.263328+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-144"}]: dispatch 2024-02-13T03:40:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:05 smithi183 bash[20824]: audit 2024-02-13T03:40:04.798070+0000 mon.a (mon.0) 3124 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:40:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:05 smithi183 bash[20824]: audit 2024-02-13T03:40:05.229621+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi161-61906-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-13T03:40:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:05 smithi183 bash[20824]: cluster 2024-02-13T03:40:05.230112+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-02-13T03:40:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:05 smithi183 bash[20824]: audit 2024-02-13T03:40:05.233670+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi161-61906-101","var": "dedup_tier","val": "test-rados-api-smithi161-61906-144"}]: dispatch 2024-02-13T03:40:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:05 smithi183 bash[20824]: audit 2024-02-13T03:40:05.262983+0000 mon.a (mon.0) 3128 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:40:06.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:05 smithi183 bash[20824]: audit 2024-02-13T03:40:05.263328+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi161-61906-101", "tierpool": "test-rados-api-smithi161-61906-144"}]: dispatch 2024-02-13T03:40:06.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:40:06 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:40:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:07 smithi183 bash[20824]: cluster 2024-02-13T03:40:05.599350+0000 mgr.y (mgr.24473) 732 : cluster [DBG] pgmap v1212: 204 pgs: 27 unknown, 177 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:40:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:07 smithi183 bash[20824]: cluster 2024-02-13T03:40:06.230672+0000 mon.a (mon.0) 3130 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-02-13T03:40:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:07 smithi161 bash[27879]: cluster 2024-02-13T03:40:05.599350+0000 mgr.y (mgr.24473) 732 : cluster [DBG] pgmap v1212: 204 pgs: 27 unknown, 177 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:40:07.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:07 smithi161 bash[27879]: cluster 2024-02-13T03:40:06.230672+0000 mon.a (mon.0) 3130 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-02-13T03:40:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:07 smithi161 bash[19769]: cluster 2024-02-13T03:40:05.599350+0000 mgr.y (mgr.24473) 732 : cluster [DBG] pgmap v1212: 204 pgs: 27 unknown, 177 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:40:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:07 smithi161 bash[19769]: cluster 2024-02-13T03:40:06.230672+0000 mon.a (mon.0) 3130 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-02-13T03:40:08.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:08 smithi161 bash[19769]: audit 2024-02-13T03:40:06.501084+0000 mgr.y (mgr.24473) 733 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:08.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:08 smithi161 bash[19769]: cluster 2024-02-13T03:40:07.241714+0000 mon.a (mon.0) 3131 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-02-13T03:40:08.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:08 smithi161 bash[19769]: audit 2024-02-13T03:40:07.243501+0000 mon.a (mon.0) 3132 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:40:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:08 smithi161 bash[27879]: audit 2024-02-13T03:40:06.501084+0000 mgr.y (mgr.24473) 733 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:08 smithi161 bash[27879]: cluster 2024-02-13T03:40:07.241714+0000 mon.a (mon.0) 3131 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-02-13T03:40:08.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:08 smithi161 bash[27879]: audit 2024-02-13T03:40:07.243501+0000 mon.a (mon.0) 3132 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:40:08.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:08 smithi183 bash[20824]: audit 2024-02-13T03:40:06.501084+0000 mgr.y (mgr.24473) 733 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:08 smithi183 bash[20824]: cluster 2024-02-13T03:40:07.241714+0000 mon.a (mon.0) 3131 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-02-13T03:40:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:08 smithi183 bash[20824]: audit 2024-02-13T03:40:07.243501+0000 mon.a (mon.0) 3132 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:40:09.255 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9070 ms) 2024-02-13T03:40:09.256 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-02-13T03:40:09.256 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795524,0 2024-02-13T03:40:09.256 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: first is 1707795524 2024-02-13T03:40:09.256 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795524,0 2024-02-13T03:40:09.256 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795524,0 2024-02-13T03:40:09.256 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795524,1707795526,1707795527,0 2024-02-13T03:40:09.256 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795524,1707795526,1707795527,0 2024-02-13T03:40:09.256 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795524,1707795526,1707795527,0 2024-02-13T03:40:09.256 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795524,1707795526,1707795527,1707795529,1707795530,0 2024-02-13T03:40:09.256 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795524,1707795526,1707795527,1707795529,1707795530,0 2024-02-13T03:40:09.257 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795524,1707795526,1707795527,1707795529,1707795530,0 2024-02-13T03:40:09.257 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795524,1707795526,1707795527,1707795529,1707795530,1707795532,1707795533,0 2024-02-13T03:40:09.257 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795524,1707795526,1707795527,1707795529,1707795530,1707795532,1707795533,0 2024-02-13T03:40:09.257 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795524,1707795526,1707795527,1707795529,1707795530,1707795532,1707795533,0 2024-02-13T03:40:09.257 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: got ls 1707795527,1707795529,1707795530,1707795532,1707795533,1707795535,1707795536,0 2024-02-13T03:40:09.257 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: first now 1707795527, trimmed 2024-02-13T03:40:09.257 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (21106 ms) 2024-02-13T03:40:09.257 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-02-13T03:40:09.257 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: foo0 2024-02-13T03:40:09.257 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-13T03:40:09.257 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14115 ms) 2024-02-13T03:40:09.257 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-02-13T03:40:09.258 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17965 ms) 2024-02-13T03:40:09.258 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-02-13T03:40:09.258 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22766 ms) 2024-02-13T03:40:09.258 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-02-13T03:40:09.258 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5044 ms) 2024-02-13T03:40:09.258 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-02-13T03:40:09.258 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3016 ms) 2024-02-13T03:40:09.258 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-02-13T03:40:09.258 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3024 ms) 2024-02-13T03:40:09.258 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-02-13T03:40:09.258 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3008 ms) 2024-02-13T03:40:09.258 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (232801 ms total) 2024-02-13T03:40:09.259 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: 2024-02-13T03:40:09.259 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-02-13T03:40:09.259 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [==========] 72 tests from 4 test suites ran. (802957 ms total) 2024-02-13T03:40:09.259 INFO:tasks.workunit.client.0.smithi161.stdout: api_tier_pp: [ PASSED ] 72 tests. 2024-02-13T03:40:09.259 INFO:tasks.workunit.client.0.smithi161.stderr:+ exit 0 2024-02-13T03:40:09.260 INFO:tasks.workunit.client.0.smithi161.stderr:+ cleanup 2024-02-13T03:40:09.260 INFO:tasks.workunit.client.0.smithi161.stderr:+ pkill -P 61709 2024-02-13T03:40:09.266 INFO:tasks.workunit.client.0.smithi161.stderr:+ true 2024-02-13T03:40:09.267 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-13T03:40:09.267 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-13T03:40:09.279 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-02-13T03:40:09.280 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-02-13T03:40:09.280 DEBUG:teuthology.orchestra.run.smithi161:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=84e714279aa684636ac645daf0a1a85c2094cfb3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-02-13T03:40:09.359 INFO:tasks.workunit.client.0.smithi161.stderr:+ uuidgen 2024-02-13T03:40:09.360 INFO:tasks.workunit.client.0.smithi161.stderr:+ p=140c6d73-9bda-4f05-af7a-8f6a188328fa 2024-02-13T03:40:09.360 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool create 140c6d73-9bda-4f05-af7a-8f6a188328fa 12 2024-02-13T03:40:09.490 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081f56c700 1 -- 172.21.15.161:0/807643907 >> v1:172.21.15.161:6790/0 conn(0x7f08180f9550 legacy=0x7f0818102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:09.490 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081f56c700 1 -- 172.21.15.161:0/807643907 shutdown_connections 2024-02-13T03:40:09.490 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081f56c700 1 -- 172.21.15.161:0/807643907 >> 172.21.15.161:0/807643907 conn(0x7f08180f2570 msgr2=0x7f08180f4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:40:09.490 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081f56c700 1 -- 172.21.15.161:0/807643907 shutdown_connections 2024-02-13T03:40:09.490 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081f56c700 1 -- 172.21.15.161:0/807643907 wait complete. 2024-02-13T03:40:09.491 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081f56c700 1 Processor -- start 2024-02-13T03:40:09.491 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081f56c700 1 -- start start 2024-02-13T03:40:09.492 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081f56c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0818102f20 con 0x7f0818104330 2024-02-13T03:40:09.492 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081f56c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0818118890 con 0x7f08180f9550 2024-02-13T03:40:09.492 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081f56c700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0818119a70 con 0x7f08180f8a60 2024-02-13T03:40:09.492 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081d308700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f08180f8a60 0x7f0818102370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:43034/0 (socket says 172.21.15.161:43034) 2024-02-13T03:40:09.492 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081d308700 1 -- 172.21.15.161:0/3891208437 learned_addr learned my addr 172.21.15.161:0/3891208437 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:40:09.493 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233403119 0 0) 0x7f0818119a70 con 0x7f08180f8a60 2024-02-13T03:40:09.494 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08040035f0 con 0x7f08180f8a60 2024-02-13T03:40:09.494 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771117687 0 0) 0x7f0818102f20 con 0x7f0818104330 2024-02-13T03:40:09.494 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0818119a70 con 0x7f0818104330 2024-02-13T03:40:09.494 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527547762 0 0) 0x7f0818118890 con 0x7f08180f9550 2024-02-13T03:40:09.494 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0818102f20 con 0x7f08180f9550 2024-02-13T03:40:09.494 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4154039114 0 0) 0x7f08040035f0 con 0x7f08180f8a60 2024-02-13T03:40:09.494 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0818118890 con 0x7f08180f8a60 2024-02-13T03:40:09.494 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f0814001940 con 0x7f08180f8a60 2024-02-13T03:40:09.494 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3489496636 0 0) 0x7f0818119a70 con 0x7f0818104330 2024-02-13T03:40:09.494 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f08040035f0 con 0x7f0818104330 2024-02-13T03:40:09.494 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f0810001cf0 con 0x7f0818104330 2024-02-13T03:40:09.494 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3045735664 0 0) 0x7f08040035f0 con 0x7f0818104330 2024-02-13T03:40:09.494 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 >> v1:172.21.15.161:6790/0 conn(0x7f08180f8a60 legacy=0x7f0818102370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:09.495 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 >> v1:172.21.15.183:6789/0 conn(0x7f08180f9550 legacy=0x7f0818112a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:09.495 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081811ac50 con 0x7f0818104330 2024-02-13T03:40:09.495 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081f56c700 1 -- 172.21.15.161:0/3891208437 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0818119c60 con 0x7f0818104330 2024-02-13T03:40:09.495 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f081f56c700 1 -- 172.21.15.161:0/3891208437 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f081811a0b0 con 0x7f0818104330 2024-02-13T03:40:09.495 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f0810002720 con 0x7f0818104330 2024-02-13T03:40:09.495 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.488+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f08100035e0 con 0x7f0818104330 2024-02-13T03:40:09.495 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.492+0000 7f081f56c700 1 -- 172.21.15.161:0/3891208437 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07fc004ee0 con 0x7f0818104330 2024-02-13T03:40:09.495 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.492+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f08100164f0 con 0x7f0818104330 2024-02-13T03:40:09.496 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.492+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(718..718 src has 1..718) v4 ==== 7198+0+0 (unknown 1093227111 0 0) 0x7f0810076520 con 0x7f0818104330 2024-02-13T03:40:09.506 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.504+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f0810049c70 con 0x7f0818104330 2024-02-13T03:40:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:09 smithi161 bash[19769]: cluster 2024-02-13T03:40:07.600103+0000 mgr.y (mgr.24473) 734 : cluster [DBG] pgmap v1215: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:40:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:09 smithi161 bash[19769]: cluster 2024-02-13T03:40:08.237274+0000 mon.a (mon.0) 3133 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:40:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:09 smithi161 bash[19769]: audit 2024-02-13T03:40:08.244531+0000 mon.a (mon.0) 3134 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:40:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:09 smithi161 bash[19769]: cluster 2024-02-13T03:40:08.245175+0000 mon.a (mon.0) 3135 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-02-13T03:40:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:09 smithi161 bash[19769]: audit 2024-02-13T03:40:08.246803+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:40:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:09 smithi161 bash[19769]: audit 2024-02-13T03:40:09.248153+0000 mon.a (mon.0) 3137 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:40:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:09 smithi161 bash[27879]: cluster 2024-02-13T03:40:07.600103+0000 mgr.y (mgr.24473) 734 : cluster [DBG] pgmap v1215: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:40:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:09 smithi161 bash[27879]: cluster 2024-02-13T03:40:08.237274+0000 mon.a (mon.0) 3133 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:40:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:09 smithi161 bash[27879]: audit 2024-02-13T03:40:08.244531+0000 mon.a (mon.0) 3134 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:40:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:09 smithi161 bash[27879]: cluster 2024-02-13T03:40:08.245175+0000 mon.a (mon.0) 3135 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-02-13T03:40:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:09 smithi161 bash[27879]: audit 2024-02-13T03:40:08.246803+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:40:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:09 smithi161 bash[27879]: audit 2024-02-13T03:40:09.248153+0000 mon.a (mon.0) 3137 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:40:09.683 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:09.680+0000 7f081f56c700 1 -- 172.21.15.161:0/3891208437 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pg_num": 12} v 0) v1 -- 0x7f07fc005710 con 0x7f0818104330 2024-02-13T03:40:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:09 smithi183 bash[20824]: cluster 2024-02-13T03:40:07.600103+0000 mgr.y (mgr.24473) 734 : cluster [DBG] pgmap v1215: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:40:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:09 smithi183 bash[20824]: cluster 2024-02-13T03:40:08.237274+0000 mon.a (mon.0) 3133 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T03:40:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:09 smithi183 bash[20824]: audit 2024-02-13T03:40:08.244531+0000 mon.a (mon.0) 3134 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:40:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:09 smithi183 bash[20824]: cluster 2024-02-13T03:40:08.245175+0000 mon.a (mon.0) 3135 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-02-13T03:40:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:09 smithi183 bash[20824]: audit 2024-02-13T03:40:08.246803+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-101"}]: dispatch 2024-02-13T03:40:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:09 smithi183 bash[20824]: audit 2024-02-13T03:40:09.248153+0000 mon.a (mon.0) 3137 : audit [INF] from='client.? v1:172.21.15.161:0/2853167347' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi161-61906-101"}]': finished 2024-02-13T03:40:09.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:40:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:40:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:40:10.269 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.264+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pg_num": 12}]=0 pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' created v719) v1 ==== 176+0+0 (unknown 3405909441 0 0) 0x7f08100480e0 con 0x7f0818104330 2024-02-13T03:40:10.385 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.380+0000 7f081f56c700 1 -- 172.21.15.161:0/3891208437 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pg_num": 12} v 0) v1 -- 0x7f07fc0023c0 con 0x7f0818104330 2024-02-13T03:40:10.386 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.384+0000 7f080e7fc700 1 -- 172.21.15.161:0/3891208437 <== mon.0 v1:172.21.15.161:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pg_num": 12}]=0 pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' already exists v719) v1 ==== 183+0+0 (unknown 432506 0 0) 0x7f08100480e0 con 0x7f0818104330 2024-02-13T03:40:10.388 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.384+0000 7f081f56c700 1 -- 172.21.15.161:0/3891208437 >> v1:172.21.15.161:6800/4114337139 conn(0x7f0804061fb0 legacy=0x7f0804064470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:10.388 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.384+0000 7f081f56c700 1 -- 172.21.15.161:0/3891208437 >> v1:172.21.15.161:6789/0 conn(0x7f0818104330 legacy=0x7f0818116160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:10.388 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.384+0000 7f081f56c700 1 -- 172.21.15.161:0/3891208437 shutdown_connections 2024-02-13T03:40:10.389 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.384+0000 7f081f56c700 1 -- 172.21.15.161:0/3891208437 >> 172.21.15.161:0/3891208437 conn(0x7f08180f2570 msgr2=0x7f08181077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:40:10.389 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.384+0000 7f081f56c700 1 -- 172.21.15.161:0/3891208437 shutdown_connections 2024-02-13T03:40:10.389 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.384+0000 7f081f56c700 1 -- 172.21.15.161:0/3891208437 wait complete. 2024-02-13T03:40:10.389 INFO:tasks.workunit.client.0.smithi161.stderr:pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' already exists 2024-02-13T03:40:10.399 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool set-quota 140c6d73-9bda-4f05-af7a-8f6a188328fa max_objects 10 2024-02-13T03:40:10.526 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa870ed6700 1 -- 172.21.15.161:0/4074952939 >> v1:172.21.15.161:6789/0 conn(0x7fa86c106630 legacy=0x7fa86c108a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:10.526 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa870ed6700 1 -- 172.21.15.161:0/4074952939 shutdown_connections 2024-02-13T03:40:10.526 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa870ed6700 1 -- 172.21.15.161:0/4074952939 >> 172.21.15.161:0/4074952939 conn(0x7fa86c0f2570 msgr2=0x7fa86c0f4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:40:10.526 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa870ed6700 1 -- 172.21.15.161:0/4074952939 shutdown_connections 2024-02-13T03:40:10.526 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa870ed6700 1 -- 172.21.15.161:0/4074952939 wait complete. 2024-02-13T03:40:10.528 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa870ed6700 1 Processor -- start 2024-02-13T03:40:10.528 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa870ed6700 1 -- start start 2024-02-13T03:40:10.528 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa870ed6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa86c119b80 con 0x7fa86c0f67f0 2024-02-13T03:40:10.528 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa870ed6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa86c11ad20 con 0x7fa86c0f72e0 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa870ed6700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa86c11bec0 con 0x7fa86c106630 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa86a59c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa86c0f67f0 0x7fa86c10ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:47526/0 (socket says 172.21.15.161:47526) 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa86a59c700 1 -- 172.21.15.161:0/2342977765 learned_addr learned my addr 172.21.15.161:0/2342977765 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290161878 0 0) 0x7fa86c119b80 con 0x7fa86c0f67f0 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8580035f0 con 0x7fa86c0f67f0 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488603949 0 0) 0x7fa86c11bec0 con 0x7fa86c106630 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa86c119b80 con 0x7fa86c106630 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1344472165 0 0) 0x7fa86c119b80 con 0x7fa86c106630 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa86c11bec0 con 0x7fa86c106630 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fa864002fa0 con 0x7fa86c106630 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349075547 0 0) 0x7fa86c11ad20 con 0x7fa86c0f72e0 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa86c119b80 con 0x7fa86c0f72e0 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3260519163 0 0) 0x7fa8580035f0 con 0x7fa86c0f67f0 2024-02-13T03:40:10.529 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa86c11ad20 con 0x7fa86c0f67f0 2024-02-13T03:40:10.530 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fa85c002910 con 0x7fa86c0f67f0 2024-02-13T03:40:10.530 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1959172989 0 0) 0x7fa86c11bec0 con 0x7fa86c106630 2024-02-13T03:40:10.530 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 >> v1:172.21.15.183:6789/0 conn(0x7fa86c0f72e0 legacy=0x7fa86c114d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:10.530 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 >> v1:172.21.15.161:6789/0 conn(0x7fa86c0f67f0 legacy=0x7fa86c10ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:10.530 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c11d060 con 0x7fa86c106630 2024-02-13T03:40:10.530 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa870ed6700 1 -- 172.21.15.161:0/2342977765 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c11c0b0 con 0x7fa86c106630 2024-02-13T03:40:10.530 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa870ed6700 1 -- 172.21.15.161:0/2342977765 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c11c500 con 0x7fa86c106630 2024-02-13T03:40:10.530 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fa864003220 con 0x7fa86c106630 2024-02-13T03:40:10.530 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.524+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fa864003ae0 con 0x7fa86c106630 2024-02-13T03:40:10.531 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.528+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fa864002340 con 0x7fa86c106630 2024-02-13T03:40:10.531 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.528+0000 7fa870ed6700 1 -- 172.21.15.161:0/2342977765 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84c004ee0 con 0x7fa86c106630 2024-02-13T03:40:10.531 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.528+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(719..719 src has 1..719) v4 ==== 7573+0+0 (unknown 3431260631 0 0) 0x7fa864076990 con 0x7fa86c106630 2024-02-13T03:40:10.541 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.536+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fa864049f70 con 0x7fa86c106630 2024-02-13T03:40:10.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:10 smithi161 bash[19769]: cluster 2024-02-13T03:40:09.248312+0000 mon.a (mon.0) 3138 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-02-13T03:40:10.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:10 smithi161 bash[19769]: audit 2024-02-13T03:40:09.683867+0000 mon.a (mon.0) 3139 : audit [INF] from='client.? v1:172.21.15.161:0/3891208437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pg_num": 12}]: dispatch 2024-02-13T03:40:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:10 smithi161 bash[27879]: cluster 2024-02-13T03:40:09.248312+0000 mon.a (mon.0) 3138 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-02-13T03:40:10.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:10 smithi161 bash[27879]: audit 2024-02-13T03:40:09.683867+0000 mon.a (mon.0) 3139 : audit [INF] from='client.? v1:172.21.15.161:0/3891208437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pg_num": 12}]: dispatch 2024-02-13T03:40:10.701 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:10.696+0000 7fa870ed6700 1 -- 172.21.15.161:0/2342977765 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fa84c005190 con 0x7fa86c106630 2024-02-13T03:40:10.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:10 smithi183 bash[20824]: cluster 2024-02-13T03:40:09.248312+0000 mon.a (mon.0) 3138 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-02-13T03:40:10.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:10 smithi183 bash[20824]: audit 2024-02-13T03:40:09.683867+0000 mon.a (mon.0) 3139 : audit [INF] from='client.? v1:172.21.15.161:0/3891208437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pg_num": 12}]: dispatch 2024-02-13T03:40:11.277 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:11.272+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.2 v1:172.21.15.161:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v720) v1 ==== 223+0+0 (unknown 376852140 0 0) 0x7fa8640483e0 con 0x7fa86c106630 2024-02-13T03:40:11.391 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:11.388+0000 7fa870ed6700 1 -- 172.21.15.161:0/2342977765 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fa84c002220 con 0x7fa86c106630 2024-02-13T03:40:11.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:11 smithi161 bash[19769]: cluster 2024-02-13T03:40:09.600864+0000 mgr.y (mgr.24473) 735 : cluster [DBG] pgmap v1218: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:40:11.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:11 smithi161 bash[19769]: audit 2024-02-13T03:40:10.268853+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? v1:172.21.15.161:0/3891208437' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pg_num": 12}]': finished 2024-02-13T03:40:11.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:11 smithi161 bash[19769]: cluster 2024-02-13T03:40:10.268965+0000 mon.a (mon.0) 3141 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-02-13T03:40:11.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:11 smithi161 bash[19769]: audit 2024-02-13T03:40:10.385656+0000 mon.a (mon.0) 3142 : audit [INF] from='client.? v1:172.21.15.161:0/3891208437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pg_num": 12}]: dispatch 2024-02-13T03:40:11.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:11 smithi161 bash[19769]: audit 2024-02-13T03:40:10.700995+0000 mon.c (mon.2) 578 : audit [INF] from='client.? v1:172.21.15.161:0/2342977765' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:40:11.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:11 smithi161 bash[19769]: audit 2024-02-13T03:40:10.701491+0000 mon.a (mon.0) 3143 : audit [INF] from='client.48404 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:40:11.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:40:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:40:11] "GET /metrics HTTP/1.1" 200 190476 "" "Prometheus/2.33.4" 2024-02-13T03:40:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:11 smithi161 bash[27879]: cluster 2024-02-13T03:40:09.600864+0000 mgr.y (mgr.24473) 735 : cluster [DBG] pgmap v1218: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:40:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:11 smithi161 bash[27879]: audit 2024-02-13T03:40:10.268853+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? v1:172.21.15.161:0/3891208437' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pg_num": 12}]': finished 2024-02-13T03:40:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:11 smithi161 bash[27879]: cluster 2024-02-13T03:40:10.268965+0000 mon.a (mon.0) 3141 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-02-13T03:40:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:11 smithi161 bash[27879]: audit 2024-02-13T03:40:10.385656+0000 mon.a (mon.0) 3142 : audit [INF] from='client.? v1:172.21.15.161:0/3891208437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pg_num": 12}]: dispatch 2024-02-13T03:40:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:11 smithi161 bash[27879]: audit 2024-02-13T03:40:10.700995+0000 mon.c (mon.2) 578 : audit [INF] from='client.? v1:172.21.15.161:0/2342977765' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:40:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:11 smithi161 bash[27879]: audit 2024-02-13T03:40:10.701491+0000 mon.a (mon.0) 3143 : audit [INF] from='client.48404 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:40:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:11 smithi183 bash[20824]: cluster 2024-02-13T03:40:09.600864+0000 mgr.y (mgr.24473) 735 : cluster [DBG] pgmap v1218: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:40:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:11 smithi183 bash[20824]: audit 2024-02-13T03:40:10.268853+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? v1:172.21.15.161:0/3891208437' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pg_num": 12}]': finished 2024-02-13T03:40:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:11 smithi183 bash[20824]: cluster 2024-02-13T03:40:10.268965+0000 mon.a (mon.0) 3141 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-02-13T03:40:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:11 smithi183 bash[20824]: audit 2024-02-13T03:40:10.385656+0000 mon.a (mon.0) 3142 : audit [INF] from='client.? v1:172.21.15.161:0/3891208437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pg_num": 12}]: dispatch 2024-02-13T03:40:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:11 smithi183 bash[20824]: audit 2024-02-13T03:40:10.700995+0000 mon.c (mon.2) 578 : audit [INF] from='client.? v1:172.21.15.161:0/2342977765' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:40:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:11 smithi183 bash[20824]: audit 2024-02-13T03:40:10.701491+0000 mon.a (mon.0) 3143 : audit [INF] from='client.48404 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:40:12.277 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.272+0000 7fa8637fe700 1 -- 172.21.15.161:0/2342977765 <== mon.2 v1:172.21.15.161:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v721) v1 ==== 223+0+0 (unknown 1597238236 0 0) 0x7fa864049b60 con 0x7fa86c106630 2024-02-13T03:40:12.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.276+0000 7fa870ed6700 1 -- 172.21.15.161:0/2342977765 >> v1:172.21.15.161:6800/4114337139 conn(0x7fa858062260 legacy=0x7fa858064720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:12.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.276+0000 7fa870ed6700 1 -- 172.21.15.161:0/2342977765 >> v1:172.21.15.161:6790/0 conn(0x7fa86c106630 legacy=0x7fa86c118460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:12.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.276+0000 7fa870ed6700 1 -- 172.21.15.161:0/2342977765 shutdown_connections 2024-02-13T03:40:12.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.276+0000 7fa870ed6700 1 -- 172.21.15.161:0/2342977765 >> 172.21.15.161:0/2342977765 conn(0x7fa86c0f2570 msgr2=0x7fa86c105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:40:12.281 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.276+0000 7fa870ed6700 1 -- 172.21.15.161:0/2342977765 shutdown_connections 2024-02-13T03:40:12.281 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.276+0000 7fa870ed6700 1 -- 172.21.15.161:0/2342977765 wait complete. 2024-02-13T03:40:12.281 INFO:tasks.workunit.client.0.smithi161.stderr:set-quota max_objects = 10 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa 2024-02-13T03:40:12.295 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool application enable 140c6d73-9bda-4f05-af7a-8f6a188328fa rados 2024-02-13T03:40:12.422 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fceffd700 1 -- 172.21.15.161:0/778343578 <== mon.0 v1:172.21.15.161:6789/0 5 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5fb8002ea0 con 0x7f5fd00feeb0 2024-02-13T03:40:12.422 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fd663c700 1 -- 172.21.15.161:0/778343578 >> v1:172.21.15.161:6789/0 conn(0x7f5fd00feeb0 legacy=0x7f5fd01012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:12.422 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fd663c700 1 -- 172.21.15.161:0/778343578 shutdown_connections 2024-02-13T03:40:12.423 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fd663c700 1 -- 172.21.15.161:0/778343578 >> 172.21.15.161:0/778343578 conn(0x7f5fd006c1a0 msgr2=0x7f5fd006c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:40:12.423 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fd663c700 1 -- 172.21.15.161:0/778343578 shutdown_connections 2024-02-13T03:40:12.423 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fd663c700 1 -- 172.21.15.161:0/778343578 wait complete. 2024-02-13T03:40:12.423 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fd663c700 1 Processor -- start 2024-02-13T03:40:12.423 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fd663c700 1 -- start start 2024-02-13T03:40:12.426 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fd663c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5fd0119690 con 0x7f5fd01029c0 2024-02-13T03:40:12.426 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fd663c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5fd011a890 con 0x7f5fd00feeb0 2024-02-13T03:40:12.426 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fd663c700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5fd011ba90 con 0x7f5fd00fc060 2024-02-13T03:40:12.426 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcffff700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f5fd00fc060 0x7f5fd00fb570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:43070/0 (socket says 172.21.15.161:43070) 2024-02-13T03:40:12.426 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcffff700 1 -- 172.21.15.161:0/310799007 learned_addr learned my addr 172.21.15.161:0/310799007 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:40:12.426 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197155434 0 0) 0x7f5fd011ba90 con 0x7f5fd00fc060 2024-02-13T03:40:12.426 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fbc0033b0 con 0x7f5fd00fc060 2024-02-13T03:40:12.426 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721265051 0 0) 0x7f5fd0119690 con 0x7f5fd01029c0 2024-02-13T03:40:12.426 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd011ba90 con 0x7f5fd01029c0 2024-02-13T03:40:12.427 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3917794383 0 0) 0x7f5fd011a890 con 0x7f5fd00feeb0 2024-02-13T03:40:12.427 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd0119690 con 0x7f5fd00feeb0 2024-02-13T03:40:12.427 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1619028132 0 0) 0x7f5fbc0033b0 con 0x7f5fd00fc060 2024-02-13T03:40:12.427 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5fd011a890 con 0x7f5fd00fc060 2024-02-13T03:40:12.427 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5fc0002950 con 0x7f5fd00fc060 2024-02-13T03:40:12.427 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2100656186 0 0) 0x7f5fd011ba90 con 0x7f5fd01029c0 2024-02-13T03:40:12.427 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5fbc0033b0 con 0x7f5fd01029c0 2024-02-13T03:40:12.427 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5fc8001b40 con 0x7f5fd01029c0 2024-02-13T03:40:12.427 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2117874555 0 0) 0x7f5fd011a890 con 0x7f5fd00fc060 2024-02-13T03:40:12.427 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 >> v1:172.21.15.183:6789/0 conn(0x7f5fd00feeb0 legacy=0x7f5fd0114420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:12.427 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 >> v1:172.21.15.161:6789/0 conn(0x7f5fd01029c0 legacy=0x7f5fd0117d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:12.427 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd011cc90 con 0x7f5fd00fc060 2024-02-13T03:40:12.427 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.420+0000 7f5fd663c700 1 -- 172.21.15.161:0/310799007 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd0119880 con 0x7f5fd00fc060 2024-02-13T03:40:12.428 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.424+0000 7f5fd663c700 1 -- 172.21.15.161:0/310799007 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd0119cd0 con 0x7f5fd00fc060 2024-02-13T03:40:12.428 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.424+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f5fc0001830 con 0x7f5fd00fc060 2024-02-13T03:40:12.428 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.424+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5fc0005a10 con 0x7f5fd00fc060 2024-02-13T03:40:12.428 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.424+0000 7f5fd663c700 1 -- 172.21.15.161:0/310799007 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb0004ee0 con 0x7f5fd00fc060 2024-02-13T03:40:12.428 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.424+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f5fc0001390 con 0x7f5fd00fc060 2024-02-13T03:40:12.429 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.424+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(721..721 src has 1..721) v4 ==== 7573+0+0 (unknown 1757960595 0 0) 0x7f5fc0078a40 con 0x7f5fd00fc060 2024-02-13T03:40:12.437 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.432+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f5fc004c020 con 0x7f5fd00fc060 2024-02-13T03:40:12.617 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:12.612+0000 7f5fd663c700 1 -- 172.21.15.161:0/310799007 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"} v 0) v1 -- 0x7f5fb0005190 con 0x7f5fd00fc060 2024-02-13T03:40:12.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:12 smithi161 bash[19769]: audit 2024-02-13T03:40:11.275308+0000 mon.a (mon.0) 3144 : audit [INF] from='client.48404 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]': finished 2024-02-13T03:40:12.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:12 smithi161 bash[19769]: cluster 2024-02-13T03:40:11.275509+0000 mon.a (mon.0) 3145 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-02-13T03:40:12.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:12 smithi161 bash[19769]: audit 2024-02-13T03:40:11.391275+0000 mon.c (mon.2) 579 : audit [INF] from='client.? v1:172.21.15.161:0/2342977765' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:40:12.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:12 smithi161 bash[19769]: audit 2024-02-13T03:40:11.392009+0000 mon.a (mon.0) 3146 : audit [INF] from='client.48404 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:40:12.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:12 smithi161 bash[19769]: audit 2024-02-13T03:40:12.275266+0000 mon.a (mon.0) 3147 : audit [INF] from='client.48404 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]': finished 2024-02-13T03:40:12.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:12 smithi161 bash[19769]: cluster 2024-02-13T03:40:12.275494+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-02-13T03:40:12.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:12 smithi161 bash[27879]: audit 2024-02-13T03:40:11.275308+0000 mon.a (mon.0) 3144 : audit [INF] from='client.48404 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]': finished 2024-02-13T03:40:12.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:12 smithi161 bash[27879]: cluster 2024-02-13T03:40:11.275509+0000 mon.a (mon.0) 3145 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-02-13T03:40:12.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:12 smithi161 bash[27879]: audit 2024-02-13T03:40:11.391275+0000 mon.c (mon.2) 579 : audit [INF] from='client.? v1:172.21.15.161:0/2342977765' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:40:12.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:12 smithi161 bash[27879]: audit 2024-02-13T03:40:11.392009+0000 mon.a (mon.0) 3146 : audit [INF] from='client.48404 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:40:12.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:12 smithi161 bash[27879]: audit 2024-02-13T03:40:12.275266+0000 mon.a (mon.0) 3147 : audit [INF] from='client.48404 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]': finished 2024-02-13T03:40:12.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:12 smithi161 bash[27879]: cluster 2024-02-13T03:40:12.275494+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-02-13T03:40:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:12 smithi183 bash[20824]: audit 2024-02-13T03:40:11.275308+0000 mon.a (mon.0) 3144 : audit [INF] from='client.48404 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]': finished 2024-02-13T03:40:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:12 smithi183 bash[20824]: cluster 2024-02-13T03:40:11.275509+0000 mon.a (mon.0) 3145 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-02-13T03:40:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:12 smithi183 bash[20824]: audit 2024-02-13T03:40:11.391275+0000 mon.c (mon.2) 579 : audit [INF] from='client.? v1:172.21.15.161:0/2342977765' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:40:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:12 smithi183 bash[20824]: audit 2024-02-13T03:40:11.392009+0000 mon.a (mon.0) 3146 : audit [INF] from='client.48404 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:40:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:12 smithi183 bash[20824]: audit 2024-02-13T03:40:12.275266+0000 mon.a (mon.0) 3147 : audit [INF] from='client.48404 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "10"}]': finished 2024-02-13T03:40:12.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:12 smithi183 bash[20824]: cluster 2024-02-13T03:40:12.275494+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-02-13T03:40:13.299 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:13.296+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.2 v1:172.21.15.161:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]=0 enabled application 'rados' on pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' v722) v1 ==== 213+0+0 (unknown 4000445203 0 0) 0x7f5fc004a430 con 0x7f5fd00fc060 2024-02-13T03:40:13.419 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:13.416+0000 7f5fd663c700 1 -- 172.21.15.161:0/310799007 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"} v 0) v1 -- 0x7f5fb0002220 con 0x7f5fd00fc060 2024-02-13T03:40:13.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:13 smithi161 bash[19769]: cluster 2024-02-13T03:40:11.601690+0000 mgr.y (mgr.24473) 736 : cluster [DBG] pgmap v1221: 176 pgs: 12 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:13.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:13 smithi161 bash[19769]: audit 2024-02-13T03:40:12.616902+0000 mon.c (mon.2) 580 : audit [INF] from='client.? v1:172.21.15.161:0/310799007' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]: dispatch 2024-02-13T03:40:13.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:13 smithi161 bash[19769]: audit 2024-02-13T03:40:12.617635+0000 mon.a (mon.0) 3149 : audit [INF] from='client.48410 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]: dispatch 2024-02-13T03:40:13.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:13 smithi161 bash[27879]: cluster 2024-02-13T03:40:11.601690+0000 mgr.y (mgr.24473) 736 : cluster [DBG] pgmap v1221: 176 pgs: 12 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:13.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:13 smithi161 bash[27879]: audit 2024-02-13T03:40:12.616902+0000 mon.c (mon.2) 580 : audit [INF] from='client.? v1:172.21.15.161:0/310799007' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]: dispatch 2024-02-13T03:40:13.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:13 smithi161 bash[27879]: audit 2024-02-13T03:40:12.617635+0000 mon.a (mon.0) 3149 : audit [INF] from='client.48410 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]: dispatch 2024-02-13T03:40:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:13 smithi183 bash[20824]: cluster 2024-02-13T03:40:11.601690+0000 mgr.y (mgr.24473) 736 : cluster [DBG] pgmap v1221: 176 pgs: 12 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:13 smithi183 bash[20824]: audit 2024-02-13T03:40:12.616902+0000 mon.c (mon.2) 580 : audit [INF] from='client.? v1:172.21.15.161:0/310799007' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]: dispatch 2024-02-13T03:40:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:13 smithi183 bash[20824]: audit 2024-02-13T03:40:12.617635+0000 mon.a (mon.0) 3149 : audit [INF] from='client.48410 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]: dispatch 2024-02-13T03:40:14.323 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:14.320+0000 7f5fcd7fa700 1 -- 172.21.15.161:0/310799007 <== mon.2 v1:172.21.15.161:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]=0 enabled application 'rados' on pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' v723) v1 ==== 213+0+0 (unknown 3562014197 0 0) 0x7f5fc004bbb0 con 0x7f5fd00fc060 2024-02-13T03:40:14.326 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:14.324+0000 7f5fd663c700 1 -- 172.21.15.161:0/310799007 >> v1:172.21.15.161:6800/4114337139 conn(0x7f5fbc061fc0 legacy=0x7f5fbc064480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:14.326 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:14.324+0000 7f5fd663c700 1 -- 172.21.15.161:0/310799007 >> v1:172.21.15.161:6790/0 conn(0x7f5fd00fc060 legacy=0x7f5fd00fb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:14.326 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:14.324+0000 7f5fd663c700 1 -- 172.21.15.161:0/310799007 shutdown_connections 2024-02-13T03:40:14.326 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:14.324+0000 7f5fd663c700 1 -- 172.21.15.161:0/310799007 >> 172.21.15.161:0/310799007 conn(0x7f5fd006c1a0 msgr2=0x7f5fd006c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:40:14.326 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:14.324+0000 7f5fd663c700 1 -- 172.21.15.161:0/310799007 shutdown_connections 2024-02-13T03:40:14.326 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:14.324+0000 7f5fd663c700 1 -- 172.21.15.161:0/310799007 wait complete. 2024-02-13T03:40:14.327 INFO:tasks.workunit.client.0.smithi161.stderr:enabled application 'rados' on pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' 2024-02-13T03:40:14.339 INFO:tasks.workunit.client.0.smithi161.stderr:+ seq 1 10 2024-02-13T03:40:14.358 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put obj1 /etc/passwd 2024-02-13T03:40:14.412 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put obj2 /etc/passwd 2024-02-13T03:40:14.455 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put obj3 /etc/passwd 2024-02-13T03:40:14.493 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put obj4 /etc/passwd 2024-02-13T03:40:14.536 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put obj5 /etc/passwd 2024-02-13T03:40:14.578 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put obj6 /etc/passwd 2024-02-13T03:40:14.619 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put obj7 /etc/passwd 2024-02-13T03:40:14.658 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put obj8 /etc/passwd 2024-02-13T03:40:14.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:14 smithi161 bash[19769]: audit 2024-02-13T03:40:13.297263+0000 mon.a (mon.0) 3150 : audit [INF] from='client.48410 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]': finished 2024-02-13T03:40:14.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:14 smithi161 bash[19769]: cluster 2024-02-13T03:40:13.297365+0000 mon.a (mon.0) 3151 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-02-13T03:40:14.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:14 smithi161 bash[19769]: audit 2024-02-13T03:40:13.419382+0000 mon.c (mon.2) 581 : audit [INF] from='client.? v1:172.21.15.161:0/310799007' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]: dispatch 2024-02-13T03:40:14.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:14 smithi161 bash[19769]: audit 2024-02-13T03:40:13.420145+0000 mon.a (mon.0) 3152 : audit [INF] from='client.48410 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]: dispatch 2024-02-13T03:40:14.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:14 smithi161 bash[27879]: audit 2024-02-13T03:40:13.297263+0000 mon.a (mon.0) 3150 : audit [INF] from='client.48410 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]': finished 2024-02-13T03:40:14.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:14 smithi161 bash[27879]: cluster 2024-02-13T03:40:13.297365+0000 mon.a (mon.0) 3151 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-02-13T03:40:14.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:14 smithi161 bash[27879]: audit 2024-02-13T03:40:13.419382+0000 mon.c (mon.2) 581 : audit [INF] from='client.? v1:172.21.15.161:0/310799007' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]: dispatch 2024-02-13T03:40:14.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:14 smithi161 bash[27879]: audit 2024-02-13T03:40:13.420145+0000 mon.a (mon.0) 3152 : audit [INF] from='client.48410 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]: dispatch 2024-02-13T03:40:14.698 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put obj9 /etc/passwd 2024-02-13T03:40:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:14 smithi183 bash[20824]: audit 2024-02-13T03:40:13.297263+0000 mon.a (mon.0) 3150 : audit [INF] from='client.48410 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]': finished 2024-02-13T03:40:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:14 smithi183 bash[20824]: cluster 2024-02-13T03:40:13.297365+0000 mon.a (mon.0) 3151 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-02-13T03:40:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:14 smithi183 bash[20824]: audit 2024-02-13T03:40:13.419382+0000 mon.c (mon.2) 581 : audit [INF] from='client.? v1:172.21.15.161:0/310799007' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]: dispatch 2024-02-13T03:40:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:14 smithi183 bash[20824]: audit 2024-02-13T03:40:13.420145+0000 mon.a (mon.0) 3152 : audit [INF] from='client.48410 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]: dispatch 2024-02-13T03:40:14.741 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put obj10 /etc/passwd 2024-02-13T03:40:14.779 INFO:tasks.workunit.client.0.smithi161.stderr:+ sleep 30 2024-02-13T03:40:15.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:15 smithi161 bash[19769]: cluster 2024-02-13T03:40:13.602515+0000 mgr.y (mgr.24473) 737 : cluster [DBG] pgmap v1224: 176 pgs: 12 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:15.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:15 smithi161 bash[19769]: audit 2024-02-13T03:40:14.319250+0000 mon.a (mon.0) 3153 : audit [INF] from='client.48410 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]': finished 2024-02-13T03:40:15.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:15 smithi161 bash[19769]: cluster 2024-02-13T03:40:14.319582+0000 mon.a (mon.0) 3154 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-02-13T03:40:15.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:15 smithi161 bash[27879]: cluster 2024-02-13T03:40:13.602515+0000 mgr.y (mgr.24473) 737 : cluster [DBG] pgmap v1224: 176 pgs: 12 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:15.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:15 smithi161 bash[27879]: audit 2024-02-13T03:40:14.319250+0000 mon.a (mon.0) 3153 : audit [INF] from='client.48410 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]': finished 2024-02-13T03:40:15.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:15 smithi161 bash[27879]: cluster 2024-02-13T03:40:14.319582+0000 mon.a (mon.0) 3154 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-02-13T03:40:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:15 smithi183 bash[20824]: cluster 2024-02-13T03:40:13.602515+0000 mgr.y (mgr.24473) 737 : cluster [DBG] pgmap v1224: 176 pgs: 12 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:15 smithi183 bash[20824]: audit 2024-02-13T03:40:14.319250+0000 mon.a (mon.0) 3153 : audit [INF] from='client.48410 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "app": "rados"}]': finished 2024-02-13T03:40:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:15 smithi183 bash[20824]: cluster 2024-02-13T03:40:14.319582+0000 mon.a (mon.0) 3154 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-02-13T03:40:16.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:40:16 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:40:17.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:17 smithi161 bash[27879]: cluster 2024-02-13T03:40:15.603513+0000 mgr.y (mgr.24473) 738 : cluster [DBG] pgmap v1226: 176 pgs: 9 creating+peering, 167 active+clean; 29 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 5.5 KiB/s wr, 0 op/s 2024-02-13T03:40:17.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:17 smithi161 bash[19769]: cluster 2024-02-13T03:40:15.603513+0000 mgr.y (mgr.24473) 738 : cluster [DBG] pgmap v1226: 176 pgs: 9 creating+peering, 167 active+clean; 29 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 5.5 KiB/s wr, 0 op/s 2024-02-13T03:40:17.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:17 smithi183 bash[20824]: cluster 2024-02-13T03:40:15.603513+0000 mgr.y (mgr.24473) 738 : cluster [DBG] pgmap v1226: 176 pgs: 9 creating+peering, 167 active+clean; 29 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 5.5 KiB/s wr, 0 op/s 2024-02-13T03:40:18.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:18 smithi161 bash[27879]: audit 2024-02-13T03:40:16.509019+0000 mgr.y (mgr.24473) 739 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:18.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:18 smithi161 bash[19769]: audit 2024-02-13T03:40:16.509019+0000 mgr.y (mgr.24473) 739 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:18 smithi183 bash[20824]: audit 2024-02-13T03:40:16.509019+0000 mgr.y (mgr.24473) 739 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:19.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:19 smithi161 bash[27879]: cluster 2024-02-13T03:40:17.605314+0000 mgr.y (mgr.24473) 740 : cluster [DBG] pgmap v1227: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 20 KiB/s wr, 2 op/s 2024-02-13T03:40:19.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:19 smithi161 bash[27879]: cluster 2024-02-13T03:40:19.091896+0000 mon.a (mon.0) 3155 : cluster [WRN] pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' is full (reached quota's max_objects: 10) 2024-02-13T03:40:19.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:19 smithi161 bash[27879]: cluster 2024-02-13T03:40:19.092515+0000 mon.a (mon.0) 3156 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-13T03:40:19.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:19 smithi161 bash[27879]: cluster 2024-02-13T03:40:19.103488+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-02-13T03:40:19.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:19 smithi161 bash[19769]: cluster 2024-02-13T03:40:17.605314+0000 mgr.y (mgr.24473) 740 : cluster [DBG] pgmap v1227: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 20 KiB/s wr, 2 op/s 2024-02-13T03:40:19.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:19 smithi161 bash[19769]: cluster 2024-02-13T03:40:19.091896+0000 mon.a (mon.0) 3155 : cluster [WRN] pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' is full (reached quota's max_objects: 10) 2024-02-13T03:40:19.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:19 smithi161 bash[19769]: cluster 2024-02-13T03:40:19.092515+0000 mon.a (mon.0) 3156 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-13T03:40:19.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:19 smithi161 bash[19769]: cluster 2024-02-13T03:40:19.103488+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-02-13T03:40:19.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:40:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:40:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:40:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:19 smithi183 bash[20824]: cluster 2024-02-13T03:40:17.605314+0000 mgr.y (mgr.24473) 740 : cluster [DBG] pgmap v1227: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 20 KiB/s wr, 2 op/s 2024-02-13T03:40:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:19 smithi183 bash[20824]: cluster 2024-02-13T03:40:19.091896+0000 mon.a (mon.0) 3155 : cluster [WRN] pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' is full (reached quota's max_objects: 10) 2024-02-13T03:40:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:19 smithi183 bash[20824]: cluster 2024-02-13T03:40:19.092515+0000 mon.a (mon.0) 3156 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-13T03:40:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:19 smithi183 bash[20824]: cluster 2024-02-13T03:40:19.103488+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-02-13T03:40:21.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:20 smithi161 bash[27879]: cluster 2024-02-13T03:40:19.606135+0000 mgr.y (mgr.24473) 741 : cluster [DBG] pgmap v1229: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 811 B/s rd, 19 KiB/s wr, 2 op/s 2024-02-13T03:40:21.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:20 smithi161 bash[27879]: audit 2024-02-13T03:40:19.797114+0000 mon.a (mon.0) 3158 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:40:21.100 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:40:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:40:21] "GET /metrics HTTP/1.1" 200 188237 "" "Prometheus/2.33.4" 2024-02-13T03:40:21.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:20 smithi161 bash[19769]: cluster 2024-02-13T03:40:19.606135+0000 mgr.y (mgr.24473) 741 : cluster [DBG] pgmap v1229: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 811 B/s rd, 19 KiB/s wr, 2 op/s 2024-02-13T03:40:21.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:20 smithi161 bash[19769]: audit 2024-02-13T03:40:19.797114+0000 mon.a (mon.0) 3158 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:40:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:20 smithi183 bash[20824]: cluster 2024-02-13T03:40:19.606135+0000 mgr.y (mgr.24473) 741 : cluster [DBG] pgmap v1229: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 811 B/s rd, 19 KiB/s wr, 2 op/s 2024-02-13T03:40:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:20 smithi183 bash[20824]: audit 2024-02-13T03:40:19.797114+0000 mon.a (mon.0) 3158 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:40:22.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:21 smithi161 bash[27879]: audit 2024-02-13T03:40:21.344992+0000 mon.c (mon.2) 582 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:40:22.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:21 smithi161 bash[27879]: audit 2024-02-13T03:40:21.677883+0000 mon.c (mon.2) 583 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:40:22.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:21 smithi161 bash[27879]: audit 2024-02-13T03:40:21.679778+0000 mon.c (mon.2) 584 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:40:22.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:21 smithi161 bash[27879]: audit 2024-02-13T03:40:21.690372+0000 mon.a (mon.0) 3159 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:40:22.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:21 smithi161 bash[19769]: audit 2024-02-13T03:40:21.344992+0000 mon.c (mon.2) 582 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:40:22.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:21 smithi161 bash[19769]: audit 2024-02-13T03:40:21.677883+0000 mon.c (mon.2) 583 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:40:22.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:21 smithi161 bash[19769]: audit 2024-02-13T03:40:21.679778+0000 mon.c (mon.2) 584 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:40:22.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:21 smithi161 bash[19769]: audit 2024-02-13T03:40:21.690372+0000 mon.a (mon.0) 3159 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:40:22.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:21 smithi183 bash[20824]: audit 2024-02-13T03:40:21.344992+0000 mon.c (mon.2) 582 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:40:22.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:21 smithi183 bash[20824]: audit 2024-02-13T03:40:21.677883+0000 mon.c (mon.2) 583 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:40:22.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:21 smithi183 bash[20824]: audit 2024-02-13T03:40:21.679778+0000 mon.c (mon.2) 584 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:40:22.239 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:21 smithi183 bash[20824]: audit 2024-02-13T03:40:21.690372+0000 mon.a (mon.0) 3159 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:40:23.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:22 smithi161 bash[27879]: cluster 2024-02-13T03:40:21.608484+0000 mgr.y (mgr.24473) 742 : cluster [DBG] pgmap v1230: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-13T03:40:23.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:22 smithi161 bash[19769]: cluster 2024-02-13T03:40:21.608484+0000 mgr.y (mgr.24473) 742 : cluster [DBG] pgmap v1230: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-13T03:40:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:22 smithi183 bash[20824]: cluster 2024-02-13T03:40:21.608484+0000 mgr.y (mgr.24473) 742 : cluster [DBG] pgmap v1230: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-13T03:40:25.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:25 smithi161 bash[27879]: cluster 2024-02-13T03:40:23.609230+0000 mgr.y (mgr.24473) 743 : cluster [DBG] pgmap v1231: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-02-13T03:40:25.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:25 smithi161 bash[19769]: cluster 2024-02-13T03:40:23.609230+0000 mgr.y (mgr.24473) 743 : cluster [DBG] pgmap v1231: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-02-13T03:40:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:25 smithi183 bash[20824]: cluster 2024-02-13T03:40:23.609230+0000 mgr.y (mgr.24473) 743 : cluster [DBG] pgmap v1231: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-02-13T03:40:26.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:40:26 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:40:27.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:27 smithi161 bash[27879]: cluster 2024-02-13T03:40:25.610194+0000 mgr.y (mgr.24473) 744 : cluster [DBG] pgmap v1232: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 9.6 KiB/s wr, 1 op/s 2024-02-13T03:40:27.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:27 smithi161 bash[19769]: cluster 2024-02-13T03:40:25.610194+0000 mgr.y (mgr.24473) 744 : cluster [DBG] pgmap v1232: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 9.6 KiB/s wr, 1 op/s 2024-02-13T03:40:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:27 smithi183 bash[20824]: cluster 2024-02-13T03:40:25.610194+0000 mgr.y (mgr.24473) 744 : cluster [DBG] pgmap v1232: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 9.6 KiB/s wr, 1 op/s 2024-02-13T03:40:28.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:28 smithi161 bash[27879]: audit 2024-02-13T03:40:26.520331+0000 mgr.y (mgr.24473) 745 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:28.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:28 smithi161 bash[19769]: audit 2024-02-13T03:40:26.520331+0000 mgr.y (mgr.24473) 745 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:28 smithi183 bash[20824]: audit 2024-02-13T03:40:26.520331+0000 mgr.y (mgr.24473) 745 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:29.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:29 smithi161 bash[19769]: cluster 2024-02-13T03:40:27.611963+0000 mgr.y (mgr.24473) 746 : cluster [DBG] pgmap v1233: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:40:29.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:29 smithi161 bash[27879]: cluster 2024-02-13T03:40:27.611963+0000 mgr.y (mgr.24473) 746 : cluster [DBG] pgmap v1233: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:40:29.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:29 smithi183 bash[20824]: cluster 2024-02-13T03:40:27.611963+0000 mgr.y (mgr.24473) 746 : cluster [DBG] pgmap v1233: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:40:29.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:40:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:40:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:40:31.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:31 smithi161 bash[19769]: cluster 2024-02-13T03:40:29.612736+0000 mgr.y (mgr.24473) 747 : cluster [DBG] pgmap v1234: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 973 B/s rd, 0 op/s 2024-02-13T03:40:31.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:40:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:40:31] "GET /metrics HTTP/1.1" 200 188237 "" "Prometheus/2.33.4" 2024-02-13T03:40:31.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:31 smithi161 bash[27879]: cluster 2024-02-13T03:40:29.612736+0000 mgr.y (mgr.24473) 747 : cluster [DBG] pgmap v1234: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 973 B/s rd, 0 op/s 2024-02-13T03:40:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:31 smithi183 bash[20824]: cluster 2024-02-13T03:40:29.612736+0000 mgr.y (mgr.24473) 747 : cluster [DBG] pgmap v1234: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 973 B/s rd, 0 op/s 2024-02-13T03:40:33.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:33 smithi161 bash[27879]: cluster 2024-02-13T03:40:31.614635+0000 mgr.y (mgr.24473) 748 : cluster [DBG] pgmap v1235: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:33.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:33 smithi161 bash[19769]: cluster 2024-02-13T03:40:31.614635+0000 mgr.y (mgr.24473) 748 : cluster [DBG] pgmap v1235: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:33 smithi183 bash[20824]: cluster 2024-02-13T03:40:31.614635+0000 mgr.y (mgr.24473) 748 : cluster [DBG] pgmap v1235: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:35.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:35 smithi161 bash[27879]: cluster 2024-02-13T03:40:33.615420+0000 mgr.y (mgr.24473) 749 : cluster [DBG] pgmap v1236: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:35.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:35 smithi161 bash[19769]: cluster 2024-02-13T03:40:33.615420+0000 mgr.y (mgr.24473) 749 : cluster [DBG] pgmap v1236: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:35 smithi183 bash[20824]: cluster 2024-02-13T03:40:33.615420+0000 mgr.y (mgr.24473) 749 : cluster [DBG] pgmap v1236: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:36.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:40:36 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:40:37.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:37 smithi161 bash[19769]: cluster 2024-02-13T03:40:35.616513+0000 mgr.y (mgr.24473) 750 : cluster [DBG] pgmap v1237: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:37.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:37 smithi161 bash[27879]: cluster 2024-02-13T03:40:35.616513+0000 mgr.y (mgr.24473) 750 : cluster [DBG] pgmap v1237: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:37 smithi183 bash[20824]: cluster 2024-02-13T03:40:35.616513+0000 mgr.y (mgr.24473) 750 : cluster [DBG] pgmap v1237: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:38.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:38 smithi161 bash[27879]: audit 2024-02-13T03:40:36.531754+0000 mgr.y (mgr.24473) 751 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:38.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:38 smithi161 bash[19769]: audit 2024-02-13T03:40:36.531754+0000 mgr.y (mgr.24473) 751 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:38.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:38 smithi183 bash[20824]: audit 2024-02-13T03:40:36.531754+0000 mgr.y (mgr.24473) 751 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:39.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:39 smithi161 bash[27879]: cluster 2024-02-13T03:40:37.618360+0000 mgr.y (mgr.24473) 752 : cluster [DBG] pgmap v1238: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:39.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:39 smithi161 bash[19769]: cluster 2024-02-13T03:40:37.618360+0000 mgr.y (mgr.24473) 752 : cluster [DBG] pgmap v1238: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:39.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:39 smithi183 bash[20824]: cluster 2024-02-13T03:40:37.618360+0000 mgr.y (mgr.24473) 752 : cluster [DBG] pgmap v1238: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:39.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:40:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:40:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:40:41.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:41 smithi161 bash[19769]: cluster 2024-02-13T03:40:39.619150+0000 mgr.y (mgr.24473) 753 : cluster [DBG] pgmap v1239: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:41.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:41 smithi161 bash[27879]: cluster 2024-02-13T03:40:39.619150+0000 mgr.y (mgr.24473) 753 : cluster [DBG] pgmap v1239: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:41.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:40:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:40:41] "GET /metrics HTTP/1.1" 200 188235 "" "Prometheus/2.33.4" 2024-02-13T03:40:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:41 smithi183 bash[20824]: cluster 2024-02-13T03:40:39.619150+0000 mgr.y (mgr.24473) 753 : cluster [DBG] pgmap v1239: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:43.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:43 smithi161 bash[27879]: cluster 2024-02-13T03:40:41.621149+0000 mgr.y (mgr.24473) 754 : cluster [DBG] pgmap v1240: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:43.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:43 smithi161 bash[19769]: cluster 2024-02-13T03:40:41.621149+0000 mgr.y (mgr.24473) 754 : cluster [DBG] pgmap v1240: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:43.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:43 smithi183 bash[20824]: cluster 2024-02-13T03:40:41.621149+0000 mgr.y (mgr.24473) 754 : cluster [DBG] pgmap v1240: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:44.781 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=128448 2024-02-13T03:40:44.781 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool set-quota 140c6d73-9bda-4f05-af7a-8f6a188328fa max_objects 100 2024-02-13T03:40:44.781 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put onemore /etc/passwd 2024-02-13T03:40:44.915 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb2ac31a700 1 -- 172.21.15.161:0/1034265773 >> v1:172.21.15.161:6789/0 conn(0x7fb2a4104060 legacy=0x7fb2a4104440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:44.915 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb2ac31a700 1 -- 172.21.15.161:0/1034265773 shutdown_connections 2024-02-13T03:40:44.915 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb2ac31a700 1 -- 172.21.15.161:0/1034265773 >> 172.21.15.161:0/1034265773 conn(0x7fb2a406d8d0 msgr2=0x7fb2a406fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:40:44.915 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb2ac31a700 1 -- 172.21.15.161:0/1034265773 shutdown_connections 2024-02-13T03:40:44.915 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb2ac31a700 1 -- 172.21.15.161:0/1034265773 wait complete. 2024-02-13T03:40:44.918 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb2ac31a700 1 Processor -- start 2024-02-13T03:40:44.918 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb2ac31a700 1 -- start start 2024-02-13T03:40:44.918 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb2ac31a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb2a411d990 con 0x7fb2a4106e00 2024-02-13T03:40:44.918 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb2ac31a700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb2a411eb70 con 0x7fb2a410a940 2024-02-13T03:40:44.918 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb2ac31a700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb2a411fd70 con 0x7fb2a4104060 2024-02-13T03:40:44.918 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb2aa0b6700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7fb2a4104060 0x7fb2a4100300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:52594/0 (socket says 172.21.15.161:52594) 2024-02-13T03:40:44.918 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb2aa0b6700 1 -- 172.21.15.161:0/2430413454 learned_addr learned my addr 172.21.15.161:0/2430413454 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:40:44.918 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757274809 0 0) 0x7fb2a411d990 con 0x7fb2a4106e00 2024-02-13T03:40:44.918 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2900033b0 con 0x7fb2a4106e00 2024-02-13T03:40:44.919 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3324948145 0 0) 0x7fb2a411fd70 con 0x7fb2a4104060 2024-02-13T03:40:44.919 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a411d990 con 0x7fb2a4104060 2024-02-13T03:40:44.919 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1088203739 0 0) 0x7fb2a411eb70 con 0x7fb2a410a940 2024-02-13T03:40:44.919 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a411fd70 con 0x7fb2a410a940 2024-02-13T03:40:44.919 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3067220333 0 0) 0x7fb2a411d990 con 0x7fb2a4104060 2024-02-13T03:40:44.919 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb2a411eb70 con 0x7fb2a4104060 2024-02-13T03:40:44.919 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb29c003090 con 0x7fb2a4104060 2024-02-13T03:40:44.919 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2602878069 0 0) 0x7fb2900033b0 con 0x7fb2a4106e00 2024-02-13T03:40:44.919 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb2a411d990 con 0x7fb2a4106e00 2024-02-13T03:40:44.919 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb294001800 con 0x7fb2a4106e00 2024-02-13T03:40:44.919 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3442672320 0 0) 0x7fb2a411eb70 con 0x7fb2a4104060 2024-02-13T03:40:44.919 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 >> v1:172.21.15.183:6789/0 conn(0x7fb2a410a940 legacy=0x7fb2a411c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:44.920 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 >> v1:172.21.15.161:6789/0 conn(0x7fb2a4106e00 legacy=0x7fb2a4100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:44.920 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a4120f70 con 0x7fb2a4104060 2024-02-13T03:40:44.920 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.912+0000 7fb2ac31a700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a411ff60 con 0x7fb2a4104060 2024-02-13T03:40:44.920 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.916+0000 7fb2ac31a700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a41203b0 con 0x7fb2a4104060 2024-02-13T03:40:44.920 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.916+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fb29c002770 con 0x7fb2a4104060 2024-02-13T03:40:44.920 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.916+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fb29c003db0 con 0x7fb2a4104060 2024-02-13T03:40:44.921 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.916+0000 7fb2ac31a700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb288004ee0 con 0x7fb2a4104060 2024-02-13T03:40:44.921 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.916+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fb29c003fb0 con 0x7fb2a4104060 2024-02-13T03:40:44.921 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.916+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(724..724 src has 1..724) v4 ==== 7586+0+0 (unknown 4005738250 0 0) 0x7fb29c079330 con 0x7fb2a4104060 2024-02-13T03:40:44.921 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.916+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=725}) v3 -- 0x7fb2a411eb70 con 0x7fb2a4104060 2024-02-13T03:40:44.928 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:44.924+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fb29c04cb10 con 0x7fb2a4104060 2024-02-13T03:40:45.108 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:45.104+0000 7fb2ac31a700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7fb288005190 con 0x7fb2a4104060 2024-02-13T03:40:45.182 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:45.176+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.2 v1:172.21.15.161:6790/0 10 ==== osd_map(725..725 src has 1..725) v4 ==== 620+0+0 (unknown 3225554207 0 0) 0x7fb29c0042e0 con 0x7fb2a4104060 2024-02-13T03:40:45.182 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:45.176+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=726}) v3 -- 0x7fb2900033b0 con 0x7fb2a4104060 2024-02-13T03:40:45.187 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:45.184+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.2 v1:172.21.15.161:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v725) v1 ==== 225+0+0 (unknown 4161736611 0 0) 0x7fb29c04af80 con 0x7fb2a4104060 2024-02-13T03:40:45.300 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:45.296+0000 7fb2ac31a700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7fb288005870 con 0x7fb2a4104060 2024-02-13T03:40:45.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:45 smithi161 bash[27879]: cluster 2024-02-13T03:40:43.621972+0000 mgr.y (mgr.24473) 755 : cluster [DBG] pgmap v1241: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:45.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:45 smithi161 bash[27879]: audit 2024-02-13T03:40:45.108787+0000 mon.c (mon.2) 585 : audit [INF] from='client.? v1:172.21.15.161:0/2430413454' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]: dispatch 2024-02-13T03:40:45.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:45 smithi161 bash[27879]: audit 2024-02-13T03:40:45.109586+0000 mon.a (mon.0) 3160 : audit [INF] from='client.48515 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]: dispatch 2024-02-13T03:40:45.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:45 smithi161 bash[19769]: cluster 2024-02-13T03:40:43.621972+0000 mgr.y (mgr.24473) 755 : cluster [DBG] pgmap v1241: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:45.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:45 smithi161 bash[19769]: audit 2024-02-13T03:40:45.108787+0000 mon.c (mon.2) 585 : audit [INF] from='client.? v1:172.21.15.161:0/2430413454' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]: dispatch 2024-02-13T03:40:45.427 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:45 smithi161 bash[19769]: audit 2024-02-13T03:40:45.109586+0000 mon.a (mon.0) 3160 : audit [INF] from='client.48515 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]: dispatch 2024-02-13T03:40:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:45 smithi183 bash[20824]: cluster 2024-02-13T03:40:43.621972+0000 mgr.y (mgr.24473) 755 : cluster [DBG] pgmap v1241: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:45 smithi183 bash[20824]: audit 2024-02-13T03:40:45.108787+0000 mon.c (mon.2) 585 : audit [INF] from='client.? v1:172.21.15.161:0/2430413454' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]: dispatch 2024-02-13T03:40:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:45 smithi183 bash[20824]: audit 2024-02-13T03:40:45.109586+0000 mon.a (mon.0) 3160 : audit [INF] from='client.48515 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]: dispatch 2024-02-13T03:40:46.186 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:46.184+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.2 v1:172.21.15.161:6790/0 12 ==== osd_map(726..726 src has 1..726) v4 ==== 620+0+0 (unknown 2071342674 0 0) 0x7fb29c04cd40 con 0x7fb2a4104060 2024-02-13T03:40:46.187 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:46.184+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=727}) v3 -- 0x7fb29006eb30 con 0x7fb2a4104060 2024-02-13T03:40:46.193 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:46.188+0000 7fb29b7fe700 1 -- 172.21.15.161:0/2430413454 <== mon.2 v1:172.21.15.161:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v726) v1 ==== 225+0+0 (unknown 2167032619 0 0) 0x7fb29c04c700 con 0x7fb2a4104060 2024-02-13T03:40:46.196 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:46.192+0000 7fb2ac31a700 1 -- 172.21.15.161:0/2430413454 >> v1:172.21.15.161:6800/4114337139 conn(0x7fb290062230 legacy=0x7fb2900646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:46.196 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:46.192+0000 7fb2ac31a700 1 -- 172.21.15.161:0/2430413454 >> v1:172.21.15.161:6790/0 conn(0x7fb2a4104060 legacy=0x7fb2a4100300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:46.196 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:46.192+0000 7fb2ac31a700 1 -- 172.21.15.161:0/2430413454 shutdown_connections 2024-02-13T03:40:46.196 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:46.192+0000 7fb2ac31a700 1 -- 172.21.15.161:0/2430413454 >> 172.21.15.161:0/2430413454 conn(0x7fb2a406d8d0 msgr2=0x7fb2a410dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:40:46.196 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:46.192+0000 7fb2ac31a700 1 -- 172.21.15.161:0/2430413454 shutdown_connections 2024-02-13T03:40:46.197 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:46.192+0000 7fb2ac31a700 1 -- 172.21.15.161:0/2430413454 wait complete. 2024-02-13T03:40:46.197 INFO:tasks.workunit.client.0.smithi161.stderr:set-quota max_objects = 100 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa 2024-02-13T03:40:46.208 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 128448 2024-02-13T03:40:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:46 smithi183 bash[20824]: audit 2024-02-13T03:40:45.178839+0000 mon.a (mon.0) 3161 : audit [INF] from='client.48515 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]': finished 2024-02-13T03:40:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:46 smithi183 bash[20824]: cluster 2024-02-13T03:40:45.178998+0000 mon.a (mon.0) 3162 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-02-13T03:40:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:46 smithi183 bash[20824]: audit 2024-02-13T03:40:45.300274+0000 mon.c (mon.2) 586 : audit [INF] from='client.? v1:172.21.15.161:0/2430413454' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]: dispatch 2024-02-13T03:40:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:46 smithi183 bash[20824]: audit 2024-02-13T03:40:45.300949+0000 mon.a (mon.0) 3163 : audit [INF] from='client.48515 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]: dispatch 2024-02-13T03:40:46.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:46 smithi161 bash[19769]: audit 2024-02-13T03:40:45.178839+0000 mon.a (mon.0) 3161 : audit [INF] from='client.48515 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]': finished 2024-02-13T03:40:46.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:46 smithi161 bash[19769]: cluster 2024-02-13T03:40:45.178998+0000 mon.a (mon.0) 3162 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-02-13T03:40:46.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:46 smithi161 bash[19769]: audit 2024-02-13T03:40:45.300274+0000 mon.c (mon.2) 586 : audit [INF] from='client.? v1:172.21.15.161:0/2430413454' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]: dispatch 2024-02-13T03:40:46.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:46 smithi161 bash[19769]: audit 2024-02-13T03:40:45.300949+0000 mon.a (mon.0) 3163 : audit [INF] from='client.48515 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]: dispatch 2024-02-13T03:40:46.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:46 smithi161 bash[27879]: audit 2024-02-13T03:40:45.178839+0000 mon.a (mon.0) 3161 : audit [INF] from='client.48515 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]': finished 2024-02-13T03:40:46.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:46 smithi161 bash[27879]: cluster 2024-02-13T03:40:45.178998+0000 mon.a (mon.0) 3162 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-02-13T03:40:46.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:46 smithi161 bash[27879]: audit 2024-02-13T03:40:45.300274+0000 mon.c (mon.2) 586 : audit [INF] from='client.? v1:172.21.15.161:0/2430413454' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]: dispatch 2024-02-13T03:40:46.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:46 smithi161 bash[27879]: audit 2024-02-13T03:40:45.300949+0000 mon.a (mon.0) 3163 : audit [INF] from='client.48515 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]: dispatch 2024-02-13T03:40:46.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:40:46 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:40:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:47 smithi183 bash[20824]: cluster 2024-02-13T03:40:45.622916+0000 mgr.y (mgr.24473) 756 : cluster [DBG] pgmap v1243: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:40:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:47 smithi183 bash[20824]: audit 2024-02-13T03:40:46.190218+0000 mon.a (mon.0) 3164 : audit [INF] from='client.48515 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]': finished 2024-02-13T03:40:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:47 smithi183 bash[20824]: cluster 2024-02-13T03:40:46.190426+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-02-13T03:40:47.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:47 smithi161 bash[27879]: cluster 2024-02-13T03:40:45.622916+0000 mgr.y (mgr.24473) 756 : cluster [DBG] pgmap v1243: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:40:47.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:47 smithi161 bash[27879]: audit 2024-02-13T03:40:46.190218+0000 mon.a (mon.0) 3164 : audit [INF] from='client.48515 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]': finished 2024-02-13T03:40:47.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:47 smithi161 bash[27879]: cluster 2024-02-13T03:40:46.190426+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-02-13T03:40:47.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:47 smithi161 bash[19769]: cluster 2024-02-13T03:40:45.622916+0000 mgr.y (mgr.24473) 756 : cluster [DBG] pgmap v1243: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:40:47.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:47 smithi161 bash[19769]: audit 2024-02-13T03:40:46.190218+0000 mon.a (mon.0) 3164 : audit [INF] from='client.48515 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "100"}]': finished 2024-02-13T03:40:47.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:47 smithi161 bash[19769]: cluster 2024-02-13T03:40:46.190426+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-02-13T03:40:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:48 smithi183 bash[20824]: audit 2024-02-13T03:40:46.541425+0000 mgr.y (mgr.24473) 757 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:48.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:48 smithi161 bash[27879]: audit 2024-02-13T03:40:46.541425+0000 mgr.y (mgr.24473) 757 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:48.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:48 smithi161 bash[19769]: audit 2024-02-13T03:40:46.541425+0000 mgr.y (mgr.24473) 757 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:49.119 INFO:tasks.workunit.client.0.smithi161.stderr:+ [ 0 -ne 0 ] 2024-02-13T03:40:49.119 INFO:tasks.workunit.client.0.smithi161.stderr:+ true 2024-02-13T03:40:49.119 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put twomore /etc/passwd 2024-02-13T03:40:49.154 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool set-quota 140c6d73-9bda-4f05-af7a-8f6a188328fa max_bytes 100 2024-02-13T03:40:49.275 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f501dbe5700 1 -- 172.21.15.161:0/1636596915 >> v1:172.21.15.161:6789/0 conn(0x7f50180f94d0 legacy=0x7f5018102a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:49.275 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f501dbe5700 1 -- 172.21.15.161:0/1636596915 shutdown_connections 2024-02-13T03:40:49.275 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f501dbe5700 1 -- 172.21.15.161:0/1636596915 >> 172.21.15.161:0/1636596915 conn(0x7f50180f2570 msgr2=0x7f50180f4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:40:49.275 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f501dbe5700 1 -- 172.21.15.161:0/1636596915 shutdown_connections 2024-02-13T03:40:49.275 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f501dbe5700 1 -- 172.21.15.161:0/1636596915 wait complete. 2024-02-13T03:40:49.278 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f501dbe5700 1 Processor -- start 2024-02-13T03:40:49.278 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f501dbe5700 1 -- start start 2024-02-13T03:40:49.278 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f501dbe5700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5018117210 con 0x7f50180f89e0 2024-02-13T03:40:49.278 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f501dbe5700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f50181183b0 con 0x7f5018104340 2024-02-13T03:40:49.278 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f501dbe5700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f50181195b0 con 0x7f50180f94d0 2024-02-13T03:40:49.279 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5016ffd700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f50180f94d0 0x7f50180ffa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:53560/0 (socket says 172.21.15.161:53560) 2024-02-13T03:40:49.279 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5016ffd700 1 -- 172.21.15.161:0/1039265695 learned_addr learned my addr 172.21.15.161:0/1039265695 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:40:49.279 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f50177fe700 1 --1- 172.21.15.161:0/1039265695 >> v1:172.21.15.161:6789/0 conn(0x7f50180f89e0 0x7f50180ff360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:39336/0 (socket says 172.21.15.161:39336) 2024-02-13T03:40:49.279 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505050238 0 0) 0x7f5018117210 con 0x7f50180f89e0 2024-02-13T03:40:49.279 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5004003030 con 0x7f50180f89e0 2024-02-13T03:40:49.279 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793331295 0 0) 0x7f50181195b0 con 0x7f50180f94d0 2024-02-13T03:40:49.279 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5018117210 con 0x7f50180f94d0 2024-02-13T03:40:49.279 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359649075 0 0) 0x7f50181183b0 con 0x7f5018104340 2024-02-13T03:40:49.279 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50181195b0 con 0x7f5018104340 2024-02-13T03:40:49.279 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3509964936 0 0) 0x7f5004003030 con 0x7f50180f89e0 2024-02-13T03:40:49.279 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f50181183b0 con 0x7f50180f89e0 2024-02-13T03:40:49.279 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5000001880 con 0x7f50180f89e0 2024-02-13T03:40:49.279 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2649107539 0 0) 0x7f5018117210 con 0x7f50180f94d0 2024-02-13T03:40:49.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5004003030 con 0x7f50180f94d0 2024-02-13T03:40:49.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2530740378 0 0) 0x7f50181195b0 con 0x7f5018104340 2024-02-13T03:40:49.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5018117210 con 0x7f5018104340 2024-02-13T03:40:49.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5008002ea0 con 0x7f50180f94d0 2024-02-13T03:40:49.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5010001d30 con 0x7f5018104340 2024-02-13T03:40:49.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 547229680 0 0) 0x7f50181183b0 con 0x7f50180f89e0 2024-02-13T03:40:49.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 >> v1:172.21.15.161:6790/0 conn(0x7f50180f94d0 legacy=0x7f50180ffa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:49.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 >> v1:172.21.15.183:6789/0 conn(0x7f5018104340 legacy=0x7f50181159c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:49.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501811a7b0 con 0x7f50180f89e0 2024-02-13T03:40:49.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f501dbe5700 1 -- 172.21.15.161:0/1039265695 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50181197a0 con 0x7f50180f89e0 2024-02-13T03:40:49.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f501dbe5700 1 -- 172.21.15.161:0/1039265695 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5018119bf0 con 0x7f50180f89e0 2024-02-13T03:40:49.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f5000002c40 con 0x7f50180f89e0 2024-02-13T03:40:49.280 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.271+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5000005b10 con 0x7f50180f89e0 2024-02-13T03:40:49.281 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.275+0000 7f501dbe5700 1 -- 172.21.15.161:0/1039265695 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff8004ee0 con 0x7f50180f89e0 2024-02-13T03:40:49.281 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.275+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f5000001e60 con 0x7f50180f89e0 2024-02-13T03:40:49.281 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.275+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(727..727 src has 1..727) v4 ==== 7586+0+0 (unknown 2923598862 0 0) 0x7f5000078b10 con 0x7f50180f89e0 2024-02-13T03:40:49.291 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.287+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f500004c0e0 con 0x7f50180f89e0 2024-02-13T03:40:49.466 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:49.463+0000 7f501dbe5700 1 -- 172.21.15.161:0/1039265695 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f4ff8005190 con 0x7f50180f89e0 2024-02-13T03:40:49.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:40:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:40:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:40:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:49 smithi183 bash[20824]: cluster 2024-02-13T03:40:47.623719+0000 mgr.y (mgr.24473) 758 : cluster [DBG] pgmap v1245: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:49 smithi183 bash[20824]: cluster 2024-02-13T03:40:49.102170+0000 mon.a (mon.0) 3166 : cluster [INF] pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' no longer out of quota; removing NO_QUOTA flag 2024-02-13T03:40:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:49 smithi183 bash[20824]: cluster 2024-02-13T03:40:49.102650+0000 mon.a (mon.0) 3167 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-13T03:40:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:49 smithi183 bash[20824]: cluster 2024-02-13T03:40:49.112191+0000 mon.a (mon.0) 3168 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-02-13T03:40:49.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:49 smithi161 bash[19769]: cluster 2024-02-13T03:40:47.623719+0000 mgr.y (mgr.24473) 758 : cluster [DBG] pgmap v1245: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:49.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:49 smithi161 bash[19769]: cluster 2024-02-13T03:40:49.102170+0000 mon.a (mon.0) 3166 : cluster [INF] pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' no longer out of quota; removing NO_QUOTA flag 2024-02-13T03:40:49.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:49 smithi161 bash[19769]: cluster 2024-02-13T03:40:49.102650+0000 mon.a (mon.0) 3167 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-13T03:40:49.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:49 smithi161 bash[19769]: cluster 2024-02-13T03:40:49.112191+0000 mon.a (mon.0) 3168 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-02-13T03:40:49.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:49 smithi161 bash[27879]: cluster 2024-02-13T03:40:47.623719+0000 mgr.y (mgr.24473) 758 : cluster [DBG] pgmap v1245: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:40:49.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:49 smithi161 bash[27879]: cluster 2024-02-13T03:40:49.102170+0000 mon.a (mon.0) 3166 : cluster [INF] pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' no longer out of quota; removing NO_QUOTA flag 2024-02-13T03:40:49.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:49 smithi161 bash[27879]: cluster 2024-02-13T03:40:49.102650+0000 mon.a (mon.0) 3167 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-13T03:40:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:49 smithi161 bash[27879]: cluster 2024-02-13T03:40:49.112191+0000 mon.a (mon.0) 3168 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-02-13T03:40:50.223 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:50.219+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v728) v1 ==== 221+0+0 (unknown 3755071492 0 0) 0x7f500004a4f0 con 0x7f50180f89e0 2024-02-13T03:40:50.337 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:50.331+0000 7f501dbe5700 1 -- 172.21.15.161:0/1039265695 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f4ff80058d0 con 0x7f50180f89e0 2024-02-13T03:40:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:50 smithi183 bash[20824]: audit 2024-02-13T03:40:49.466750+0000 mon.a (mon.0) 3169 : audit [INF] from='client.? v1:172.21.15.161:0/1039265695' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-13T03:40:50.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:50 smithi183 bash[20824]: audit 2024-02-13T03:40:49.799699+0000 mon.a (mon.0) 3170 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:40:50.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:50 smithi161 bash[27879]: audit 2024-02-13T03:40:49.466750+0000 mon.a (mon.0) 3169 : audit [INF] from='client.? v1:172.21.15.161:0/1039265695' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-13T03:40:50.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:50 smithi161 bash[27879]: audit 2024-02-13T03:40:49.799699+0000 mon.a (mon.0) 3170 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:40:50.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:50 smithi161 bash[19769]: audit 2024-02-13T03:40:49.466750+0000 mon.a (mon.0) 3169 : audit [INF] from='client.? v1:172.21.15.161:0/1039265695' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-13T03:40:50.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:50 smithi161 bash[19769]: audit 2024-02-13T03:40:49.799699+0000 mon.a (mon.0) 3170 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:40:51.225 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:51.219+0000 7f5014ff9700 1 -- 172.21.15.161:0/1039265695 <== mon.0 v1:172.21.15.161:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v729) v1 ==== 221+0+0 (unknown 1602292197 0 0) 0x7f500004bc70 con 0x7f50180f89e0 2024-02-13T03:40:51.228 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:51.223+0000 7f501dbe5700 1 -- 172.21.15.161:0/1039265695 >> v1:172.21.15.161:6800/4114337139 conn(0x7f50040623a0 legacy=0x7f5004064860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:51.228 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:51.223+0000 7f501dbe5700 1 -- 172.21.15.161:0/1039265695 >> v1:172.21.15.161:6789/0 conn(0x7f50180f89e0 legacy=0x7f50180ff360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:40:51.228 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:51.223+0000 7f501dbe5700 1 -- 172.21.15.161:0/1039265695 shutdown_connections 2024-02-13T03:40:51.228 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:51.223+0000 7f501dbe5700 1 -- 172.21.15.161:0/1039265695 >> 172.21.15.161:0/1039265695 conn(0x7f50180f2570 msgr2=0x7f50181077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:40:51.229 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:51.223+0000 7f501dbe5700 1 -- 172.21.15.161:0/1039265695 shutdown_connections 2024-02-13T03:40:51.229 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:40:51.223+0000 7f501dbe5700 1 -- 172.21.15.161:0/1039265695 wait complete. 2024-02-13T03:40:51.229 INFO:tasks.workunit.client.0.smithi161.stderr:set-quota max_bytes = 100 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa 2024-02-13T03:40:51.241 INFO:tasks.workunit.client.0.smithi161.stderr:+ sleep 30 2024-02-13T03:40:51.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:51 smithi161 bash[19769]: cluster 2024-02-13T03:40:49.624492+0000 mgr.y (mgr.24473) 759 : cluster [DBG] pgmap v1247: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:51.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:51 smithi161 bash[19769]: audit 2024-02-13T03:40:50.222426+0000 mon.a (mon.0) 3171 : audit [INF] from='client.? v1:172.21.15.161:0/1039265695' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]': finished 2024-02-13T03:40:51.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:51 smithi161 bash[19769]: cluster 2024-02-13T03:40:50.222587+0000 mon.a (mon.0) 3172 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-02-13T03:40:51.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:51 smithi161 bash[19769]: audit 2024-02-13T03:40:50.337643+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? v1:172.21.15.161:0/1039265695' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-13T03:40:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:51 smithi161 bash[27879]: cluster 2024-02-13T03:40:49.624492+0000 mgr.y (mgr.24473) 759 : cluster [DBG] pgmap v1247: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:51 smithi161 bash[27879]: audit 2024-02-13T03:40:50.222426+0000 mon.a (mon.0) 3171 : audit [INF] from='client.? v1:172.21.15.161:0/1039265695' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]': finished 2024-02-13T03:40:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:51 smithi161 bash[27879]: cluster 2024-02-13T03:40:50.222587+0000 mon.a (mon.0) 3172 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-02-13T03:40:51.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:51 smithi161 bash[27879]: audit 2024-02-13T03:40:50.337643+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? v1:172.21.15.161:0/1039265695' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-13T03:40:51.427 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:40:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:40:51] "GET /metrics HTTP/1.1" 200 188249 "" "Prometheus/2.33.4" 2024-02-13T03:40:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:51 smithi183 bash[20824]: cluster 2024-02-13T03:40:49.624492+0000 mgr.y (mgr.24473) 759 : cluster [DBG] pgmap v1247: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:40:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:51 smithi183 bash[20824]: audit 2024-02-13T03:40:50.222426+0000 mon.a (mon.0) 3171 : audit [INF] from='client.? v1:172.21.15.161:0/1039265695' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]': finished 2024-02-13T03:40:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:51 smithi183 bash[20824]: cluster 2024-02-13T03:40:50.222587+0000 mon.a (mon.0) 3172 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-02-13T03:40:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:51 smithi183 bash[20824]: audit 2024-02-13T03:40:50.337643+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? v1:172.21.15.161:0/1039265695' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-13T03:40:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:52 smithi183 bash[20824]: audit 2024-02-13T03:40:51.224635+0000 mon.a (mon.0) 3174 : audit [INF] from='client.? v1:172.21.15.161:0/1039265695' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]': finished 2024-02-13T03:40:52.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:52 smithi183 bash[20824]: cluster 2024-02-13T03:40:51.224722+0000 mon.a (mon.0) 3175 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-02-13T03:40:52.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:52 smithi161 bash[27879]: audit 2024-02-13T03:40:51.224635+0000 mon.a (mon.0) 3174 : audit [INF] from='client.? v1:172.21.15.161:0/1039265695' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]': finished 2024-02-13T03:40:52.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:52 smithi161 bash[27879]: cluster 2024-02-13T03:40:51.224722+0000 mon.a (mon.0) 3175 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-02-13T03:40:52.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:52 smithi161 bash[19769]: audit 2024-02-13T03:40:51.224635+0000 mon.a (mon.0) 3174 : audit [INF] from='client.? v1:172.21.15.161:0/1039265695' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "100"}]': finished 2024-02-13T03:40:52.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:52 smithi161 bash[19769]: cluster 2024-02-13T03:40:51.224722+0000 mon.a (mon.0) 3175 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-02-13T03:40:53.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:53 smithi183 bash[20824]: cluster 2024-02-13T03:40:51.625245+0000 mgr.y (mgr.24473) 760 : cluster [DBG] pgmap v1250: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 941 B/s rd, 4.4 KiB/s wr, 1 op/s 2024-02-13T03:40:53.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:53 smithi161 bash[27879]: cluster 2024-02-13T03:40:51.625245+0000 mgr.y (mgr.24473) 760 : cluster [DBG] pgmap v1250: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 941 B/s rd, 4.4 KiB/s wr, 1 op/s 2024-02-13T03:40:53.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:53 smithi161 bash[19769]: cluster 2024-02-13T03:40:51.625245+0000 mgr.y (mgr.24473) 760 : cluster [DBG] pgmap v1250: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 941 B/s rd, 4.4 KiB/s wr, 1 op/s 2024-02-13T03:40:54.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:54 smithi161 bash[27879]: cluster 2024-02-13T03:40:54.104505+0000 mon.a (mon.0) 3176 : cluster [WRN] pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' is full (reached quota's max_bytes: 100 B) 2024-02-13T03:40:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:54 smithi161 bash[27879]: cluster 2024-02-13T03:40:54.105020+0000 mon.a (mon.0) 3177 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-13T03:40:54.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:54 smithi161 bash[27879]: cluster 2024-02-13T03:40:54.118619+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-02-13T03:40:54.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:54 smithi161 bash[19769]: cluster 2024-02-13T03:40:54.104505+0000 mon.a (mon.0) 3176 : cluster [WRN] pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' is full (reached quota's max_bytes: 100 B) 2024-02-13T03:40:54.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:54 smithi161 bash[19769]: cluster 2024-02-13T03:40:54.105020+0000 mon.a (mon.0) 3177 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-13T03:40:54.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:54 smithi161 bash[19769]: cluster 2024-02-13T03:40:54.118619+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-02-13T03:40:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:54 smithi183 bash[20824]: cluster 2024-02-13T03:40:54.104505+0000 mon.a (mon.0) 3176 : cluster [WRN] pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' is full (reached quota's max_bytes: 100 B) 2024-02-13T03:40:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:54 smithi183 bash[20824]: cluster 2024-02-13T03:40:54.105020+0000 mon.a (mon.0) 3177 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-13T03:40:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:54 smithi183 bash[20824]: cluster 2024-02-13T03:40:54.118619+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-02-13T03:40:55.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:55 smithi161 bash[27879]: cluster 2024-02-13T03:40:53.626067+0000 mgr.y (mgr.24473) 761 : cluster [DBG] pgmap v1251: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-13T03:40:55.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:55 smithi161 bash[19769]: cluster 2024-02-13T03:40:53.626067+0000 mgr.y (mgr.24473) 761 : cluster [DBG] pgmap v1251: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-13T03:40:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:55 smithi183 bash[20824]: cluster 2024-02-13T03:40:53.626067+0000 mgr.y (mgr.24473) 761 : cluster [DBG] pgmap v1251: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-13T03:40:56.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:40:56 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:40:57.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:57 smithi161 bash[27879]: cluster 2024-02-13T03:40:55.627014+0000 mgr.y (mgr.24473) 762 : cluster [DBG] pgmap v1253: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-13T03:40:57.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:57 smithi161 bash[19769]: cluster 2024-02-13T03:40:55.627014+0000 mgr.y (mgr.24473) 762 : cluster [DBG] pgmap v1253: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-13T03:40:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:57 smithi183 bash[20824]: cluster 2024-02-13T03:40:55.627014+0000 mgr.y (mgr.24473) 762 : cluster [DBG] pgmap v1253: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-13T03:40:58.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:58 smithi161 bash[27879]: audit 2024-02-13T03:40:56.547893+0000 mgr.y (mgr.24473) 763 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:58 smithi161 bash[19769]: audit 2024-02-13T03:40:56.547893+0000 mgr.y (mgr.24473) 763 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:58 smithi183 bash[20824]: audit 2024-02-13T03:40:56.547893+0000 mgr.y (mgr.24473) 763 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:40:59.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:40:59 smithi161 bash[27879]: cluster 2024-02-13T03:40:57.628949+0000 mgr.y (mgr.24473) 764 : cluster [DBG] pgmap v1254: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-13T03:40:59.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:40:59 smithi161 bash[19769]: cluster 2024-02-13T03:40:57.628949+0000 mgr.y (mgr.24473) 764 : cluster [DBG] pgmap v1254: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-13T03:40:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:40:59 smithi183 bash[20824]: cluster 2024-02-13T03:40:57.628949+0000 mgr.y (mgr.24473) 764 : cluster [DBG] pgmap v1254: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-13T03:40:59.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:40:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:40:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:41:01.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:01 smithi161 bash[27879]: cluster 2024-02-13T03:40:59.629810+0000 mgr.y (mgr.24473) 765 : cluster [DBG] pgmap v1255: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 608 B/s rd, 0 op/s 2024-02-13T03:41:01.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:41:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:41:01] "GET /metrics HTTP/1.1" 200 188249 "" "Prometheus/2.33.4" 2024-02-13T03:41:01.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:01 smithi161 bash[19769]: cluster 2024-02-13T03:40:59.629810+0000 mgr.y (mgr.24473) 765 : cluster [DBG] pgmap v1255: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 608 B/s rd, 0 op/s 2024-02-13T03:41:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:01 smithi183 bash[20824]: cluster 2024-02-13T03:40:59.629810+0000 mgr.y (mgr.24473) 765 : cluster [DBG] pgmap v1255: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 608 B/s rd, 0 op/s 2024-02-13T03:41:03.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:03 smithi161 bash[27879]: cluster 2024-02-13T03:41:01.631759+0000 mgr.y (mgr.24473) 766 : cluster [DBG] pgmap v1256: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:41:03.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:03 smithi161 bash[19769]: cluster 2024-02-13T03:41:01.631759+0000 mgr.y (mgr.24473) 766 : cluster [DBG] pgmap v1256: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:41:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:03 smithi183 bash[20824]: cluster 2024-02-13T03:41:01.631759+0000 mgr.y (mgr.24473) 766 : cluster [DBG] pgmap v1256: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:41:05.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:05 smithi161 bash[27879]: cluster 2024-02-13T03:41:03.632511+0000 mgr.y (mgr.24473) 767 : cluster [DBG] pgmap v1257: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:41:05.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:05 smithi161 bash[27879]: audit 2024-02-13T03:41:04.798888+0000 mon.a (mon.0) 3179 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:41:05.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:05 smithi161 bash[19769]: cluster 2024-02-13T03:41:03.632511+0000 mgr.y (mgr.24473) 767 : cluster [DBG] pgmap v1257: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:41:05.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:05 smithi161 bash[19769]: audit 2024-02-13T03:41:04.798888+0000 mon.a (mon.0) 3179 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:41:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:05 smithi183 bash[20824]: cluster 2024-02-13T03:41:03.632511+0000 mgr.y (mgr.24473) 767 : cluster [DBG] pgmap v1257: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:41:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:05 smithi183 bash[20824]: audit 2024-02-13T03:41:04.798888+0000 mon.a (mon.0) 3179 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:41:06.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:41:06 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:41:07.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:07 smithi161 bash[27879]: cluster 2024-02-13T03:41:05.633504+0000 mgr.y (mgr.24473) 768 : cluster [DBG] pgmap v1258: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 888 B/s rd, 0 op/s 2024-02-13T03:41:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:07 smithi161 bash[19769]: cluster 2024-02-13T03:41:05.633504+0000 mgr.y (mgr.24473) 768 : cluster [DBG] pgmap v1258: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 888 B/s rd, 0 op/s 2024-02-13T03:41:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:07 smithi183 bash[20824]: cluster 2024-02-13T03:41:05.633504+0000 mgr.y (mgr.24473) 768 : cluster [DBG] pgmap v1258: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 888 B/s rd, 0 op/s 2024-02-13T03:41:08.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:08 smithi161 bash[27879]: audit 2024-02-13T03:41:06.559159+0000 mgr.y (mgr.24473) 769 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:08.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:08 smithi161 bash[19769]: audit 2024-02-13T03:41:06.559159+0000 mgr.y (mgr.24473) 769 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:08 smithi183 bash[20824]: audit 2024-02-13T03:41:06.559159+0000 mgr.y (mgr.24473) 769 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:09 smithi161 bash[27879]: cluster 2024-02-13T03:41:07.635337+0000 mgr.y (mgr.24473) 770 : cluster [DBG] pgmap v1259: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:09 smithi161 bash[19769]: cluster 2024-02-13T03:41:07.635337+0000 mgr.y (mgr.24473) 770 : cluster [DBG] pgmap v1259: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:09 smithi183 bash[20824]: cluster 2024-02-13T03:41:07.635337+0000 mgr.y (mgr.24473) 770 : cluster [DBG] pgmap v1259: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:09.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:41:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:41:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:41:11.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:41:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:41:11] "GET /metrics HTTP/1.1" 200 188238 "" "Prometheus/2.33.4" 2024-02-13T03:41:11.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:11 smithi161 bash[27879]: cluster 2024-02-13T03:41:09.636151+0000 mgr.y (mgr.24473) 771 : cluster [DBG] pgmap v1260: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:11.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:11 smithi161 bash[19769]: cluster 2024-02-13T03:41:09.636151+0000 mgr.y (mgr.24473) 771 : cluster [DBG] pgmap v1260: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:11.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:11 smithi183 bash[20824]: cluster 2024-02-13T03:41:09.636151+0000 mgr.y (mgr.24473) 771 : cluster [DBG] pgmap v1260: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:13.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:13 smithi161 bash[27879]: cluster 2024-02-13T03:41:11.638055+0000 mgr.y (mgr.24473) 772 : cluster [DBG] pgmap v1261: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:13.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:13 smithi161 bash[19769]: cluster 2024-02-13T03:41:11.638055+0000 mgr.y (mgr.24473) 772 : cluster [DBG] pgmap v1261: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:13 smithi183 bash[20824]: cluster 2024-02-13T03:41:11.638055+0000 mgr.y (mgr.24473) 772 : cluster [DBG] pgmap v1261: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:15.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:15 smithi161 bash[27879]: cluster 2024-02-13T03:41:13.638867+0000 mgr.y (mgr.24473) 773 : cluster [DBG] pgmap v1262: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:15.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:15 smithi161 bash[19769]: cluster 2024-02-13T03:41:13.638867+0000 mgr.y (mgr.24473) 773 : cluster [DBG] pgmap v1262: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:15 smithi183 bash[20824]: cluster 2024-02-13T03:41:13.638867+0000 mgr.y (mgr.24473) 773 : cluster [DBG] pgmap v1262: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:16.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:41:16 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:41:17.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:17 smithi161 bash[27879]: cluster 2024-02-13T03:41:15.639849+0000 mgr.y (mgr.24473) 774 : cluster [DBG] pgmap v1263: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:17.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:17 smithi161 bash[19769]: cluster 2024-02-13T03:41:15.639849+0000 mgr.y (mgr.24473) 774 : cluster [DBG] pgmap v1263: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:17.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:17 smithi183 bash[20824]: cluster 2024-02-13T03:41:15.639849+0000 mgr.y (mgr.24473) 774 : cluster [DBG] pgmap v1263: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:18.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:18 smithi161 bash[27879]: audit 2024-02-13T03:41:16.570453+0000 mgr.y (mgr.24473) 775 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:18.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:18 smithi161 bash[19769]: audit 2024-02-13T03:41:16.570453+0000 mgr.y (mgr.24473) 775 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:18 smithi183 bash[20824]: audit 2024-02-13T03:41:16.570453+0000 mgr.y (mgr.24473) 775 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:19.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:19 smithi161 bash[27879]: cluster 2024-02-13T03:41:17.640899+0000 mgr.y (mgr.24473) 776 : cluster [DBG] pgmap v1264: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:19.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:19 smithi161 bash[19769]: cluster 2024-02-13T03:41:17.640899+0000 mgr.y (mgr.24473) 776 : cluster [DBG] pgmap v1264: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:19 smithi183 bash[20824]: cluster 2024-02-13T03:41:17.640899+0000 mgr.y (mgr.24473) 776 : cluster [DBG] pgmap v1264: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:19.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:41:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:41:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:41:21.242 INFO:tasks.workunit.client.0.smithi161.stderr:+ pid=128538 2024-02-13T03:41:21.242 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool set-quota 140c6d73-9bda-4f05-af7a-8f6a188328fa max_bytes 0 2024-02-13T03:41:21.242 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put two /etc/passwd 2024-02-13T03:41:21.378 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.371+0000 7f2e2ae39700 1 -- 172.21.15.161:0/3905300661 >> v1:172.21.15.161:6789/0 conn(0x7f2e240feed0 legacy=0x7f2e241012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:21.378 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.371+0000 7f2e2ae39700 1 -- 172.21.15.161:0/3905300661 shutdown_connections 2024-02-13T03:41:21.378 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.371+0000 7f2e2ae39700 1 -- 172.21.15.161:0/3905300661 >> 172.21.15.161:0/3905300661 conn(0x7f2e2406c130 msgr2=0x7f2e2406c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:41:21.378 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e2ae39700 1 -- 172.21.15.161:0/3905300661 shutdown_connections 2024-02-13T03:41:21.378 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e2ae39700 1 -- 172.21.15.161:0/3905300661 wait complete. 2024-02-13T03:41:21.379 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e2ae39700 1 Processor -- start 2024-02-13T03:41:21.379 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e2ae39700 1 -- start start 2024-02-13T03:41:21.379 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e2ae39700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2e241157e0 con 0x7f2e241029e0 2024-02-13T03:41:21.379 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e2ae39700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2e241169c0 con 0x7f2e240feed0 2024-02-13T03:41:21.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e2ae39700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2e24117ba0 con 0x7f2e240fc060 2024-02-13T03:41:21.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e28bd5700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f2e240fc060 0x7f2e240f7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:44578/0 (socket says 172.21.15.161:44578) 2024-02-13T03:41:21.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e28bd5700 1 -- 172.21.15.161:0/4105393228 learned_addr learned my addr 172.21.15.161:0/4105393228 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:41:21.381 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237700640 0 0) 0x7f2e24117ba0 con 0x7f2e240fc060 2024-02-13T03:41:21.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e0c003030 con 0x7f2e240fc060 2024-02-13T03:41:21.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841715369 0 0) 0x7f2e241157e0 con 0x7f2e241029e0 2024-02-13T03:41:21.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e24117ba0 con 0x7f2e241029e0 2024-02-13T03:41:21.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187597867 0 0) 0x7f2e241169c0 con 0x7f2e240feed0 2024-02-13T03:41:21.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e241157e0 con 0x7f2e240feed0 2024-02-13T03:41:21.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2612084402 0 0) 0x7f2e24117ba0 con 0x7f2e241029e0 2024-02-13T03:41:21.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2e241169c0 con 0x7f2e241029e0 2024-02-13T03:41:21.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f2e1c001cb0 con 0x7f2e241029e0 2024-02-13T03:41:21.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2203189090 0 0) 0x7f2e0c003030 con 0x7f2e240fc060 2024-02-13T03:41:21.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2e24117ba0 con 0x7f2e240fc060 2024-02-13T03:41:21.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f2e20002910 con 0x7f2e240fc060 2024-02-13T03:41:21.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 285906834 0 0) 0x7f2e241157e0 con 0x7f2e240feed0 2024-02-13T03:41:21.383 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2e0c003030 con 0x7f2e240feed0 2024-02-13T03:41:21.383 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2117573253 0 0) 0x7f2e241169c0 con 0x7f2e241029e0 2024-02-13T03:41:21.383 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 >> v1:172.21.15.161:6790/0 conn(0x7f2e240fc060 legacy=0x7f2e240f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:21.383 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 >> v1:172.21.15.183:6789/0 conn(0x7f2e240feed0 legacy=0x7f2e240f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:21.383 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e24118dc0 con 0x7f2e241029e0 2024-02-13T03:41:21.383 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e2ae39700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e24060420 con 0x7f2e241029e0 2024-02-13T03:41:21.383 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e2ae39700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e240608f0 con 0x7f2e241029e0 2024-02-13T03:41:21.383 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.375+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f2e1c002770 con 0x7f2e241029e0 2024-02-13T03:41:21.384 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.379+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f2e1c003650 con 0x7f2e241029e0 2024-02-13T03:41:21.384 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.379+0000 7f2e2ae39700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2405f160 con 0x7f2e241029e0 2024-02-13T03:41:21.384 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.379+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f2e1c002290 con 0x7f2e241029e0 2024-02-13T03:41:21.384 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.379+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(730..730 src has 1..730) v4 ==== 7586+0+0 (unknown 1787452860 0 0) 0x7f2e1c076610 con 0x7f2e241029e0 2024-02-13T03:41:21.384 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.379+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=731}) v3 -- 0x7f2e241169c0 con 0x7f2e241029e0 2024-02-13T03:41:21.389 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.383+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f2e1c049be0 con 0x7f2e241029e0 2024-02-13T03:41:21.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:41:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:41:21] "GET /metrics HTTP/1.1" 200 188243 "" "Prometheus/2.33.4" 2024-02-13T03:41:21.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:21 smithi161 bash[27879]: cluster 2024-02-13T03:41:19.641772+0000 mgr.y (mgr.24473) 777 : cluster [DBG] pgmap v1265: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:21.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:21 smithi161 bash[19769]: cluster 2024-02-13T03:41:19.641772+0000 mgr.y (mgr.24473) 777 : cluster [DBG] pgmap v1265: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:21.564 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:21.559+0000 7f2e2ae39700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f2e240475a0 con 0x7f2e241029e0 2024-02-13T03:41:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:21 smithi183 bash[20824]: cluster 2024-02-13T03:41:19.641772+0000 mgr.y (mgr.24473) 777 : cluster [DBG] pgmap v1265: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:22.353 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:22.347+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.0 v1:172.21.15.161:6789/0 10 ==== osd_map(731..731 src has 1..731) v4 ==== 620+0+0 (unknown 1401300726 0 0) 0x7f2e1c049e10 con 0x7f2e241029e0 2024-02-13T03:41:22.354 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:22.351+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=732}) v3 -- 0x7f2e241157e0 con 0x7f2e241029e0 2024-02-13T03:41:22.359 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:22.355+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.0 v1:172.21.15.161:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v731) v1 ==== 217+0+0 (unknown 1660593040 0 0) 0x7f2e1c047ff0 con 0x7f2e241029e0 2024-02-13T03:41:22.475 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:22.471+0000 7f2e2ae39700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f2e24108500 con 0x7f2e241029e0 2024-02-13T03:41:22.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:22 smithi161 bash[19769]: audit 2024-02-13T03:41:21.564953+0000 mon.a (mon.0) 3180 : audit [INF] from='client.? v1:172.21.15.161:0/4105393228' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-13T03:41:22.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:22 smithi161 bash[19769]: audit 2024-02-13T03:41:21.703973+0000 mon.c (mon.2) 587 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:41:22.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:22 smithi161 bash[19769]: audit 2024-02-13T03:41:22.032300+0000 mon.c (mon.2) 588 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:41:22.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:22 smithi161 bash[19769]: audit 2024-02-13T03:41:22.033495+0000 mon.c (mon.2) 589 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:41:22.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:22 smithi161 bash[19769]: audit 2024-02-13T03:41:22.041977+0000 mon.a (mon.0) 3181 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:41:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:22 smithi161 bash[27879]: audit 2024-02-13T03:41:21.564953+0000 mon.a (mon.0) 3180 : audit [INF] from='client.? v1:172.21.15.161:0/4105393228' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-13T03:41:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:22 smithi161 bash[27879]: audit 2024-02-13T03:41:21.703973+0000 mon.c (mon.2) 587 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:41:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:22 smithi161 bash[27879]: audit 2024-02-13T03:41:22.032300+0000 mon.c (mon.2) 588 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:41:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:22 smithi161 bash[27879]: audit 2024-02-13T03:41:22.033495+0000 mon.c (mon.2) 589 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:41:22.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:22 smithi161 bash[27879]: audit 2024-02-13T03:41:22.041977+0000 mon.a (mon.0) 3181 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:41:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:22 smithi183 bash[20824]: audit 2024-02-13T03:41:21.564953+0000 mon.a (mon.0) 3180 : audit [INF] from='client.? v1:172.21.15.161:0/4105393228' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-13T03:41:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:22 smithi183 bash[20824]: audit 2024-02-13T03:41:21.703973+0000 mon.c (mon.2) 587 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:41:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:22 smithi183 bash[20824]: audit 2024-02-13T03:41:22.032300+0000 mon.c (mon.2) 588 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:41:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:22 smithi183 bash[20824]: audit 2024-02-13T03:41:22.033495+0000 mon.c (mon.2) 589 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:41:22.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:22 smithi183 bash[20824]: audit 2024-02-13T03:41:22.041977+0000 mon.a (mon.0) 3181 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:41:23.364 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.359+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.0 v1:172.21.15.161:6789/0 12 ==== osd_map(732..732 src has 1..732) v4 ==== 620+0+0 (unknown 4068225081 0 0) 0x7f2e1c04f760 con 0x7f2e241029e0 2024-02-13T03:41:23.364 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.359+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=733}) v3 -- 0x7f2e0c06fd80 con 0x7f2e241029e0 2024-02-13T03:41:23.368 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.363+0000 7f2e19ffb700 1 -- 172.21.15.161:0/4105393228 <== mon.0 v1:172.21.15.161:6789/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v732) v1 ==== 217+0+0 (unknown 1604546328 0 0) 0x7f2e1c049770 con 0x7f2e241029e0 2024-02-13T03:41:23.373 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.367+0000 7f2e2ae39700 1 -- 172.21.15.161:0/4105393228 >> v1:172.21.15.161:6800/4114337139 conn(0x7f2e0c0633a0 legacy=0x7f2e0c065860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:23.373 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.367+0000 7f2e2ae39700 1 -- 172.21.15.161:0/4105393228 >> v1:172.21.15.161:6789/0 conn(0x7f2e241029e0 legacy=0x7f2e24114030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:23.373 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.367+0000 7f2e2ae39700 1 -- 172.21.15.161:0/4105393228 shutdown_connections 2024-02-13T03:41:23.373 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.367+0000 7f2e2ae39700 1 -- 172.21.15.161:0/4105393228 >> 172.21.15.161:0/4105393228 conn(0x7f2e2406c130 msgr2=0x7f2e24106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:41:23.373 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.371+0000 7f2e2ae39700 1 -- 172.21.15.161:0/4105393228 shutdown_connections 2024-02-13T03:41:23.373 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.371+0000 7f2e2ae39700 1 -- 172.21.15.161:0/4105393228 wait complete. 2024-02-13T03:41:23.374 INFO:tasks.workunit.client.0.smithi161.stderr:set-quota max_bytes = 0 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa 2024-02-13T03:41:23.386 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool set-quota 140c6d73-9bda-4f05-af7a-8f6a188328fa max_objects 0 2024-02-13T03:41:23.511 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7523b1700 1 -- 172.21.15.161:0/2646626281 >> v1:172.21.15.161:6789/0 conn(0x7fd74c0f89e0 legacy=0x7fd74c0f8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:23.511 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7523b1700 1 -- 172.21.15.161:0/2646626281 shutdown_connections 2024-02-13T03:41:23.511 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7523b1700 1 -- 172.21.15.161:0/2646626281 >> 172.21.15.161:0/2646626281 conn(0x7fd74c0f2530 msgr2=0x7fd74c0f4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:41:23.511 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7523b1700 1 -- 172.21.15.161:0/2646626281 shutdown_connections 2024-02-13T03:41:23.511 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7523b1700 1 -- 172.21.15.161:0/2646626281 wait complete. 2024-02-13T03:41:23.512 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7523b1700 1 Processor -- start 2024-02-13T03:41:23.512 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7523b1700 1 -- start start 2024-02-13T03:41:23.512 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7523b1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd74c1173c0 con 0x7fd74c0f94d0 2024-02-13T03:41:23.512 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7523b1700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd74c118560 con 0x7fd74c0f89e0 2024-02-13T03:41:23.513 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7523b1700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd74c119760 con 0x7fd74c104340 2024-02-13T03:41:23.513 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd75094e700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7fd74c104340 0x7fd74c115a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:44604/0 (socket says 172.21.15.161:44604) 2024-02-13T03:41:23.513 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd75094e700 1 -- 172.21.15.161:0/3163649295 learned_addr learned my addr 172.21.15.161:0/3163649295 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:41:23.515 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995988238 0 0) 0x7fd74c119760 con 0x7fd74c104340 2024-02-13T03:41:23.515 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7380033b0 con 0x7fd74c104340 2024-02-13T03:41:23.515 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 62009616 0 0) 0x7fd74c1173c0 con 0x7fd74c0f94d0 2024-02-13T03:41:23.515 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd74c119760 con 0x7fd74c0f94d0 2024-02-13T03:41:23.515 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438032708 0 0) 0x7fd74c118560 con 0x7fd74c0f89e0 2024-02-13T03:41:23.515 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.507+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd74c1173c0 con 0x7fd74c0f89e0 2024-02-13T03:41:23.515 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2821488661 0 0) 0x7fd7380033b0 con 0x7fd74c104340 2024-02-13T03:41:23.515 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd74c118560 con 0x7fd74c104340 2024-02-13T03:41:23.515 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fd744001be0 con 0x7fd74c104340 2024-02-13T03:41:23.516 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 148413178 0 0) 0x7fd74c119760 con 0x7fd74c0f94d0 2024-02-13T03:41:23.516 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd7380033b0 con 0x7fd74c0f94d0 2024-02-13T03:41:23.516 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fd73c0019d0 con 0x7fd74c0f94d0 2024-02-13T03:41:23.516 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3007322715 0 0) 0x7fd74c1173c0 con 0x7fd74c0f89e0 2024-02-13T03:41:23.516 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd74c119760 con 0x7fd74c0f89e0 2024-02-13T03:41:23.516 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fd734002dc0 con 0x7fd74c0f89e0 2024-02-13T03:41:23.516 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1443719520 0 0) 0x7fd74c118560 con 0x7fd74c104340 2024-02-13T03:41:23.516 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 >> v1:172.21.15.183:6789/0 conn(0x7fd74c0f89e0 legacy=0x7fd74c0ff590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:23.516 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 >> v1:172.21.15.161:6789/0 conn(0x7fd74c0f94d0 legacy=0x7fd74c0ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:23.516 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74c11a960 con 0x7fd74c104340 2024-02-13T03:41:23.516 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7523b1700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74c1175b0 con 0x7fd74c104340 2024-02-13T03:41:23.516 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7523b1700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74c117a00 con 0x7fd74c104340 2024-02-13T03:41:23.516 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fd744001f90 con 0x7fd74c104340 2024-02-13T03:41:23.517 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fd744005e20 con 0x7fd74c104340 2024-02-13T03:41:23.517 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7523b1700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72c004ee0 con 0x7fd74c104340 2024-02-13T03:41:23.517 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.511+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fd7440025b0 con 0x7fd74c104340 2024-02-13T03:41:23.517 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.515+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(732..732 src has 1..732) v4 ==== 7586+0+0 (unknown 3951696820 0 0) 0x7fd744078c50 con 0x7fd74c104340 2024-02-13T03:41:23.518 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.515+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=733}) v3 -- 0x7fd74c118560 con 0x7fd74c104340 2024-02-13T03:41:23.523 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.519+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fd74404c280 con 0x7fd74c104340 2024-02-13T03:41:23.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:23 smithi161 bash[19769]: cluster 2024-02-13T03:41:21.643149+0000 mgr.y (mgr.24473) 778 : cluster [DBG] pgmap v1266: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-13T03:41:23.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:23 smithi161 bash[19769]: audit 2024-02-13T03:41:22.359554+0000 mon.a (mon.0) 3182 : audit [INF] from='client.? v1:172.21.15.161:0/4105393228' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]': finished 2024-02-13T03:41:23.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:23 smithi161 bash[19769]: cluster 2024-02-13T03:41:22.359637+0000 mon.a (mon.0) 3183 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-02-13T03:41:23.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:23 smithi161 bash[19769]: audit 2024-02-13T03:41:22.475731+0000 mon.a (mon.0) 3184 : audit [INF] from='client.? v1:172.21.15.161:0/4105393228' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-13T03:41:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:23 smithi161 bash[27879]: cluster 2024-02-13T03:41:21.643149+0000 mgr.y (mgr.24473) 778 : cluster [DBG] pgmap v1266: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-13T03:41:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:23 smithi161 bash[27879]: audit 2024-02-13T03:41:22.359554+0000 mon.a (mon.0) 3182 : audit [INF] from='client.? v1:172.21.15.161:0/4105393228' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]': finished 2024-02-13T03:41:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:23 smithi161 bash[27879]: cluster 2024-02-13T03:41:22.359637+0000 mon.a (mon.0) 3183 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-02-13T03:41:23.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:23 smithi161 bash[27879]: audit 2024-02-13T03:41:22.475731+0000 mon.a (mon.0) 3184 : audit [INF] from='client.? v1:172.21.15.161:0/4105393228' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-13T03:41:23.699 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:23.695+0000 7fd7523b1700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fd72c005190 con 0x7fd74c104340 2024-02-13T03:41:23.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:23 smithi183 bash[20824]: cluster 2024-02-13T03:41:21.643149+0000 mgr.y (mgr.24473) 778 : cluster [DBG] pgmap v1266: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-13T03:41:23.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:23 smithi183 bash[20824]: audit 2024-02-13T03:41:22.359554+0000 mon.a (mon.0) 3182 : audit [INF] from='client.? v1:172.21.15.161:0/4105393228' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]': finished 2024-02-13T03:41:23.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:23 smithi183 bash[20824]: cluster 2024-02-13T03:41:22.359637+0000 mon.a (mon.0) 3183 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-02-13T03:41:23.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:23 smithi183 bash[20824]: audit 2024-02-13T03:41:22.475731+0000 mon.a (mon.0) 3184 : audit [INF] from='client.? v1:172.21.15.161:0/4105393228' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-13T03:41:24.121 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:24.115+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.2 v1:172.21.15.161:6790/0 10 ==== osd_map(733..733 src has 1..733) v4 ==== 620+0+0 (unknown 895092641 0 0) 0x7fd74404c4b0 con 0x7fd74c104340 2024-02-13T03:41:24.121 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:24.115+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=734}) v3 -- 0x7fd7380033b0 con 0x7fd74c104340 2024-02-13T03:41:24.127 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:24.123+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.2 v1:172.21.15.161:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v733) v1 ==== 221+0+0 (unknown 840967021 0 0) 0x7fd74404a690 con 0x7fd74c104340 2024-02-13T03:41:24.243 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:24.239+0000 7fd7523b1700 1 -- 172.21.15.161:0/3163649295 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fd72c0058d0 con 0x7fd74c104340 2024-02-13T03:41:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[27879]: audit 2024-02-13T03:41:23.367458+0000 mon.a (mon.0) 3185 : audit [INF] from='client.? v1:172.21.15.161:0/4105393228' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]': finished 2024-02-13T03:41:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[27879]: cluster 2024-02-13T03:41:23.367542+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-02-13T03:41:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[27879]: audit 2024-02-13T03:41:23.699901+0000 mon.c (mon.2) 590 : audit [INF] from='client.? v1:172.21.15.161:0/3163649295' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:41:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[27879]: audit 2024-02-13T03:41:23.700576+0000 mon.a (mon.0) 3187 : audit [INF] from='client.48551 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:41:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[27879]: cluster 2024-02-13T03:41:24.115924+0000 mon.a (mon.0) 3188 : cluster [INF] pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' no longer out of quota; removing NO_QUOTA flag 2024-02-13T03:41:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[27879]: cluster 2024-02-13T03:41:24.116374+0000 mon.a (mon.0) 3189 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-13T03:41:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[27879]: audit 2024-02-13T03:41:24.124965+0000 mon.a (mon.0) 3190 : audit [INF] from='client.48551 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]': finished 2024-02-13T03:41:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[27879]: cluster 2024-02-13T03:41:24.125030+0000 mon.a (mon.0) 3191 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-02-13T03:41:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[27879]: audit 2024-02-13T03:41:24.243908+0000 mon.c (mon.2) 591 : audit [INF] from='client.? v1:172.21.15.161:0/3163649295' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:41:24.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[27879]: audit 2024-02-13T03:41:24.244662+0000 mon.a (mon.0) 3192 : audit [INF] from='client.48551 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:41:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[19769]: audit 2024-02-13T03:41:23.367458+0000 mon.a (mon.0) 3185 : audit [INF] from='client.? v1:172.21.15.161:0/4105393228' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]': finished 2024-02-13T03:41:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[19769]: cluster 2024-02-13T03:41:23.367542+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-02-13T03:41:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[19769]: audit 2024-02-13T03:41:23.699901+0000 mon.c (mon.2) 590 : audit [INF] from='client.? v1:172.21.15.161:0/3163649295' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:41:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[19769]: audit 2024-02-13T03:41:23.700576+0000 mon.a (mon.0) 3187 : audit [INF] from='client.48551 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:41:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[19769]: cluster 2024-02-13T03:41:24.115924+0000 mon.a (mon.0) 3188 : cluster [INF] pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' no longer out of quota; removing NO_QUOTA flag 2024-02-13T03:41:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[19769]: cluster 2024-02-13T03:41:24.116374+0000 mon.a (mon.0) 3189 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-13T03:41:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[19769]: audit 2024-02-13T03:41:24.124965+0000 mon.a (mon.0) 3190 : audit [INF] from='client.48551 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]': finished 2024-02-13T03:41:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[19769]: cluster 2024-02-13T03:41:24.125030+0000 mon.a (mon.0) 3191 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-02-13T03:41:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[19769]: audit 2024-02-13T03:41:24.243908+0000 mon.c (mon.2) 591 : audit [INF] from='client.? v1:172.21.15.161:0/3163649295' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:41:24.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:24 smithi161 bash[19769]: audit 2024-02-13T03:41:24.244662+0000 mon.a (mon.0) 3192 : audit [INF] from='client.48551 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:41:24.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:24 smithi183 bash[20824]: audit 2024-02-13T03:41:23.367458+0000 mon.a (mon.0) 3185 : audit [INF] from='client.? v1:172.21.15.161:0/4105393228' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]': finished 2024-02-13T03:41:24.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:24 smithi183 bash[20824]: cluster 2024-02-13T03:41:23.367542+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-02-13T03:41:24.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:24 smithi183 bash[20824]: audit 2024-02-13T03:41:23.699901+0000 mon.c (mon.2) 590 : audit [INF] from='client.? v1:172.21.15.161:0/3163649295' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:41:24.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:24 smithi183 bash[20824]: audit 2024-02-13T03:41:23.700576+0000 mon.a (mon.0) 3187 : audit [INF] from='client.48551 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:41:24.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:24 smithi183 bash[20824]: cluster 2024-02-13T03:41:24.115924+0000 mon.a (mon.0) 3188 : cluster [INF] pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' no longer out of quota; removing NO_QUOTA flag 2024-02-13T03:41:24.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:24 smithi183 bash[20824]: cluster 2024-02-13T03:41:24.116374+0000 mon.a (mon.0) 3189 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-13T03:41:24.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:24 smithi183 bash[20824]: audit 2024-02-13T03:41:24.124965+0000 mon.a (mon.0) 3190 : audit [INF] from='client.48551 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]': finished 2024-02-13T03:41:24.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:24 smithi183 bash[20824]: cluster 2024-02-13T03:41:24.125030+0000 mon.a (mon.0) 3191 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-02-13T03:41:24.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:24 smithi183 bash[20824]: audit 2024-02-13T03:41:24.243908+0000 mon.c (mon.2) 591 : audit [INF] from='client.? v1:172.21.15.161:0/3163649295' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:41:24.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:24 smithi183 bash[20824]: audit 2024-02-13T03:41:24.244662+0000 mon.a (mon.0) 3192 : audit [INF] from='client.48551 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:41:25.136 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.131+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.2 v1:172.21.15.161:6790/0 12 ==== osd_map(734..734 src has 1..734) v4 ==== 620+0+0 (unknown 1257204289 0 0) 0x7fd744051e00 con 0x7fd74c104340 2024-02-13T03:41:25.144 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.139+0000 7fd7497fa700 1 -- 172.21.15.161:0/3163649295 <== mon.2 v1:172.21.15.161:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v734) v1 ==== 221+0+0 (unknown 3074238399 0 0) 0x7fd74404be10 con 0x7fd74c104340 2024-02-13T03:41:25.147 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.143+0000 7fd7523b1700 1 -- 172.21.15.161:0/3163649295 >> v1:172.21.15.161:6800/4114337139 conn(0x7fd738062260 legacy=0x7fd738064720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:25.147 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.143+0000 7fd7523b1700 1 -- 172.21.15.161:0/3163649295 >> v1:172.21.15.161:6790/0 conn(0x7fd74c104340 legacy=0x7fd74c115a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:25.147 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.143+0000 7fd7523b1700 1 -- 172.21.15.161:0/3163649295 shutdown_connections 2024-02-13T03:41:25.147 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.143+0000 7fd7523b1700 1 -- 172.21.15.161:0/3163649295 >> 172.21.15.161:0/3163649295 conn(0x7fd74c0f2530 msgr2=0x7fd74c1077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:41:25.147 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.143+0000 7fd7523b1700 1 -- 172.21.15.161:0/3163649295 shutdown_connections 2024-02-13T03:41:25.147 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.143+0000 7fd7523b1700 1 -- 172.21.15.161:0/3163649295 wait complete. 2024-02-13T03:41:25.148 INFO:tasks.workunit.client.0.smithi161.stderr:set-quota max_objects = 0 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa 2024-02-13T03:41:25.159 INFO:tasks.workunit.client.0.smithi161.stderr:+ wait 128538 2024-02-13T03:41:25.159 INFO:tasks.workunit.client.0.smithi161.stderr:+ [ 0 -ne 0 ] 2024-02-13T03:41:25.159 INFO:tasks.workunit.client.0.smithi161.stderr:+ true 2024-02-13T03:41:25.159 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put three /etc/passwd 2024-02-13T03:41:25.197 INFO:tasks.workunit.client.0.smithi161.stderr:+ uuidgen 2024-02-13T03:41:25.199 INFO:tasks.workunit.client.0.smithi161.stderr:+ pp=3842dba2-777f-4b83-b6dc-f0e3b09c5464 2024-02-13T03:41:25.199 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool create 3842dba2-777f-4b83-b6dc-f0e3b09c5464 12 2024-02-13T03:41:25.320 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.315+0000 7f1622eb6700 1 -- 172.21.15.161:0/1192511854 >> v1:172.21.15.161:6790/0 conn(0x7f161c100a70 legacy=0x7f161c102e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:25.320 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.315+0000 7f1622eb6700 1 -- 172.21.15.161:0/1192511854 shutdown_connections 2024-02-13T03:41:25.320 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.315+0000 7f1622eb6700 1 -- 172.21.15.161:0/1192511854 >> 172.21.15.161:0/1192511854 conn(0x7f161c06c130 msgr2=0x7f161c06c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:41:25.320 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.315+0000 7f1622eb6700 1 -- 172.21.15.161:0/1192511854 shutdown_connections 2024-02-13T03:41:25.320 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.315+0000 7f1622eb6700 1 -- 172.21.15.161:0/1192511854 wait complete. 2024-02-13T03:41:25.321 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.315+0000 7f1622eb6700 1 Processor -- start 2024-02-13T03:41:25.321 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.315+0000 7f1622eb6700 1 -- start start 2024-02-13T03:41:25.321 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.315+0000 7f1622eb6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f161c117310 con 0x7f161c104580 2024-02-13T03:41:25.321 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.315+0000 7f1622eb6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f161c1184b0 con 0x7f161c0fdc20 2024-02-13T03:41:25.321 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.315+0000 7f1622eb6700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f161c119690 con 0x7f161c100a70 2024-02-13T03:41:25.321 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.315+0000 7f161bfff700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f161c100a70 0x7f161c0fa030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:44646/0 (socket says 172.21.15.161:44646) 2024-02-13T03:41:25.321 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.315+0000 7f161bfff700 1 -- 172.21.15.161:0/2853976981 learned_addr learned my addr 172.21.15.161:0/2853976981 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:41:25.323 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851194125 0 0) 0x7f161c119690 con 0x7f161c100a70 2024-02-13T03:41:25.323 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1608003110 con 0x7f161c100a70 2024-02-13T03:41:25.323 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460069283 0 0) 0x7f161c117310 con 0x7f161c104580 2024-02-13T03:41:25.323 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f161c119690 con 0x7f161c104580 2024-02-13T03:41:25.323 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196134881 0 0) 0x7f161c1184b0 con 0x7f161c0fdc20 2024-02-13T03:41:25.323 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f161c117310 con 0x7f161c0fdc20 2024-02-13T03:41:25.323 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3147485003 0 0) 0x7f161c119690 con 0x7f161c104580 2024-02-13T03:41:25.323 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f161c1184b0 con 0x7f161c104580 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f16140019e0 con 0x7f161c104580 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2978085472 0 0) 0x7f1608003110 con 0x7f161c100a70 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f161c119690 con 0x7f161c100a70 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f160c002b60 con 0x7f161c100a70 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2453988248 0 0) 0x7f161c117310 con 0x7f161c0fdc20 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1608003110 con 0x7f161c0fdc20 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f1604001800 con 0x7f161c0fdc20 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2493062635 0 0) 0x7f161c119690 con 0x7f161c100a70 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 >> v1:172.21.15.183:6789/0 conn(0x7f161c0fdc20 legacy=0x7f161c0f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 >> v1:172.21.15.161:6789/0 conn(0x7f161c104580 legacy=0x7f161c115bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c11a870 con 0x7f161c100a70 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1622eb6700 1 -- 172.21.15.161:0/2853976981 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161c117500 con 0x7f161c100a70 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f160c001a90 con 0x7f161c100a70 2024-02-13T03:41:25.324 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1622eb6700 1 -- 172.21.15.161:0/2853976981 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f161c1179d0 con 0x7f161c100a70 2024-02-13T03:41:25.325 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f160c0038c0 con 0x7f161c100a70 2024-02-13T03:41:25.325 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1622eb6700 1 -- 172.21.15.161:0/2853976981 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161c05f160 con 0x7f161c100a70 2024-02-13T03:41:25.325 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f160c003b60 con 0x7f161c100a70 2024-02-13T03:41:25.325 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.319+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(734..734 src has 1..734) v4 ==== 7586+0+0 (unknown 1747848163 0 0) 0x7f160c078f50 con 0x7f161c100a70 2024-02-13T03:41:25.334 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.331+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f160c04c5c0 con 0x7f161c100a70 2024-02-13T03:41:25.507 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:25.503+0000 7f1622eb6700 1 -- 172.21.15.161:0/2853976981 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12} v 0) v1 -- 0x7f161c0475a0 con 0x7f161c100a70 2024-02-13T03:41:25.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:25 smithi161 bash[27879]: cluster 2024-02-13T03:41:23.643889+0000 mgr.y (mgr.24473) 779 : cluster [DBG] pgmap v1269: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s 2024-02-13T03:41:25.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:25 smithi161 bash[27879]: audit 2024-02-13T03:41:25.141359+0000 mon.a (mon.0) 3193 : audit [INF] from='client.48551 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]': finished 2024-02-13T03:41:25.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:25 smithi161 bash[27879]: cluster 2024-02-13T03:41:25.142817+0000 mon.a (mon.0) 3194 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-02-13T03:41:25.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:25 smithi161 bash[19769]: cluster 2024-02-13T03:41:23.643889+0000 mgr.y (mgr.24473) 779 : cluster [DBG] pgmap v1269: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s 2024-02-13T03:41:25.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:25 smithi161 bash[19769]: audit 2024-02-13T03:41:25.141359+0000 mon.a (mon.0) 3193 : audit [INF] from='client.48551 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]': finished 2024-02-13T03:41:25.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:25 smithi161 bash[19769]: cluster 2024-02-13T03:41:25.142817+0000 mon.a (mon.0) 3194 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-02-13T03:41:25.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:25 smithi183 bash[20824]: cluster 2024-02-13T03:41:23.643889+0000 mgr.y (mgr.24473) 779 : cluster [DBG] pgmap v1269: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s 2024-02-13T03:41:25.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:25 smithi183 bash[20824]: audit 2024-02-13T03:41:25.141359+0000 mon.a (mon.0) 3193 : audit [INF] from='client.48551 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]': finished 2024-02-13T03:41:25.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:25 smithi183 bash[20824]: cluster 2024-02-13T03:41:25.142817+0000 mon.a (mon.0) 3194 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-02-13T03:41:26.398 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.395+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.2 v1:172.21.15.161:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]=0 pool '3842dba2-777f-4b83-b6dc-f0e3b09c5464' created v735) v1 ==== 176+0+0 (unknown 1661331282 0 0) 0x7f160c04aa30 con 0x7f161c100a70 2024-02-13T03:41:26.507 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.503+0000 7f1622eb6700 1 -- 172.21.15.161:0/2853976981 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12} v 0) v1 -- 0x7f161c060a90 con 0x7f161c100a70 2024-02-13T03:41:26.508 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.503+0000 7f1619ffb700 1 -- 172.21.15.161:0/2853976981 <== mon.2 v1:172.21.15.161:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]=0 pool '3842dba2-777f-4b83-b6dc-f0e3b09c5464' already exists v735) v1 ==== 183+0+0 (unknown 2389965998 0 0) 0x7f160c04c1b0 con 0x7f161c100a70 2024-02-13T03:41:26.511 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.507+0000 7f1622eb6700 1 -- 172.21.15.161:0/2853976981 >> v1:172.21.15.161:6800/4114337139 conn(0x7f1608062310 legacy=0x7f16080647d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:26.511 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.507+0000 7f1622eb6700 1 -- 172.21.15.161:0/2853976981 >> v1:172.21.15.161:6790/0 conn(0x7f161c100a70 legacy=0x7f161c0fa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:26.511 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.507+0000 7f1622eb6700 1 -- 172.21.15.161:0/2853976981 shutdown_connections 2024-02-13T03:41:26.511 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.507+0000 7f1622eb6700 1 -- 172.21.15.161:0/2853976981 >> 172.21.15.161:0/2853976981 conn(0x7f161c06c130 msgr2=0x7f161c1049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:41:26.511 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.507+0000 7f1622eb6700 1 -- 172.21.15.161:0/2853976981 shutdown_connections 2024-02-13T03:41:26.511 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.507+0000 7f1622eb6700 1 -- 172.21.15.161:0/2853976981 wait complete. 2024-02-13T03:41:26.512 INFO:tasks.workunit.client.0.smithi161.stderr:pool '3842dba2-777f-4b83-b6dc-f0e3b09c5464' already exists 2024-02-13T03:41:26.521 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool application enable 3842dba2-777f-4b83-b6dc-f0e3b09c5464 rados 2024-02-13T03:41:26.651 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5962090700 1 -- 172.21.15.161:0/3234376800 >> v1:172.21.15.161:6789/0 conn(0x7f595c103240 legacy=0x7f595c1056e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:26.651 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5962090700 1 -- 172.21.15.161:0/3234376800 shutdown_connections 2024-02-13T03:41:26.651 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5962090700 1 -- 172.21.15.161:0/3234376800 >> 172.21.15.161:0/3234376800 conn(0x7f595c0f2570 msgr2=0x7f595c0f4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:41:26.651 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5962090700 1 -- 172.21.15.161:0/3234376800 shutdown_connections 2024-02-13T03:41:26.651 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5962090700 1 -- 172.21.15.161:0/3234376800 wait complete. 2024-02-13T03:41:26.651 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5962090700 1 Processor -- start 2024-02-13T03:41:26.651 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5962090700 1 -- start start 2024-02-13T03:41:26.651 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5962090700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f595c119450 con 0x7f595c103240 2024-02-13T03:41:26.651 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5962090700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f595c11a650 con 0x7f595c0fc910 2024-02-13T03:41:26.654 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5962090700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f595c11b850 con 0x7f595c0ff730 2024-02-13T03:41:26.654 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f595affd700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f595c0ff730 0x7f595c061380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:44662/0 (socket says 172.21.15.161:44662) 2024-02-13T03:41:26.654 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f595affd700 1 -- 172.21.15.161:0/3859725148 learned_addr learned my addr 172.21.15.161:0/3859725148 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:41:26.654 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951572458 0 0) 0x7f595c11b850 con 0x7f595c0ff730 2024-02-13T03:41:26.654 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59480035f0 con 0x7f595c0ff730 2024-02-13T03:41:26.654 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597124415 0 0) 0x7f595c119450 con 0x7f595c103240 2024-02-13T03:41:26.654 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f595c11b850 con 0x7f595c103240 2024-02-13T03:41:26.654 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946678672 0 0) 0x7f595c11a650 con 0x7f595c0fc910 2024-02-13T03:41:26.654 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f595c119450 con 0x7f595c0fc910 2024-02-13T03:41:26.654 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2650432731 0 0) 0x7f595c11b850 con 0x7f595c103240 2024-02-13T03:41:26.654 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f595c11a650 con 0x7f595c103240 2024-02-13T03:41:26.654 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2568793422 0 0) 0x7f59480035f0 con 0x7f595c0ff730 2024-02-13T03:41:26.654 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f595c11b850 con 0x7f595c0ff730 2024-02-13T03:41:26.655 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5954002fb0 con 0x7f595c103240 2024-02-13T03:41:26.655 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f594c001880 con 0x7f595c0ff730 2024-02-13T03:41:26.655 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2100804865 0 0) 0x7f595c11a650 con 0x7f595c103240 2024-02-13T03:41:26.655 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 >> v1:172.21.15.161:6790/0 conn(0x7f595c0ff730 legacy=0x7f595c061380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:26.655 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 >> v1:172.21.15.183:6789/0 conn(0x7f595c0fc910 legacy=0x7f595c060c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:26.655 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c11ca50 con 0x7f595c103240 2024-02-13T03:41:26.655 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5962090700 1 -- 172.21.15.161:0/3859725148 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595c119640 con 0x7f595c103240 2024-02-13T03:41:26.655 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5962090700 1 -- 172.21.15.161:0/3859725148 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f595c119a90 con 0x7f595c103240 2024-02-13T03:41:26.655 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f5954001570 con 0x7f595c103240 2024-02-13T03:41:26.655 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.647+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5954003a80 con 0x7f595c103240 2024-02-13T03:41:26.656 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.651+0000 7f5962090700 1 -- 172.21.15.161:0/3859725148 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593c004ee0 con 0x7f595c103240 2024-02-13T03:41:26.656 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.651+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f5954002550 con 0x7f595c103240 2024-02-13T03:41:26.656 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.651+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(735..735 src has 1..735) v4 ==== 7961+0+0 (unknown 1179706430 0 0) 0x7f5954076b60 con 0x7f595c103240 2024-02-13T03:41:26.660 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.655+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f595404a070 con 0x7f595c103240 2024-02-13T03:41:26.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:26 smithi161 bash[27879]: audit 2024-02-13T03:41:25.507399+0000 mon.c (mon.2) 592 : audit [INF] from='client.? v1:172.21.15.161:0/2853976981' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]: dispatch 2024-02-13T03:41:26.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:26 smithi161 bash[27879]: audit 2024-02-13T03:41:25.508169+0000 mon.a (mon.0) 3195 : audit [INF] from='client.48566 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]: dispatch 2024-02-13T03:41:26.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:26 smithi161 bash[19769]: audit 2024-02-13T03:41:25.507399+0000 mon.c (mon.2) 592 : audit [INF] from='client.? v1:172.21.15.161:0/2853976981' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]: dispatch 2024-02-13T03:41:26.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:26 smithi161 bash[19769]: audit 2024-02-13T03:41:25.508169+0000 mon.a (mon.0) 3195 : audit [INF] from='client.48566 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]: dispatch 2024-02-13T03:41:26.737 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:41:26 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:41:26.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:26 smithi183 bash[20824]: audit 2024-02-13T03:41:25.507399+0000 mon.c (mon.2) 592 : audit [INF] from='client.? v1:172.21.15.161:0/2853976981' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]: dispatch 2024-02-13T03:41:26.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:26 smithi183 bash[20824]: audit 2024-02-13T03:41:25.508169+0000 mon.a (mon.0) 3195 : audit [INF] from='client.48566 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]: dispatch 2024-02-13T03:41:26.845 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:26.839+0000 7f5962090700 1 -- 172.21.15.161:0/3859725148 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"} v 0) v1 -- 0x7f593c005190 con 0x7f595c103240 2024-02-13T03:41:27.397 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:27.391+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]=0 enabled application 'rados' on pool '3842dba2-777f-4b83-b6dc-f0e3b09c5464' v736) v1 ==== 213+0+0 (unknown 2344144975 0 0) 0x7f59540484e0 con 0x7f595c103240 2024-02-13T03:41:27.527 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:27.523+0000 7f5962090700 1 -- 172.21.15.161:0/3859725148 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"} v 0) v1 -- 0x7f593c002220 con 0x7f595c103240 2024-02-13T03:41:27.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:27 smithi161 bash[27879]: cluster 2024-02-13T03:41:25.644754+0000 mgr.y (mgr.24473) 780 : cluster [DBG] pgmap v1272: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:41:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:27 smithi161 bash[27879]: audit 2024-02-13T03:41:26.390081+0000 mon.a (mon.0) 3196 : audit [INF] from='client.48566 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]': finished 2024-02-13T03:41:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:27 smithi161 bash[27879]: cluster 2024-02-13T03:41:26.390181+0000 mon.a (mon.0) 3197 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-02-13T03:41:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:27 smithi161 bash[27879]: audit 2024-02-13T03:41:26.507520+0000 mon.c (mon.2) 593 : audit [INF] from='client.? v1:172.21.15.161:0/2853976981' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]: dispatch 2024-02-13T03:41:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:27 smithi161 bash[27879]: audit 2024-02-13T03:41:26.508255+0000 mon.a (mon.0) 3198 : audit [INF] from='client.48566 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]: dispatch 2024-02-13T03:41:27.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:27 smithi161 bash[27879]: audit 2024-02-13T03:41:26.845398+0000 mon.a (mon.0) 3199 : audit [INF] from='client.? v1:172.21.15.161:0/3859725148' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]: dispatch 2024-02-13T03:41:27.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:27 smithi161 bash[19769]: cluster 2024-02-13T03:41:25.644754+0000 mgr.y (mgr.24473) 780 : cluster [DBG] pgmap v1272: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:41:27.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:27 smithi161 bash[19769]: audit 2024-02-13T03:41:26.390081+0000 mon.a (mon.0) 3196 : audit [INF] from='client.48566 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]': finished 2024-02-13T03:41:27.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:27 smithi161 bash[19769]: cluster 2024-02-13T03:41:26.390181+0000 mon.a (mon.0) 3197 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-02-13T03:41:27.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:27 smithi161 bash[19769]: audit 2024-02-13T03:41:26.507520+0000 mon.c (mon.2) 593 : audit [INF] from='client.? v1:172.21.15.161:0/2853976981' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]: dispatch 2024-02-13T03:41:27.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:27 smithi161 bash[19769]: audit 2024-02-13T03:41:26.508255+0000 mon.a (mon.0) 3198 : audit [INF] from='client.48566 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]: dispatch 2024-02-13T03:41:27.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:27 smithi161 bash[19769]: audit 2024-02-13T03:41:26.845398+0000 mon.a (mon.0) 3199 : audit [INF] from='client.? v1:172.21.15.161:0/3859725148' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]: dispatch 2024-02-13T03:41:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:27 smithi183 bash[20824]: cluster 2024-02-13T03:41:25.644754+0000 mgr.y (mgr.24473) 780 : cluster [DBG] pgmap v1272: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:41:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:27 smithi183 bash[20824]: audit 2024-02-13T03:41:26.390081+0000 mon.a (mon.0) 3196 : audit [INF] from='client.48566 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]': finished 2024-02-13T03:41:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:27 smithi183 bash[20824]: cluster 2024-02-13T03:41:26.390181+0000 mon.a (mon.0) 3197 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-02-13T03:41:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:27 smithi183 bash[20824]: audit 2024-02-13T03:41:26.507520+0000 mon.c (mon.2) 593 : audit [INF] from='client.? v1:172.21.15.161:0/2853976981' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]: dispatch 2024-02-13T03:41:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:27 smithi183 bash[20824]: audit 2024-02-13T03:41:26.508255+0000 mon.a (mon.0) 3198 : audit [INF] from='client.48566 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pg_num": 12}]: dispatch 2024-02-13T03:41:27.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:27 smithi183 bash[20824]: audit 2024-02-13T03:41:26.845398+0000 mon.a (mon.0) 3199 : audit [INF] from='client.? v1:172.21.15.161:0/3859725148' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]: dispatch 2024-02-13T03:41:28.414 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.411+0000 7f5958ff9700 1 -- 172.21.15.161:0/3859725148 <== mon.0 v1:172.21.15.161:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]=0 enabled application 'rados' on pool '3842dba2-777f-4b83-b6dc-f0e3b09c5464' v737) v1 ==== 213+0+0 (unknown 2979604137 0 0) 0x7f5954049c60 con 0x7f595c103240 2024-02-13T03:41:28.419 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.415+0000 7f5962090700 1 -- 172.21.15.161:0/3859725148 >> v1:172.21.15.161:6800/4114337139 conn(0x7f5948061ff0 legacy=0x7f59480644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:28.419 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.415+0000 7f5962090700 1 -- 172.21.15.161:0/3859725148 >> v1:172.21.15.161:6789/0 conn(0x7f595c103240 legacy=0x7f595c117ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:28.419 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.415+0000 7f5962090700 1 -- 172.21.15.161:0/3859725148 shutdown_connections 2024-02-13T03:41:28.420 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.415+0000 7f5962090700 1 -- 172.21.15.161:0/3859725148 >> 172.21.15.161:0/3859725148 conn(0x7f595c0f2570 msgr2=0x7f595c1066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:41:28.420 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.415+0000 7f5962090700 1 -- 172.21.15.161:0/3859725148 shutdown_connections 2024-02-13T03:41:28.420 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.415+0000 7f5962090700 1 -- 172.21.15.161:0/3859725148 wait complete. 2024-02-13T03:41:28.420 INFO:tasks.workunit.client.0.smithi161.stderr:enabled application 'rados' on pool '3842dba2-777f-4b83-b6dc-f0e3b09c5464' 2024-02-13T03:41:28.433 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool set-quota 3842dba2-777f-4b83-b6dc-f0e3b09c5464 max_objects 10 2024-02-13T03:41:28.554 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1917708467 >> v1:172.21.15.161:6789/0 conn(0x7fa6a01065d0 legacy=0x7fa6a01089c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:28.554 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1917708467 shutdown_connections 2024-02-13T03:41:28.555 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1917708467 >> 172.21.15.161:0/1917708467 conn(0x7fa6a00f2530 msgr2=0x7fa6a00f4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:41:28.555 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1917708467 shutdown_connections 2024-02-13T03:41:28.555 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1917708467 wait complete. 2024-02-13T03:41:28.555 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa6a6ba9700 1 Processor -- start 2024-02-13T03:41:28.555 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa6a6ba9700 1 -- start start 2024-02-13T03:41:28.555 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa6a6ba9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa6a0119750 con 0x7fa6a01065d0 2024-02-13T03:41:28.555 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa6a6ba9700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa6a011a950 con 0x7fa6a00f6830 2024-02-13T03:41:28.555 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa6a6ba9700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa6a011bb50 con 0x7fa6a00f7320 2024-02-13T03:41:28.555 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa69ffff700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7fa6a00f7320 0x7fa6a0103430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:43132/0 (socket says 172.21.15.161:43132) 2024-02-13T03:41:28.555 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa69ffff700 1 -- 172.21.15.161:0/1431887899 learned_addr learned my addr 172.21.15.161:0/1431887899 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:41:28.555 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa6a5146700 1 --1- 172.21.15.161:0/1431887899 >> v1:172.21.15.161:6789/0 conn(0x7fa6a01065d0 0x7fa6a0117e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:50618/0 (socket says 172.21.15.161:50618) 2024-02-13T03:41:28.555 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931572186 0 0) 0x7fa6a0119750 con 0x7fa6a01065d0 2024-02-13T03:41:28.555 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa688003030 con 0x7fa6a01065d0 2024-02-13T03:41:28.556 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840551169 0 0) 0x7fa6a011bb50 con 0x7fa6a00f7320 2024-02-13T03:41:28.556 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6a0119750 con 0x7fa6a00f7320 2024-02-13T03:41:28.556 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200963780 0 0) 0x7fa6a011a950 con 0x7fa6a00f6830 2024-02-13T03:41:28.556 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6a011bb50 con 0x7fa6a00f6830 2024-02-13T03:41:28.556 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 951159220 0 0) 0x7fa688003030 con 0x7fa6a01065d0 2024-02-13T03:41:28.556 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6a011a950 con 0x7fa6a01065d0 2024-02-13T03:41:28.556 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.547+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fa698001910 con 0x7fa6a01065d0 2024-02-13T03:41:28.556 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 207700009 0 0) 0x7fa6a0119750 con 0x7fa6a00f7320 2024-02-13T03:41:28.556 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa688003030 con 0x7fa6a00f7320 2024-02-13T03:41:28.556 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4055323353 0 0) 0x7fa6a011bb50 con 0x7fa6a00f6830 2024-02-13T03:41:28.556 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 --> v1:172.21.15.183:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6a0119750 con 0x7fa6a00f6830 2024-02-13T03:41:28.556 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3374220720 0 0) 0x7fa6a011a950 con 0x7fa6a01065d0 2024-02-13T03:41:28.556 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 >> v1:172.21.15.161:6790/0 conn(0x7fa6a00f7320 legacy=0x7fa6a0103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:28.557 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 >> v1:172.21.15.183:6789/0 conn(0x7fa6a00f6830 legacy=0x7fa6a0102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:28.557 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a011cd50 con 0x7fa6a01065d0 2024-02-13T03:41:28.557 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1431887899 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a0119940 con 0x7fa6a01065d0 2024-02-13T03:41:28.557 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1431887899 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a0119d90 con 0x7fa6a01065d0 2024-02-13T03:41:28.557 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fa698001f30 con 0x7fa6a01065d0 2024-02-13T03:41:28.557 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fa6980039c0 con 0x7fa6a01065d0 2024-02-13T03:41:28.558 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1431887899 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68c004f10 con 0x7fa6a01065d0 2024-02-13T03:41:28.558 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fa698003bc0 con 0x7fa6a01065d0 2024-02-13T03:41:28.558 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.551+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(737..737 src has 1..737) v4 ==== 7974+0+0 (unknown 90827903 0 0) 0x7fa698079200 con 0x7fa6a01065d0 2024-02-13T03:41:28.563 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.559+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fa69804c710 con 0x7fa6a01065d0 2024-02-13T03:41:28.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:28 smithi161 bash[27879]: audit 2024-02-13T03:41:26.581723+0000 mgr.y (mgr.24473) 781 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:28 smithi161 bash[27879]: audit 2024-02-13T03:41:27.396974+0000 mon.a (mon.0) 3200 : audit [INF] from='client.? v1:172.21.15.161:0/3859725148' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]': finished 2024-02-13T03:41:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:28 smithi161 bash[27879]: cluster 2024-02-13T03:41:27.397064+0000 mon.a (mon.0) 3201 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-02-13T03:41:28.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:28 smithi161 bash[27879]: audit 2024-02-13T03:41:27.527620+0000 mon.a (mon.0) 3202 : audit [INF] from='client.? v1:172.21.15.161:0/3859725148' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]: dispatch 2024-02-13T03:41:28.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:28 smithi161 bash[19769]: audit 2024-02-13T03:41:26.581723+0000 mgr.y (mgr.24473) 781 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:28.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:28 smithi161 bash[19769]: audit 2024-02-13T03:41:27.396974+0000 mon.a (mon.0) 3200 : audit [INF] from='client.? v1:172.21.15.161:0/3859725148' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]': finished 2024-02-13T03:41:28.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:28 smithi161 bash[19769]: cluster 2024-02-13T03:41:27.397064+0000 mon.a (mon.0) 3201 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-02-13T03:41:28.677 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:28 smithi161 bash[19769]: audit 2024-02-13T03:41:27.527620+0000 mon.a (mon.0) 3202 : audit [INF] from='client.? v1:172.21.15.161:0/3859725148' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]: dispatch 2024-02-13T03:41:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:28 smithi183 bash[20824]: audit 2024-02-13T03:41:26.581723+0000 mgr.y (mgr.24473) 781 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:28 smithi183 bash[20824]: audit 2024-02-13T03:41:27.396974+0000 mon.a (mon.0) 3200 : audit [INF] from='client.? v1:172.21.15.161:0/3859725148' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]': finished 2024-02-13T03:41:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:28 smithi183 bash[20824]: cluster 2024-02-13T03:41:27.397064+0000 mon.a (mon.0) 3201 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-02-13T03:41:28.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:28 smithi183 bash[20824]: audit 2024-02-13T03:41:27.527620+0000 mon.a (mon.0) 3202 : audit [INF] from='client.? v1:172.21.15.161:0/3859725148' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]: dispatch 2024-02-13T03:41:28.749 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:28.743+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1431887899 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fa68c0051c0 con 0x7fa6a01065d0 2024-02-13T03:41:29.436 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:29.431+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.0 v1:172.21.15.161:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 3842dba2-777f-4b83-b6dc-f0e3b09c5464 v738) v1 ==== 223+0+0 (unknown 149868130 0 0) 0x7fa69804ab80 con 0x7fa6a01065d0 2024-02-13T03:41:29.552 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:29.547+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1431887899 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fa68c005900 con 0x7fa6a01065d0 2024-02-13T03:41:29.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:29 smithi161 bash[27879]: cluster 2024-02-13T03:41:27.645495+0000 mgr.y (mgr.24473) 782 : cluster [DBG] pgmap v1275: 188 pgs: 12 unknown, 176 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:41:29.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:29 smithi161 bash[27879]: audit 2024-02-13T03:41:28.413654+0000 mon.a (mon.0) 3203 : audit [INF] from='client.? v1:172.21.15.161:0/3859725148' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]': finished 2024-02-13T03:41:29.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:29 smithi161 bash[27879]: cluster 2024-02-13T03:41:28.413829+0000 mon.a (mon.0) 3204 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-02-13T03:41:29.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:29 smithi161 bash[27879]: audit 2024-02-13T03:41:28.749332+0000 mon.a (mon.0) 3205 : audit [INF] from='client.? v1:172.21.15.161:0/1431887899' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:41:29.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:29 smithi161 bash[19769]: cluster 2024-02-13T03:41:27.645495+0000 mgr.y (mgr.24473) 782 : cluster [DBG] pgmap v1275: 188 pgs: 12 unknown, 176 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:41:29.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:29 smithi161 bash[19769]: audit 2024-02-13T03:41:28.413654+0000 mon.a (mon.0) 3203 : audit [INF] from='client.? v1:172.21.15.161:0/3859725148' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]': finished 2024-02-13T03:41:29.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:29 smithi161 bash[19769]: cluster 2024-02-13T03:41:28.413829+0000 mon.a (mon.0) 3204 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-02-13T03:41:29.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:29 smithi161 bash[19769]: audit 2024-02-13T03:41:28.749332+0000 mon.a (mon.0) 3205 : audit [INF] from='client.? v1:172.21.15.161:0/1431887899' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:41:29.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:41:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:41:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:41:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:29 smithi183 bash[20824]: cluster 2024-02-13T03:41:27.645495+0000 mgr.y (mgr.24473) 782 : cluster [DBG] pgmap v1275: 188 pgs: 12 unknown, 176 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:41:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:29 smithi183 bash[20824]: audit 2024-02-13T03:41:28.413654+0000 mon.a (mon.0) 3203 : audit [INF] from='client.? v1:172.21.15.161:0/3859725148' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "app": "rados"}]': finished 2024-02-13T03:41:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:29 smithi183 bash[20824]: cluster 2024-02-13T03:41:28.413829+0000 mon.a (mon.0) 3204 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-02-13T03:41:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:29 smithi183 bash[20824]: audit 2024-02-13T03:41:28.749332+0000 mon.a (mon.0) 3205 : audit [INF] from='client.? v1:172.21.15.161:0/1431887899' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:41:30.453 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:30.447+0000 7fa69dffb700 1 -- 172.21.15.161:0/1431887899 <== mon.0 v1:172.21.15.161:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 3842dba2-777f-4b83-b6dc-f0e3b09c5464 v739) v1 ==== 223+0+0 (unknown 1101753106 0 0) 0x7fa69804c300 con 0x7fa6a01065d0 2024-02-13T03:41:30.456 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:30.451+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1431887899 >> v1:172.21.15.161:6800/4114337139 conn(0x7fa6880622b0 legacy=0x7fa688064770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:30.457 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:30.451+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1431887899 >> v1:172.21.15.161:6789/0 conn(0x7fa6a01065d0 legacy=0x7fa6a0117e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:41:30.457 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:30.451+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1431887899 shutdown_connections 2024-02-13T03:41:30.457 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:30.451+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1431887899 >> 172.21.15.161:0/1431887899 conn(0x7fa6a00f2530 msgr2=0x7fa6a01099b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:41:30.457 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:30.451+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1431887899 shutdown_connections 2024-02-13T03:41:30.457 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:41:30.451+0000 7fa6a6ba9700 1 -- 172.21.15.161:0/1431887899 wait complete. 2024-02-13T03:41:30.457 INFO:tasks.workunit.client.0.smithi161.stderr:set-quota max_objects = 10 for pool 3842dba2-777f-4b83-b6dc-f0e3b09c5464 2024-02-13T03:41:30.470 INFO:tasks.workunit.client.0.smithi161.stderr:+ sleep 30 2024-02-13T03:41:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:30 smithi183 bash[20824]: audit 2024-02-13T03:41:29.435902+0000 mon.a (mon.0) 3206 : audit [INF] from='client.? v1:172.21.15.161:0/1431887899' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]': finished 2024-02-13T03:41:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:30 smithi183 bash[20824]: cluster 2024-02-13T03:41:29.435994+0000 mon.a (mon.0) 3207 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-02-13T03:41:30.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:30 smithi183 bash[20824]: audit 2024-02-13T03:41:29.552578+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? v1:172.21.15.161:0/1431887899' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:41:30.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:30 smithi161 bash[27879]: audit 2024-02-13T03:41:29.435902+0000 mon.a (mon.0) 3206 : audit [INF] from='client.? v1:172.21.15.161:0/1431887899' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]': finished 2024-02-13T03:41:30.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:30 smithi161 bash[27879]: cluster 2024-02-13T03:41:29.435994+0000 mon.a (mon.0) 3207 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-02-13T03:41:30.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:30 smithi161 bash[27879]: audit 2024-02-13T03:41:29.552578+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? v1:172.21.15.161:0/1431887899' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:41:30.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:30 smithi161 bash[19769]: audit 2024-02-13T03:41:29.435902+0000 mon.a (mon.0) 3206 : audit [INF] from='client.? v1:172.21.15.161:0/1431887899' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]': finished 2024-02-13T03:41:30.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:30 smithi161 bash[19769]: cluster 2024-02-13T03:41:29.435994+0000 mon.a (mon.0) 3207 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-02-13T03:41:30.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:30 smithi161 bash[19769]: audit 2024-02-13T03:41:29.552578+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? v1:172.21.15.161:0/1431887899' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]: dispatch 2024-02-13T03:41:31.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:41:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:41:31] "GET /metrics HTTP/1.1" 200 188243 "" "Prometheus/2.33.4" 2024-02-13T03:41:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:31 smithi183 bash[20824]: cluster 2024-02-13T03:41:29.646268+0000 mgr.y (mgr.24473) 783 : cluster [DBG] pgmap v1278: 188 pgs: 12 unknown, 176 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:41:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:31 smithi183 bash[20824]: audit 2024-02-13T03:41:30.452365+0000 mon.a (mon.0) 3209 : audit [INF] from='client.? v1:172.21.15.161:0/1431887899' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]': finished 2024-02-13T03:41:31.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:31 smithi183 bash[20824]: cluster 2024-02-13T03:41:30.452524+0000 mon.a (mon.0) 3210 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-02-13T03:41:31.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:31 smithi161 bash[27879]: cluster 2024-02-13T03:41:29.646268+0000 mgr.y (mgr.24473) 783 : cluster [DBG] pgmap v1278: 188 pgs: 12 unknown, 176 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:41:31.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:31 smithi161 bash[27879]: audit 2024-02-13T03:41:30.452365+0000 mon.a (mon.0) 3209 : audit [INF] from='client.? v1:172.21.15.161:0/1431887899' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]': finished 2024-02-13T03:41:31.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:31 smithi161 bash[27879]: cluster 2024-02-13T03:41:30.452524+0000 mon.a (mon.0) 3210 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-02-13T03:41:31.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:31 smithi161 bash[19769]: cluster 2024-02-13T03:41:29.646268+0000 mgr.y (mgr.24473) 783 : cluster [DBG] pgmap v1278: 188 pgs: 12 unknown, 176 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:41:31.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:31 smithi161 bash[19769]: audit 2024-02-13T03:41:30.452365+0000 mon.a (mon.0) 3209 : audit [INF] from='client.? v1:172.21.15.161:0/1431887899' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "field": "max_objects", "val": "10"}]': finished 2024-02-13T03:41:31.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:31 smithi161 bash[19769]: cluster 2024-02-13T03:41:30.452524+0000 mon.a (mon.0) 3210 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-02-13T03:41:33.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:33 smithi183 bash[20824]: cluster 2024-02-13T03:41:31.648254+0000 mgr.y (mgr.24473) 784 : cluster [DBG] pgmap v1280: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:33.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:33 smithi161 bash[27879]: cluster 2024-02-13T03:41:31.648254+0000 mgr.y (mgr.24473) 784 : cluster [DBG] pgmap v1280: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:33.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:33 smithi161 bash[19769]: cluster 2024-02-13T03:41:31.648254+0000 mgr.y (mgr.24473) 784 : cluster [DBG] pgmap v1280: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:35 smithi183 bash[20824]: cluster 2024-02-13T03:41:33.649030+0000 mgr.y (mgr.24473) 785 : cluster [DBG] pgmap v1281: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:35.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:35 smithi183 bash[20824]: audit 2024-02-13T03:41:34.800317+0000 mon.a (mon.0) 3211 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:41:35.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:35 smithi161 bash[27879]: cluster 2024-02-13T03:41:33.649030+0000 mgr.y (mgr.24473) 785 : cluster [DBG] pgmap v1281: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:35.926 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:35 smithi161 bash[27879]: audit 2024-02-13T03:41:34.800317+0000 mon.a (mon.0) 3211 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:41:35.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:35 smithi161 bash[19769]: cluster 2024-02-13T03:41:33.649030+0000 mgr.y (mgr.24473) 785 : cluster [DBG] pgmap v1281: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:35.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:35 smithi161 bash[19769]: audit 2024-02-13T03:41:34.800317+0000 mon.a (mon.0) 3211 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:41:36.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:36 smithi183 bash[20824]: cluster 2024-02-13T03:41:35.650042+0000 mgr.y (mgr.24473) 786 : cluster [DBG] pgmap v1282: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 707 B/s rd, 0 op/s 2024-02-13T03:41:36.737 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:41:36 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:41:36.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:36 smithi161 bash[27879]: cluster 2024-02-13T03:41:35.650042+0000 mgr.y (mgr.24473) 786 : cluster [DBG] pgmap v1282: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 707 B/s rd, 0 op/s 2024-02-13T03:41:36.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:36 smithi161 bash[19769]: cluster 2024-02-13T03:41:35.650042+0000 mgr.y (mgr.24473) 786 : cluster [DBG] pgmap v1282: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 707 B/s rd, 0 op/s 2024-02-13T03:41:37.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:37 smithi183 bash[20824]: audit 2024-02-13T03:41:36.592626+0000 mgr.y (mgr.24473) 787 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:37.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:37 smithi161 bash[27879]: audit 2024-02-13T03:41:36.592626+0000 mgr.y (mgr.24473) 787 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:37.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:37 smithi161 bash[19769]: audit 2024-02-13T03:41:36.592626+0000 mgr.y (mgr.24473) 787 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:38.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:38 smithi183 bash[20824]: cluster 2024-02-13T03:41:37.651940+0000 mgr.y (mgr.24473) 788 : cluster [DBG] pgmap v1283: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:38.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:38 smithi161 bash[27879]: cluster 2024-02-13T03:41:37.651940+0000 mgr.y (mgr.24473) 788 : cluster [DBG] pgmap v1283: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:38.926 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:38 smithi161 bash[19769]: cluster 2024-02-13T03:41:37.651940+0000 mgr.y (mgr.24473) 788 : cluster [DBG] pgmap v1283: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:39.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:41:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:41:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:41:41.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:41 smithi161 bash[27879]: cluster 2024-02-13T03:41:39.652818+0000 mgr.y (mgr.24473) 789 : cluster [DBG] pgmap v1284: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:41:41.428 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:41:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:41:41] "GET /metrics HTTP/1.1" 200 190783 "" "Prometheus/2.33.4" 2024-02-13T03:41:41.428 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:41 smithi161 bash[19769]: cluster 2024-02-13T03:41:39.652818+0000 mgr.y (mgr.24473) 789 : cluster [DBG] pgmap v1284: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:41:41.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:41 smithi183 bash[20824]: cluster 2024-02-13T03:41:39.652818+0000 mgr.y (mgr.24473) 789 : cluster [DBG] pgmap v1284: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:41:43.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:43 smithi161 bash[27879]: cluster 2024-02-13T03:41:41.654958+0000 mgr.y (mgr.24473) 790 : cluster [DBG] pgmap v1285: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-13T03:41:43.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:43 smithi161 bash[19769]: cluster 2024-02-13T03:41:41.654958+0000 mgr.y (mgr.24473) 790 : cluster [DBG] pgmap v1285: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-13T03:41:43.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:43 smithi183 bash[20824]: cluster 2024-02-13T03:41:41.654958+0000 mgr.y (mgr.24473) 790 : cluster [DBG] pgmap v1285: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-13T03:41:45.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:45 smithi161 bash[27879]: cluster 2024-02-13T03:41:43.655907+0000 mgr.y (mgr.24473) 791 : cluster [DBG] pgmap v1286: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:45.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:45 smithi161 bash[19769]: cluster 2024-02-13T03:41:43.655907+0000 mgr.y (mgr.24473) 791 : cluster [DBG] pgmap v1286: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:45.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:45 smithi183 bash[20824]: cluster 2024-02-13T03:41:43.655907+0000 mgr.y (mgr.24473) 791 : cluster [DBG] pgmap v1286: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:46.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:41:46 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:41:47.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:47 smithi161 bash[27879]: cluster 2024-02-13T03:41:45.656925+0000 mgr.y (mgr.24473) 792 : cluster [DBG] pgmap v1287: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:47.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:47 smithi161 bash[19769]: cluster 2024-02-13T03:41:45.656925+0000 mgr.y (mgr.24473) 792 : cluster [DBG] pgmap v1287: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:47 smithi183 bash[20824]: cluster 2024-02-13T03:41:45.656925+0000 mgr.y (mgr.24473) 792 : cluster [DBG] pgmap v1287: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:48.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:48 smithi161 bash[27879]: audit 2024-02-13T03:41:46.597327+0000 mgr.y (mgr.24473) 793 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:48.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:48 smithi161 bash[19769]: audit 2024-02-13T03:41:46.597327+0000 mgr.y (mgr.24473) 793 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:48 smithi183 bash[20824]: audit 2024-02-13T03:41:46.597327+0000 mgr.y (mgr.24473) 793 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:49.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:49 smithi161 bash[27879]: cluster 2024-02-13T03:41:47.658934+0000 mgr.y (mgr.24473) 794 : cluster [DBG] pgmap v1288: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:49.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:49 smithi161 bash[19769]: cluster 2024-02-13T03:41:47.658934+0000 mgr.y (mgr.24473) 794 : cluster [DBG] pgmap v1288: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:49.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:49 smithi183 bash[20824]: cluster 2024-02-13T03:41:47.658934+0000 mgr.y (mgr.24473) 794 : cluster [DBG] pgmap v1288: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:49.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:41:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:41:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:41:51.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:51 smithi161 bash[27879]: cluster 2024-02-13T03:41:49.659780+0000 mgr.y (mgr.24473) 795 : cluster [DBG] pgmap v1289: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:41:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:41:51] "GET /metrics HTTP/1.1" 200 190799 "" "Prometheus/2.33.4" 2024-02-13T03:41:51.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:51 smithi161 bash[19769]: cluster 2024-02-13T03:41:49.659780+0000 mgr.y (mgr.24473) 795 : cluster [DBG] pgmap v1289: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:51.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:51 smithi183 bash[20824]: cluster 2024-02-13T03:41:49.659780+0000 mgr.y (mgr.24473) 795 : cluster [DBG] pgmap v1289: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:53.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:53 smithi183 bash[20824]: cluster 2024-02-13T03:41:51.661850+0000 mgr.y (mgr.24473) 796 : cluster [DBG] pgmap v1290: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:53.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:53 smithi161 bash[27879]: cluster 2024-02-13T03:41:51.661850+0000 mgr.y (mgr.24473) 796 : cluster [DBG] pgmap v1290: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:53.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:53 smithi161 bash[19769]: cluster 2024-02-13T03:41:51.661850+0000 mgr.y (mgr.24473) 796 : cluster [DBG] pgmap v1290: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:55.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:55 smithi183 bash[20824]: cluster 2024-02-13T03:41:53.662662+0000 mgr.y (mgr.24473) 797 : cluster [DBG] pgmap v1291: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:55.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:55 smithi161 bash[27879]: cluster 2024-02-13T03:41:53.662662+0000 mgr.y (mgr.24473) 797 : cluster [DBG] pgmap v1291: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:55.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:55 smithi161 bash[19769]: cluster 2024-02-13T03:41:53.662662+0000 mgr.y (mgr.24473) 797 : cluster [DBG] pgmap v1291: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:56.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:41:56 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:41:57.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:57 smithi183 bash[20824]: cluster 2024-02-13T03:41:55.663692+0000 mgr.y (mgr.24473) 798 : cluster [DBG] pgmap v1292: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:57.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:57 smithi161 bash[27879]: cluster 2024-02-13T03:41:55.663692+0000 mgr.y (mgr.24473) 798 : cluster [DBG] pgmap v1292: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:57.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:57 smithi161 bash[19769]: cluster 2024-02-13T03:41:55.663692+0000 mgr.y (mgr.24473) 798 : cluster [DBG] pgmap v1292: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:41:58.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:58 smithi183 bash[20824]: audit 2024-02-13T03:41:56.601338+0000 mgr.y (mgr.24473) 799 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:58.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:58 smithi161 bash[27879]: audit 2024-02-13T03:41:56.601338+0000 mgr.y (mgr.24473) 799 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:58.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:58 smithi161 bash[19769]: audit 2024-02-13T03:41:56.601338+0000 mgr.y (mgr.24473) 799 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:41:59.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:41:59 smithi183 bash[20824]: cluster 2024-02-13T03:41:57.665566+0000 mgr.y (mgr.24473) 800 : cluster [DBG] pgmap v1293: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:59.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:41:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:41:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:41:59.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:41:59 smithi161 bash[27879]: cluster 2024-02-13T03:41:57.665566+0000 mgr.y (mgr.24473) 800 : cluster [DBG] pgmap v1293: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:41:59.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:41:59 smithi161 bash[19769]: cluster 2024-02-13T03:41:57.665566+0000 mgr.y (mgr.24473) 800 : cluster [DBG] pgmap v1293: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:00.472 INFO:tasks.workunit.client.0.smithi161.stderr:+ seq 1 10 2024-02-13T03:42:00.473 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 3842dba2-777f-4b83-b6dc-f0e3b09c5464 put obj1 /etc/passwd 2024-02-13T03:42:00.523 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 3842dba2-777f-4b83-b6dc-f0e3b09c5464 put obj2 /etc/passwd 2024-02-13T03:42:00.563 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 3842dba2-777f-4b83-b6dc-f0e3b09c5464 put obj3 /etc/passwd 2024-02-13T03:42:00.607 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 3842dba2-777f-4b83-b6dc-f0e3b09c5464 put obj4 /etc/passwd 2024-02-13T03:42:00.653 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 3842dba2-777f-4b83-b6dc-f0e3b09c5464 put obj5 /etc/passwd 2024-02-13T03:42:00.700 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 3842dba2-777f-4b83-b6dc-f0e3b09c5464 put obj6 /etc/passwd 2024-02-13T03:42:00.746 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 3842dba2-777f-4b83-b6dc-f0e3b09c5464 put obj7 /etc/passwd 2024-02-13T03:42:00.791 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 3842dba2-777f-4b83-b6dc-f0e3b09c5464 put obj8 /etc/passwd 2024-02-13T03:42:00.831 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 3842dba2-777f-4b83-b6dc-f0e3b09c5464 put obj9 /etc/passwd 2024-02-13T03:42:00.869 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 3842dba2-777f-4b83-b6dc-f0e3b09c5464 put obj10 /etc/passwd 2024-02-13T03:42:00.908 INFO:tasks.workunit.client.0.smithi161.stderr:+ sleep 30 2024-02-13T03:42:01.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:01 smithi161 bash[19769]: cluster 2024-02-13T03:41:59.666410+0000 mgr.y (mgr.24473) 801 : cluster [DBG] pgmap v1294: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:01 smithi161 bash[27879]: cluster 2024-02-13T03:41:59.666410+0000 mgr.y (mgr.24473) 801 : cluster [DBG] pgmap v1294: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:01.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:42:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:42:01] "GET /metrics HTTP/1.1" 200 190799 "" "Prometheus/2.33.4" 2024-02-13T03:42:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:01 smithi183 bash[20824]: cluster 2024-02-13T03:41:59.666410+0000 mgr.y (mgr.24473) 801 : cluster [DBG] pgmap v1294: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:03 smithi183 bash[20824]: cluster 2024-02-13T03:42:01.668428+0000 mgr.y (mgr.24473) 802 : cluster [DBG] pgmap v1295: 188 pgs: 188 active+clean; 224 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 1 op/s 2024-02-13T03:42:03.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:03 smithi161 bash[27879]: cluster 2024-02-13T03:42:01.668428+0000 mgr.y (mgr.24473) 802 : cluster [DBG] pgmap v1295: 188 pgs: 188 active+clean; 224 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 1 op/s 2024-02-13T03:42:03.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:03 smithi161 bash[19769]: cluster 2024-02-13T03:42:01.668428+0000 mgr.y (mgr.24473) 802 : cluster [DBG] pgmap v1295: 188 pgs: 188 active+clean; 224 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 1 op/s 2024-02-13T03:42:05.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:05 smithi183 bash[20824]: cluster 2024-02-13T03:42:03.669253+0000 mgr.y (mgr.24473) 803 : cluster [DBG] pgmap v1296: 188 pgs: 188 active+clean; 224 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 5.0 KiB/s wr, 1 op/s 2024-02-13T03:42:05.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:05 smithi161 bash[27879]: cluster 2024-02-13T03:42:03.669253+0000 mgr.y (mgr.24473) 803 : cluster [DBG] pgmap v1296: 188 pgs: 188 active+clean; 224 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 5.0 KiB/s wr, 1 op/s 2024-02-13T03:42:05.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:05 smithi161 bash[19769]: cluster 2024-02-13T03:42:03.669253+0000 mgr.y (mgr.24473) 803 : cluster [DBG] pgmap v1296: 188 pgs: 188 active+clean; 224 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 5.0 KiB/s wr, 1 op/s 2024-02-13T03:42:06.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:42:06 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:42:07.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:07 smithi183 bash[20824]: cluster 2024-02-13T03:42:05.670266+0000 mgr.y (mgr.24473) 804 : cluster [DBG] pgmap v1297: 188 pgs: 188 active+clean; 224 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 5.0 KiB/s wr, 1 op/s 2024-02-13T03:42:07.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:07 smithi161 bash[27879]: cluster 2024-02-13T03:42:05.670266+0000 mgr.y (mgr.24473) 804 : cluster [DBG] pgmap v1297: 188 pgs: 188 active+clean; 224 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 5.0 KiB/s wr, 1 op/s 2024-02-13T03:42:07.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:07 smithi161 bash[19769]: cluster 2024-02-13T03:42:05.670266+0000 mgr.y (mgr.24473) 804 : cluster [DBG] pgmap v1297: 188 pgs: 188 active+clean; 224 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 5.0 KiB/s wr, 1 op/s 2024-02-13T03:42:08.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:08 smithi161 bash[27879]: audit 2024-02-13T03:42:06.612246+0000 mgr.y (mgr.24473) 805 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:08.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:08 smithi161 bash[19769]: audit 2024-02-13T03:42:06.612246+0000 mgr.y (mgr.24473) 805 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:08 smithi183 bash[20824]: audit 2024-02-13T03:42:06.612246+0000 mgr.y (mgr.24473) 805 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:09.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:09 smithi161 bash[27879]: cluster 2024-02-13T03:42:07.672172+0000 mgr.y (mgr.24473) 806 : cluster [DBG] pgmap v1298: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-13T03:42:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:09 smithi161 bash[27879]: cluster 2024-02-13T03:42:09.149798+0000 mon.a (mon.0) 3212 : cluster [WRN] pool '3842dba2-777f-4b83-b6dc-f0e3b09c5464' is full (reached quota's max_objects: 10) 2024-02-13T03:42:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:09 smithi161 bash[27879]: cluster 2024-02-13T03:42:09.150285+0000 mon.a (mon.0) 3213 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-13T03:42:09.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:09 smithi161 bash[27879]: cluster 2024-02-13T03:42:09.160562+0000 mon.a (mon.0) 3214 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-02-13T03:42:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:09 smithi161 bash[19769]: cluster 2024-02-13T03:42:07.672172+0000 mgr.y (mgr.24473) 806 : cluster [DBG] pgmap v1298: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-13T03:42:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:09 smithi161 bash[19769]: cluster 2024-02-13T03:42:09.149798+0000 mon.a (mon.0) 3212 : cluster [WRN] pool '3842dba2-777f-4b83-b6dc-f0e3b09c5464' is full (reached quota's max_objects: 10) 2024-02-13T03:42:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:09 smithi161 bash[19769]: cluster 2024-02-13T03:42:09.150285+0000 mon.a (mon.0) 3213 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-13T03:42:09.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:09 smithi161 bash[19769]: cluster 2024-02-13T03:42:09.160562+0000 mon.a (mon.0) 3214 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-02-13T03:42:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:09 smithi183 bash[20824]: cluster 2024-02-13T03:42:07.672172+0000 mgr.y (mgr.24473) 806 : cluster [DBG] pgmap v1298: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-13T03:42:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:09 smithi183 bash[20824]: cluster 2024-02-13T03:42:09.149798+0000 mon.a (mon.0) 3212 : cluster [WRN] pool '3842dba2-777f-4b83-b6dc-f0e3b09c5464' is full (reached quota's max_objects: 10) 2024-02-13T03:42:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:09 smithi183 bash[20824]: cluster 2024-02-13T03:42:09.150285+0000 mon.a (mon.0) 3213 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-13T03:42:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:09 smithi183 bash[20824]: cluster 2024-02-13T03:42:09.160562+0000 mon.a (mon.0) 3214 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-02-13T03:42:09.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:42:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:42:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:42:11.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:11 smithi161 bash[27879]: cluster 2024-02-13T03:42:09.673021+0000 mgr.y (mgr.24473) 807 : cluster [DBG] pgmap v1300: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-13T03:42:11.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:42:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:42:11] "GET /metrics HTTP/1.1" 200 190832 "" "Prometheus/2.33.4" 2024-02-13T03:42:11.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:11 smithi161 bash[19769]: cluster 2024-02-13T03:42:09.673021+0000 mgr.y (mgr.24473) 807 : cluster [DBG] pgmap v1300: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-13T03:42:11.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:11 smithi183 bash[20824]: cluster 2024-02-13T03:42:09.673021+0000 mgr.y (mgr.24473) 807 : cluster [DBG] pgmap v1300: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-13T03:42:13.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:13 smithi161 bash[27879]: cluster 2024-02-13T03:42:11.675063+0000 mgr.y (mgr.24473) 808 : cluster [DBG] pgmap v1301: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:42:13.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:13 smithi161 bash[19769]: cluster 2024-02-13T03:42:11.675063+0000 mgr.y (mgr.24473) 808 : cluster [DBG] pgmap v1301: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:42:13.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:13 smithi183 bash[20824]: cluster 2024-02-13T03:42:11.675063+0000 mgr.y (mgr.24473) 808 : cluster [DBG] pgmap v1301: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:42:15.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:15 smithi161 bash[27879]: cluster 2024-02-13T03:42:13.675919+0000 mgr.y (mgr.24473) 809 : cluster [DBG] pgmap v1302: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:42:15.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:15 smithi161 bash[19769]: cluster 2024-02-13T03:42:13.675919+0000 mgr.y (mgr.24473) 809 : cluster [DBG] pgmap v1302: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:42:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:15 smithi183 bash[20824]: cluster 2024-02-13T03:42:13.675919+0000 mgr.y (mgr.24473) 809 : cluster [DBG] pgmap v1302: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:42:16.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:42:16 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:42:17.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:17 smithi161 bash[27879]: cluster 2024-02-13T03:42:15.676893+0000 mgr.y (mgr.24473) 810 : cluster [DBG] pgmap v1303: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:42:17.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:17 smithi161 bash[19769]: cluster 2024-02-13T03:42:15.676893+0000 mgr.y (mgr.24473) 810 : cluster [DBG] pgmap v1303: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:42:17.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:17 smithi183 bash[20824]: cluster 2024-02-13T03:42:15.676893+0000 mgr.y (mgr.24473) 810 : cluster [DBG] pgmap v1303: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-13T03:42:18.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:18 smithi161 bash[27879]: audit 2024-02-13T03:42:16.621831+0000 mgr.y (mgr.24473) 811 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:18.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:18 smithi161 bash[19769]: audit 2024-02-13T03:42:16.621831+0000 mgr.y (mgr.24473) 811 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:18 smithi183 bash[20824]: audit 2024-02-13T03:42:16.621831+0000 mgr.y (mgr.24473) 811 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:19.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:19 smithi161 bash[27879]: cluster 2024-02-13T03:42:17.678849+0000 mgr.y (mgr.24473) 812 : cluster [DBG] pgmap v1304: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:42:19.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:19 smithi161 bash[19769]: cluster 2024-02-13T03:42:17.678849+0000 mgr.y (mgr.24473) 812 : cluster [DBG] pgmap v1304: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:42:19.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:19 smithi183 bash[20824]: cluster 2024-02-13T03:42:17.678849+0000 mgr.y (mgr.24473) 812 : cluster [DBG] pgmap v1304: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T03:42:19.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:42:19 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:42:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:42:21.099 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:20 smithi161 bash[27879]: cluster 2024-02-13T03:42:19.679689+0000 mgr.y (mgr.24473) 813 : cluster [DBG] pgmap v1305: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 973 B/s rd, 0 op/s 2024-02-13T03:42:21.100 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:20 smithi161 bash[27879]: audit 2024-02-13T03:42:19.800606+0000 mon.a (mon.0) 3215 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:42:21.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:20 smithi161 bash[19769]: cluster 2024-02-13T03:42:19.679689+0000 mgr.y (mgr.24473) 813 : cluster [DBG] pgmap v1305: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 973 B/s rd, 0 op/s 2024-02-13T03:42:21.100 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:20 smithi161 bash[19769]: audit 2024-02-13T03:42:19.800606+0000 mon.a (mon.0) 3215 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:42:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:20 smithi183 bash[20824]: cluster 2024-02-13T03:42:19.679689+0000 mgr.y (mgr.24473) 813 : cluster [DBG] pgmap v1305: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 973 B/s rd, 0 op/s 2024-02-13T03:42:21.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:20 smithi183 bash[20824]: audit 2024-02-13T03:42:19.800606+0000 mon.a (mon.0) 3215 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:42:21.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:42:21 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:42:21] "GET /metrics HTTP/1.1" 200 190835 "" "Prometheus/2.33.4" 2024-02-13T03:42:23.175 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:22 smithi161 bash[19769]: cluster 2024-02-13T03:42:21.681744+0000 mgr.y (mgr.24473) 814 : cluster [DBG] pgmap v1306: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:23.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:22 smithi161 bash[19769]: audit 2024-02-13T03:42:22.051038+0000 mon.c (mon.2) 594 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:42:23.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:22 smithi161 bash[19769]: audit 2024-02-13T03:42:22.375316+0000 mon.c (mon.2) 595 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:42:23.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:22 smithi161 bash[19769]: audit 2024-02-13T03:42:22.376671+0000 mon.c (mon.2) 596 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:42:23.176 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:22 smithi161 bash[19769]: audit 2024-02-13T03:42:22.385571+0000 mon.a (mon.0) 3216 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:42:23.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:22 smithi161 bash[27879]: cluster 2024-02-13T03:42:21.681744+0000 mgr.y (mgr.24473) 814 : cluster [DBG] pgmap v1306: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:23.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:22 smithi161 bash[27879]: audit 2024-02-13T03:42:22.051038+0000 mon.c (mon.2) 594 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:42:23.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:22 smithi161 bash[27879]: audit 2024-02-13T03:42:22.375316+0000 mon.c (mon.2) 595 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:42:23.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:22 smithi161 bash[27879]: audit 2024-02-13T03:42:22.376671+0000 mon.c (mon.2) 596 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:42:23.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:22 smithi161 bash[27879]: audit 2024-02-13T03:42:22.385571+0000 mon.a (mon.0) 3216 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:42:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:22 smithi183 bash[20824]: cluster 2024-02-13T03:42:21.681744+0000 mgr.y (mgr.24473) 814 : cluster [DBG] pgmap v1306: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:22 smithi183 bash[20824]: audit 2024-02-13T03:42:22.051038+0000 mon.c (mon.2) 594 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T03:42:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:22 smithi183 bash[20824]: audit 2024-02-13T03:42:22.375316+0000 mon.c (mon.2) 595 : audit [DBG] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T03:42:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:22 smithi183 bash[20824]: audit 2024-02-13T03:42:22.376671+0000 mon.c (mon.2) 596 : audit [INF] from='mgr.24473 v1:172.21.15.161:0/4244559287' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T03:42:23.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:22 smithi183 bash[20824]: audit 2024-02-13T03:42:22.385571+0000 mon.a (mon.0) 3216 : audit [INF] from='mgr.24473 ' entity='mgr.y' 2024-02-13T03:42:25.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:25 smithi161 bash[27879]: cluster 2024-02-13T03:42:23.682523+0000 mgr.y (mgr.24473) 815 : cluster [DBG] pgmap v1307: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:25.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:25 smithi161 bash[19769]: cluster 2024-02-13T03:42:23.682523+0000 mgr.y (mgr.24473) 815 : cluster [DBG] pgmap v1307: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:25 smithi183 bash[20824]: cluster 2024-02-13T03:42:23.682523+0000 mgr.y (mgr.24473) 815 : cluster [DBG] pgmap v1307: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:26.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:42:26 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:42:27.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:27 smithi161 bash[27879]: cluster 2024-02-13T03:42:25.683532+0000 mgr.y (mgr.24473) 816 : cluster [DBG] pgmap v1308: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:27.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:27 smithi161 bash[19769]: cluster 2024-02-13T03:42:25.683532+0000 mgr.y (mgr.24473) 816 : cluster [DBG] pgmap v1308: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:27.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:27 smithi183 bash[20824]: cluster 2024-02-13T03:42:25.683532+0000 mgr.y (mgr.24473) 816 : cluster [DBG] pgmap v1308: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:28.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:28 smithi161 bash[27879]: audit 2024-02-13T03:42:26.633228+0000 mgr.y (mgr.24473) 817 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:28.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:28 smithi161 bash[19769]: audit 2024-02-13T03:42:26.633228+0000 mgr.y (mgr.24473) 817 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:28 smithi183 bash[20824]: audit 2024-02-13T03:42:26.633228+0000 mgr.y (mgr.24473) 817 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:29.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:29 smithi183 bash[20824]: cluster 2024-02-13T03:42:27.685418+0000 mgr.y (mgr.24473) 818 : cluster [DBG] pgmap v1309: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:29.487 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:42:29 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:42:29] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:42:29.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:29 smithi161 bash[19769]: cluster 2024-02-13T03:42:27.685418+0000 mgr.y (mgr.24473) 818 : cluster [DBG] pgmap v1309: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:29.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:29 smithi161 bash[27879]: cluster 2024-02-13T03:42:27.685418+0000 mgr.y (mgr.24473) 818 : cluster [DBG] pgmap v1309: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:30.909 INFO:tasks.workunit.client.0.smithi161.stderr:+ rados -p 140c6d73-9bda-4f05-af7a-8f6a188328fa put threemore /etc/passwd 2024-02-13T03:42:30.958 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool set-quota 140c6d73-9bda-4f05-af7a-8f6a188328fa max_bytes 0 2024-02-13T03:42:31.088 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.081+0000 7fba6675a700 1 -- 172.21.15.161:0/705792249 >> v1:172.21.15.161:6790/0 conn(0x7fba600f8a80 legacy=0x7fba600f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:42:31.088 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.081+0000 7fba6675a700 1 -- 172.21.15.161:0/705792249 shutdown_connections 2024-02-13T03:42:31.089 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.081+0000 7fba6675a700 1 -- 172.21.15.161:0/705792249 >> 172.21.15.161:0/705792249 conn(0x7fba600f23b0 msgr2=0x7fba600f47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:42:31.089 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.081+0000 7fba6675a700 1 -- 172.21.15.161:0/705792249 shutdown_connections 2024-02-13T03:42:31.089 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.081+0000 7fba6675a700 1 -- 172.21.15.161:0/705792249 wait complete. 2024-02-13T03:42:31.089 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.081+0000 7fba6675a700 1 Processor -- start 2024-02-13T03:42:31.089 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba6675a700 1 -- start start 2024-02-13T03:42:31.090 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba6675a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fba601173a0 con 0x7fba60104460 2024-02-13T03:42:31.090 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba6675a700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fba60118580 con 0x7fba60100950 2024-02-13T03:42:31.090 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba6675a700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fba60119780 con 0x7fba600f8a80 2024-02-13T03:42:31.091 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba65f59700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fba60104460 0x7fba60115a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59588/0 (socket says 172.21.15.161:59588) 2024-02-13T03:42:31.091 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba65f59700 1 -- 172.21.15.161:0/279017417 learned_addr learned my addr 172.21.15.161:0/279017417 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:42:31.091 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2460774353 0 0) 0x7fba601173a0 con 0x7fba60104460 2024-02-13T03:42:31.091 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba4c0033b0 con 0x7fba60104460 2024-02-13T03:42:31.092 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765764654 0 0) 0x7fba60119780 con 0x7fba600f8a80 2024-02-13T03:42:31.092 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba601173a0 con 0x7fba600f8a80 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3432351520 0 0) 0x7fba4c0033b0 con 0x7fba60104460 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba60119780 con 0x7fba60104460 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961496083 0 0) 0x7fba60118580 con 0x7fba60100950 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba4c0033b0 con 0x7fba60100950 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fba58002c30 con 0x7fba60104460 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3481534731 0 0) 0x7fba601173a0 con 0x7fba600f8a80 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba60118580 con 0x7fba600f8a80 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fba5c000fa0 con 0x7fba600f8a80 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1046330544 0 0) 0x7fba60119780 con 0x7fba60104460 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 >> v1:172.21.15.161:6790/0 conn(0x7fba600f8a80 legacy=0x7fba600fcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 >> v1:172.21.15.183:6789/0 conn(0x7fba60100950 legacy=0x7fba600fd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6011a980 con 0x7fba60104460 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba6675a700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba60119970 con 0x7fba60104460 2024-02-13T03:42:31.093 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba6675a700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fba60119dc0 con 0x7fba60104460 2024-02-13T03:42:31.094 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fba58002e20 con 0x7fba60104460 2024-02-13T03:42:31.094 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.085+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fba58005d40 con 0x7fba60104460 2024-02-13T03:42:31.094 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.089+0000 7fba6675a700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba44004ee0 con 0x7fba60104460 2024-02-13T03:42:31.094 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.089+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fba580018a0 con 0x7fba60104460 2024-02-13T03:42:31.095 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.089+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(740..740 src has 1..740) v4 ==== 7974+0+0 (unknown 2592853794 0 0) 0x7fba58078e50 con 0x7fba60104460 2024-02-13T03:42:31.096 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.089+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=741}) v3 -- 0x7fba60119780 con 0x7fba60104460 2024-02-13T03:42:31.100 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.093+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fba5804c2a0 con 0x7fba60104460 2024-02-13T03:42:31.278 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:31.273+0000 7fba6675a700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fba44005190 con 0x7fba60104460 2024-02-13T03:42:31.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:31 smithi161 bash[27879]: cluster 2024-02-13T03:42:29.686248+0000 mgr.y (mgr.24473) 819 : cluster [DBG] pgmap v1310: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:31.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:31 smithi161 bash[19769]: cluster 2024-02-13T03:42:29.686248+0000 mgr.y (mgr.24473) 819 : cluster [DBG] pgmap v1310: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:31.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:42:31 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:42:31] "GET /metrics HTTP/1.1" 200 190835 "" "Prometheus/2.33.4" 2024-02-13T03:42:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:31 smithi183 bash[20824]: cluster 2024-02-13T03:42:29.686248+0000 mgr.y (mgr.24473) 819 : cluster [DBG] pgmap v1310: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:32.198 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:32.189+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.0 v1:172.21.15.161:6789/0 10 ==== osd_map(741..741 src has 1..741) v4 ==== 620+0+0 (unknown 1283124852 0 0) 0x7fba5804c4d0 con 0x7fba60104460 2024-02-13T03:42:32.198 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:32.193+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=742}) v3 -- 0x7fba601173a0 con 0x7fba60104460 2024-02-13T03:42:32.204 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:32.197+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.0 v1:172.21.15.161:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v741) v1 ==== 217+0+0 (unknown 1641705575 0 0) 0x7fba5804a6b0 con 0x7fba60104460 2024-02-13T03:42:32.322 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:32.317+0000 7fba6675a700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fba44005800 con 0x7fba60104460 2024-02-13T03:42:32.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:32 smithi183 bash[20824]: audit 2024-02-13T03:42:31.278741+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? v1:172.21.15.161:0/279017417' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-13T03:42:32.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:32 smithi161 bash[27879]: audit 2024-02-13T03:42:31.278741+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? v1:172.21.15.161:0/279017417' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-13T03:42:32.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:32 smithi161 bash[19769]: audit 2024-02-13T03:42:31.278741+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? v1:172.21.15.161:0/279017417' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-13T03:42:33.208 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.201+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.0 v1:172.21.15.161:6789/0 12 ==== osd_map(742..742 src has 1..742) v4 ==== 620+0+0 (unknown 2851743423 0 0) 0x7fba58051e20 con 0x7fba60104460 2024-02-13T03:42:33.208 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.201+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=743}) v3 -- 0x7fba4c06f6d0 con 0x7fba60104460 2024-02-13T03:42:33.214 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.209+0000 7fba567fc700 1 -- 172.21.15.161:0/279017417 <== mon.0 v1:172.21.15.161:6789/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v742) v1 ==== 217+0+0 (unknown 400232644 0 0) 0x7fba5804be30 con 0x7fba60104460 2024-02-13T03:42:33.218 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.213+0000 7fba6675a700 1 -- 172.21.15.161:0/279017417 >> v1:172.21.15.161:6800/4114337139 conn(0x7fba4c0621a0 legacy=0x7fba4c064660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:42:33.218 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.213+0000 7fba6675a700 1 -- 172.21.15.161:0/279017417 >> v1:172.21.15.161:6789/0 conn(0x7fba60104460 legacy=0x7fba60115a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:42:33.218 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.213+0000 7fba6675a700 1 -- 172.21.15.161:0/279017417 shutdown_connections 2024-02-13T03:42:33.218 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.213+0000 7fba6675a700 1 -- 172.21.15.161:0/279017417 >> 172.21.15.161:0/279017417 conn(0x7fba600f23b0 msgr2=0x7fba600f39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:42:33.218 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.213+0000 7fba6675a700 1 -- 172.21.15.161:0/279017417 shutdown_connections 2024-02-13T03:42:33.218 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.213+0000 7fba6675a700 1 -- 172.21.15.161:0/279017417 wait complete. 2024-02-13T03:42:33.219 INFO:tasks.workunit.client.0.smithi161.stderr:set-quota max_bytes = 0 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa 2024-02-13T03:42:33.231 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool set-quota 140c6d73-9bda-4f05-af7a-8f6a188328fa max_objects 0 2024-02-13T03:42:33.352 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.345+0000 7f5ae884e700 1 -- 172.21.15.161:0/3349185986 >> v1:172.21.15.161:6789/0 conn(0x7f5ae40f9560 legacy=0x7f5ae4102a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:42:33.353 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.345+0000 7f5ae884e700 1 -- 172.21.15.161:0/3349185986 shutdown_connections 2024-02-13T03:42:33.353 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.345+0000 7f5ae884e700 1 -- 172.21.15.161:0/3349185986 >> 172.21.15.161:0/3349185986 conn(0x7f5ae40f2510 msgr2=0x7f5ae40f4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:42:33.353 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.345+0000 7f5ae884e700 1 -- 172.21.15.161:0/3349185986 shutdown_connections 2024-02-13T03:42:33.353 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.345+0000 7f5ae884e700 1 -- 172.21.15.161:0/3349185986 wait complete. 2024-02-13T03:42:33.355 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.345+0000 7f5ae884e700 1 Processor -- start 2024-02-13T03:42:33.356 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.345+0000 7f5ae884e700 1 -- start start 2024-02-13T03:42:33.356 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5ae884e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5ae41152d0 con 0x7f5ae40f89e0 2024-02-13T03:42:33.356 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5ae884e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5ae41164b0 con 0x7f5ae40f9560 2024-02-13T03:42:33.356 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5ae884e700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5ae41176b0 con 0x7f5ae4104310 2024-02-13T03:42:33.356 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5ae259c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f5ae40f89e0 0x7f5ae40fd400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59606/0 (socket says 172.21.15.161:59606) 2024-02-13T03:42:33.359 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5ae2d9d700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7f5ae4104310 0x7f5ae4113990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:54546/0 (socket says 172.21.15.161:54546) 2024-02-13T03:42:33.359 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5ae259c700 1 -- 172.21.15.161:0/3765360871 learned_addr learned my addr 172.21.15.161:0/3765360871 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:42:33.359 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255161485 0 0) 0x7f5ae41152d0 con 0x7f5ae40f89e0 2024-02-13T03:42:33.359 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ad00033b0 con 0x7f5ae40f89e0 2024-02-13T03:42:33.359 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969071105 0 0) 0x7f5ae41176b0 con 0x7f5ae4104310 2024-02-13T03:42:33.359 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ae41152d0 con 0x7f5ae4104310 2024-02-13T03:42:33.359 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1746331809 0 0) 0x7f5ad00033b0 con 0x7f5ae40f89e0 2024-02-13T03:42:33.359 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ae41176b0 con 0x7f5ae40f89e0 2024-02-13T03:42:33.359 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409144345 0 0) 0x7f5ae41164b0 con 0x7f5ae40f9560 2024-02-13T03:42:33.359 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ad00033b0 con 0x7f5ae40f9560 2024-02-13T03:42:33.359 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5ad4002950 con 0x7f5ae40f89e0 2024-02-13T03:42:33.360 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3354284600 0 0) 0x7f5ae41152d0 con 0x7f5ae4104310 2024-02-13T03:42:33.360 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ae41164b0 con 0x7f5ae4104310 2024-02-13T03:42:33.360 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5adc001b40 con 0x7f5ae4104310 2024-02-13T03:42:33.360 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1210776580 0 0) 0x7f5ae41176b0 con 0x7f5ae40f89e0 2024-02-13T03:42:33.360 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 >> v1:172.21.15.161:6790/0 conn(0x7f5ae4104310 legacy=0x7f5ae4113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:42:33.360 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 >> v1:172.21.15.183:6789/0 conn(0x7f5ae40f9560 legacy=0x7f5ae40fdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:42:33.360 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae41188b0 con 0x7f5ae40f89e0 2024-02-13T03:42:33.360 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5ae884e700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae41154c0 con 0x7f5ae40f89e0 2024-02-13T03:42:33.360 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5ae884e700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae4115910 con 0x7f5ae40f89e0 2024-02-13T03:42:33.360 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7f5ad4002b40 con 0x7f5ae40f89e0 2024-02-13T03:42:33.360 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7f5ad4005a80 con 0x7f5ae40f89e0 2024-02-13T03:42:33.361 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7f5ad40015c0 con 0x7f5ae40f89e0 2024-02-13T03:42:33.361 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.349+0000 7f5ae884e700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac4004ee0 con 0x7f5ae40f89e0 2024-02-13T03:42:33.361 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.353+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(742..742 src has 1..742) v4 ==== 7974+0+0 (unknown 2541502261 0 0) 0x7f5ad4078c30 con 0x7f5ae40f89e0 2024-02-13T03:42:33.361 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.353+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=743}) v3 -- 0x7f5ae41176b0 con 0x7f5ae40f89e0 2024-02-13T03:42:33.364 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.357+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f5ad404c0e0 con 0x7f5ae40f89e0 2024-02-13T03:42:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:33 smithi183 bash[20824]: cluster 2024-02-13T03:42:31.688277+0000 mgr.y (mgr.24473) 820 : cluster [DBG] pgmap v1311: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:33 smithi183 bash[20824]: audit 2024-02-13T03:42:32.203382+0000 mon.a (mon.0) 3218 : audit [INF] from='client.? v1:172.21.15.161:0/279017417' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]': finished 2024-02-13T03:42:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:33 smithi183 bash[20824]: cluster 2024-02-13T03:42:32.203535+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-02-13T03:42:33.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:33 smithi183 bash[20824]: audit 2024-02-13T03:42:32.322516+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? v1:172.21.15.161:0/279017417' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-13T03:42:33.541 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:33.533+0000 7f5ae884e700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f5ac4005190 con 0x7f5ae40f89e0 2024-02-13T03:42:33.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:33 smithi161 bash[19769]: cluster 2024-02-13T03:42:31.688277+0000 mgr.y (mgr.24473) 820 : cluster [DBG] pgmap v1311: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:33.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:33 smithi161 bash[19769]: audit 2024-02-13T03:42:32.203382+0000 mon.a (mon.0) 3218 : audit [INF] from='client.? v1:172.21.15.161:0/279017417' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]': finished 2024-02-13T03:42:33.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:33 smithi161 bash[19769]: cluster 2024-02-13T03:42:32.203535+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-02-13T03:42:33.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:33 smithi161 bash[19769]: audit 2024-02-13T03:42:32.322516+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? v1:172.21.15.161:0/279017417' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-13T03:42:33.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:33 smithi161 bash[27879]: cluster 2024-02-13T03:42:31.688277+0000 mgr.y (mgr.24473) 820 : cluster [DBG] pgmap v1311: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:33.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:33 smithi161 bash[27879]: audit 2024-02-13T03:42:32.203382+0000 mon.a (mon.0) 3218 : audit [INF] from='client.? v1:172.21.15.161:0/279017417' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]': finished 2024-02-13T03:42:33.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:33 smithi161 bash[27879]: cluster 2024-02-13T03:42:32.203535+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-02-13T03:42:33.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:33 smithi161 bash[27879]: audit 2024-02-13T03:42:32.322516+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? v1:172.21.15.161:0/279017417' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-13T03:42:34.216 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:34.209+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.0 v1:172.21.15.161:6789/0 10 ==== osd_map(743..743 src has 1..743) v4 ==== 620+0+0 (unknown 148421958 0 0) 0x7f5ad40020b0 con 0x7f5ae40f89e0 2024-02-13T03:42:34.217 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:34.209+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=744}) v3 -- 0x7f5ae41152d0 con 0x7f5ae40f89e0 2024-02-13T03:42:34.223 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:34.217+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.0 v1:172.21.15.161:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v743) v1 ==== 221+0+0 (unknown 1558796346 0 0) 0x7f5ad404a4f0 con 0x7f5ae40f89e0 2024-02-13T03:42:34.341 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:34.333+0000 7f5ae884e700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f5ac4005800 con 0x7f5ae40f89e0 2024-02-13T03:42:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:34 smithi183 bash[20824]: audit 2024-02-13T03:42:33.213563+0000 mon.a (mon.0) 3221 : audit [INF] from='client.? v1:172.21.15.161:0/279017417' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]': finished 2024-02-13T03:42:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:34 smithi183 bash[20824]: cluster 2024-02-13T03:42:33.213713+0000 mon.a (mon.0) 3222 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-02-13T03:42:34.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:34 smithi183 bash[20824]: audit 2024-02-13T03:42:33.541466+0000 mon.a (mon.0) 3223 : audit [INF] from='client.? v1:172.21.15.161:0/3765360871' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:42:34.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:34 smithi161 bash[27879]: audit 2024-02-13T03:42:33.213563+0000 mon.a (mon.0) 3221 : audit [INF] from='client.? v1:172.21.15.161:0/279017417' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]': finished 2024-02-13T03:42:34.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:34 smithi161 bash[27879]: cluster 2024-02-13T03:42:33.213713+0000 mon.a (mon.0) 3222 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-02-13T03:42:34.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:34 smithi161 bash[27879]: audit 2024-02-13T03:42:33.541466+0000 mon.a (mon.0) 3223 : audit [INF] from='client.? v1:172.21.15.161:0/3765360871' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:42:34.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:34 smithi161 bash[19769]: audit 2024-02-13T03:42:33.213563+0000 mon.a (mon.0) 3221 : audit [INF] from='client.? v1:172.21.15.161:0/279017417' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_bytes", "val": "0"}]': finished 2024-02-13T03:42:34.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:34 smithi161 bash[19769]: cluster 2024-02-13T03:42:33.213713+0000 mon.a (mon.0) 3222 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-02-13T03:42:34.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:34 smithi161 bash[19769]: audit 2024-02-13T03:42:33.541466+0000 mon.a (mon.0) 3223 : audit [INF] from='client.? v1:172.21.15.161:0/3765360871' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:42:35.219 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:35.213+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.0 v1:172.21.15.161:6789/0 12 ==== osd_map(744..744 src has 1..744) v4 ==== 620+0+0 (unknown 123928589 0 0) 0x7f5ad4051c60 con 0x7f5ae40f89e0 2024-02-13T03:42:35.219 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:35.213+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=745}) v3 -- 0x7f5ad0073950 con 0x7f5ae40f89e0 2024-02-13T03:42:35.225 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:35.221+0000 7f5adbfff700 1 -- 172.21.15.161:0/3765360871 <== mon.0 v1:172.21.15.161:6789/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa v744) v1 ==== 221+0+0 (unknown 3327697474 0 0) 0x7f5ad404bc70 con 0x7f5ae40f89e0 2024-02-13T03:42:35.229 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:35.221+0000 7f5ae884e700 1 -- 172.21.15.161:0/3765360871 >> v1:172.21.15.161:6800/4114337139 conn(0x7f5ad0066680 legacy=0x7f5ad0068b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:42:35.229 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:35.221+0000 7f5ae884e700 1 -- 172.21.15.161:0/3765360871 >> v1:172.21.15.161:6789/0 conn(0x7f5ae40f89e0 legacy=0x7f5ae40fd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:42:35.229 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:35.221+0000 7f5ae884e700 1 -- 172.21.15.161:0/3765360871 shutdown_connections 2024-02-13T03:42:35.229 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:35.221+0000 7f5ae884e700 1 -- 172.21.15.161:0/3765360871 >> 172.21.15.161:0/3765360871 conn(0x7f5ae40f2510 msgr2=0x7f5ae41077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:42:35.231 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:35.225+0000 7f5ae884e700 1 -- 172.21.15.161:0/3765360871 shutdown_connections 2024-02-13T03:42:35.231 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:42:35.225+0000 7f5ae884e700 1 -- 172.21.15.161:0/3765360871 wait complete. 2024-02-13T03:42:35.232 INFO:tasks.workunit.client.0.smithi161.stderr:set-quota max_objects = 0 for pool 140c6d73-9bda-4f05-af7a-8f6a188328fa 2024-02-13T03:42:35.244 INFO:tasks.workunit.client.0.smithi161.stderr:+ sleep 30 2024-02-13T03:42:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:35 smithi183 bash[20824]: cluster 2024-02-13T03:42:33.689100+0000 mgr.y (mgr.24473) 821 : cluster [DBG] pgmap v1314: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:35 smithi183 bash[20824]: audit 2024-02-13T03:42:34.223274+0000 mon.a (mon.0) 3224 : audit [INF] from='client.? v1:172.21.15.161:0/3765360871' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]': finished 2024-02-13T03:42:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:35 smithi183 bash[20824]: cluster 2024-02-13T03:42:34.223541+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-02-13T03:42:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:35 smithi183 bash[20824]: audit 2024-02-13T03:42:34.341343+0000 mon.a (mon.0) 3226 : audit [INF] from='client.? v1:172.21.15.161:0/3765360871' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:42:35.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:35 smithi161 bash[19769]: cluster 2024-02-13T03:42:33.689100+0000 mgr.y (mgr.24473) 821 : cluster [DBG] pgmap v1314: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:35.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:35 smithi161 bash[19769]: audit 2024-02-13T03:42:34.223274+0000 mon.a (mon.0) 3224 : audit [INF] from='client.? v1:172.21.15.161:0/3765360871' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]': finished 2024-02-13T03:42:35.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:35 smithi161 bash[19769]: cluster 2024-02-13T03:42:34.223541+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-02-13T03:42:35.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:35 smithi161 bash[19769]: audit 2024-02-13T03:42:34.341343+0000 mon.a (mon.0) 3226 : audit [INF] from='client.? v1:172.21.15.161:0/3765360871' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:42:35.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:35 smithi161 bash[27879]: cluster 2024-02-13T03:42:33.689100+0000 mgr.y (mgr.24473) 821 : cluster [DBG] pgmap v1314: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:35.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:35 smithi161 bash[27879]: audit 2024-02-13T03:42:34.223274+0000 mon.a (mon.0) 3224 : audit [INF] from='client.? v1:172.21.15.161:0/3765360871' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]': finished 2024-02-13T03:42:35.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:35 smithi161 bash[27879]: cluster 2024-02-13T03:42:34.223541+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-02-13T03:42:35.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:35 smithi161 bash[27879]: audit 2024-02-13T03:42:34.341343+0000 mon.a (mon.0) 3226 : audit [INF] from='client.? v1:172.21.15.161:0/3765360871' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]: dispatch 2024-02-13T03:42:36.646 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:36 smithi183 bash[20824]: audit 2024-02-13T03:42:35.225474+0000 mon.a (mon.0) 3227 : audit [INF] from='client.? v1:172.21.15.161:0/3765360871' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]': finished 2024-02-13T03:42:36.646 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:36 smithi183 bash[20824]: cluster 2024-02-13T03:42:35.225577+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-02-13T03:42:36.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:36 smithi161 bash[27879]: audit 2024-02-13T03:42:35.225474+0000 mon.a (mon.0) 3227 : audit [INF] from='client.? v1:172.21.15.161:0/3765360871' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]': finished 2024-02-13T03:42:36.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:36 smithi161 bash[27879]: cluster 2024-02-13T03:42:35.225577+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-02-13T03:42:36.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:36 smithi161 bash[19769]: audit 2024-02-13T03:42:35.225474+0000 mon.a (mon.0) 3227 : audit [INF] from='client.? v1:172.21.15.161:0/3765360871' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "field": "max_objects", "val": "0"}]': finished 2024-02-13T03:42:36.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:36 smithi161 bash[19769]: cluster 2024-02-13T03:42:35.225577+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-02-13T03:42:36.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:42:36 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:42:37.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:37 smithi161 bash[27879]: cluster 2024-02-13T03:42:35.690155+0000 mgr.y (mgr.24473) 822 : cluster [DBG] pgmap v1317: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:42:37.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:37 smithi161 bash[19769]: cluster 2024-02-13T03:42:35.690155+0000 mgr.y (mgr.24473) 822 : cluster [DBG] pgmap v1317: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:42:37.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:37 smithi183 bash[20824]: cluster 2024-02-13T03:42:35.690155+0000 mgr.y (mgr.24473) 822 : cluster [DBG] pgmap v1317: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-13T03:42:38.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:38 smithi161 bash[27879]: audit 2024-02-13T03:42:36.644582+0000 mgr.y (mgr.24473) 823 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:38.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:38 smithi161 bash[19769]: audit 2024-02-13T03:42:36.644582+0000 mgr.y (mgr.24473) 823 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:38.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:38 smithi183 bash[20824]: audit 2024-02-13T03:42:36.644582+0000 mgr.y (mgr.24473) 823 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:39.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:39 smithi161 bash[19769]: cluster 2024-02-13T03:42:37.691902+0000 mgr.y (mgr.24473) 824 : cluster [DBG] pgmap v1318: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 932 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-02-13T03:42:39.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:39 smithi161 bash[27879]: cluster 2024-02-13T03:42:37.691902+0000 mgr.y (mgr.24473) 824 : cluster [DBG] pgmap v1318: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 932 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-02-13T03:42:39.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:39 smithi183 bash[20824]: cluster 2024-02-13T03:42:37.691902+0000 mgr.y (mgr.24473) 824 : cluster [DBG] pgmap v1318: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 932 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-02-13T03:42:39.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:42:39 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:42:39] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:42:41.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:41 smithi161 bash[27879]: cluster 2024-02-13T03:42:39.692700+0000 mgr.y (mgr.24473) 825 : cluster [DBG] pgmap v1319: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 789 B/s rd, 1.9 KiB/s wr, 0 op/s 2024-02-13T03:42:41.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:41 smithi161 bash[19769]: cluster 2024-02-13T03:42:39.692700+0000 mgr.y (mgr.24473) 825 : cluster [DBG] pgmap v1319: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 789 B/s rd, 1.9 KiB/s wr, 0 op/s 2024-02-13T03:42:41.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:42:41 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:42:41] "GET /metrics HTTP/1.1" 200 190832 "" "Prometheus/2.33.4" 2024-02-13T03:42:41.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:41 smithi183 bash[20824]: cluster 2024-02-13T03:42:39.692700+0000 mgr.y (mgr.24473) 825 : cluster [DBG] pgmap v1319: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 789 B/s rd, 1.9 KiB/s wr, 0 op/s 2024-02-13T03:42:43.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:43 smithi161 bash[27879]: cluster 2024-02-13T03:42:41.694826+0000 mgr.y (mgr.24473) 826 : cluster [DBG] pgmap v1320: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-13T03:42:43.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:43 smithi161 bash[19769]: cluster 2024-02-13T03:42:41.694826+0000 mgr.y (mgr.24473) 826 : cluster [DBG] pgmap v1320: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-13T03:42:43.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:43 smithi183 bash[20824]: cluster 2024-02-13T03:42:41.694826+0000 mgr.y (mgr.24473) 826 : cluster [DBG] pgmap v1320: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-13T03:42:45.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:45 smithi161 bash[27879]: cluster 2024-02-13T03:42:43.695674+0000 mgr.y (mgr.24473) 827 : cluster [DBG] pgmap v1321: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-02-13T03:42:45.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:45 smithi161 bash[19769]: cluster 2024-02-13T03:42:43.695674+0000 mgr.y (mgr.24473) 827 : cluster [DBG] pgmap v1321: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-02-13T03:42:45.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:45 smithi183 bash[20824]: cluster 2024-02-13T03:42:43.695674+0000 mgr.y (mgr.24473) 827 : cluster [DBG] pgmap v1321: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-02-13T03:42:46.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:42:46 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:42:47.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:47 smithi161 bash[27879]: cluster 2024-02-13T03:42:45.696650+0000 mgr.y (mgr.24473) 828 : cluster [DBG] pgmap v1322: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 977 B/s rd, 1.1 KiB/s wr, 1 op/s 2024-02-13T03:42:47.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:47 smithi161 bash[19769]: cluster 2024-02-13T03:42:45.696650+0000 mgr.y (mgr.24473) 828 : cluster [DBG] pgmap v1322: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 977 B/s rd, 1.1 KiB/s wr, 1 op/s 2024-02-13T03:42:47.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:47 smithi183 bash[20824]: cluster 2024-02-13T03:42:45.696650+0000 mgr.y (mgr.24473) 828 : cluster [DBG] pgmap v1322: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 977 B/s rd, 1.1 KiB/s wr, 1 op/s 2024-02-13T03:42:48.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:48 smithi161 bash[27879]: audit 2024-02-13T03:42:46.651986+0000 mgr.y (mgr.24473) 829 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:48.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:48 smithi161 bash[19769]: audit 2024-02-13T03:42:46.651986+0000 mgr.y (mgr.24473) 829 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:48.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:48 smithi183 bash[20824]: audit 2024-02-13T03:42:46.651986+0000 mgr.y (mgr.24473) 829 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:49.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:49 smithi161 bash[27879]: cluster 2024-02-13T03:42:47.698656+0000 mgr.y (mgr.24473) 830 : cluster [DBG] pgmap v1323: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 1 op/s 2024-02-13T03:42:49.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:49 smithi161 bash[19769]: cluster 2024-02-13T03:42:47.698656+0000 mgr.y (mgr.24473) 830 : cluster [DBG] pgmap v1323: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 1 op/s 2024-02-13T03:42:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:49 smithi183 bash[20824]: cluster 2024-02-13T03:42:47.698656+0000 mgr.y (mgr.24473) 830 : cluster [DBG] pgmap v1323: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 1 op/s 2024-02-13T03:42:49.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:42:49 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:42:49] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:42:51.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:51 smithi161 bash[27879]: cluster 2024-02-13T03:42:49.699598+0000 mgr.y (mgr.24473) 831 : cluster [DBG] pgmap v1324: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:51.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:51 smithi161 bash[19769]: cluster 2024-02-13T03:42:49.699598+0000 mgr.y (mgr.24473) 831 : cluster [DBG] pgmap v1324: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:51.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:42:51 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:42:51] "GET /metrics HTTP/1.1" 200 190846 "" "Prometheus/2.33.4" 2024-02-13T03:42:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:51 smithi183 bash[20824]: cluster 2024-02-13T03:42:49.699598+0000 mgr.y (mgr.24473) 831 : cluster [DBG] pgmap v1324: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:53.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:53 smithi161 bash[27879]: cluster 2024-02-13T03:42:51.701791+0000 mgr.y (mgr.24473) 832 : cluster [DBG] pgmap v1325: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:53.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:53 smithi161 bash[19769]: cluster 2024-02-13T03:42:51.701791+0000 mgr.y (mgr.24473) 832 : cluster [DBG] pgmap v1325: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:53.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:53 smithi183 bash[20824]: cluster 2024-02-13T03:42:51.701791+0000 mgr.y (mgr.24473) 832 : cluster [DBG] pgmap v1325: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:55.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:55 smithi161 bash[19769]: cluster 2024-02-13T03:42:53.702712+0000 mgr.y (mgr.24473) 833 : cluster [DBG] pgmap v1326: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:55.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:55 smithi161 bash[27879]: cluster 2024-02-13T03:42:53.702712+0000 mgr.y (mgr.24473) 833 : cluster [DBG] pgmap v1326: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:55.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:55 smithi183 bash[20824]: cluster 2024-02-13T03:42:53.702712+0000 mgr.y (mgr.24473) 833 : cluster [DBG] pgmap v1326: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:56.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:42:56 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:42:57.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:57 smithi161 bash[27879]: cluster 2024-02-13T03:42:55.703740+0000 mgr.y (mgr.24473) 834 : cluster [DBG] pgmap v1327: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:57.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:57 smithi161 bash[19769]: cluster 2024-02-13T03:42:55.703740+0000 mgr.y (mgr.24473) 834 : cluster [DBG] pgmap v1327: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:57.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:57 smithi183 bash[20824]: cluster 2024-02-13T03:42:55.703740+0000 mgr.y (mgr.24473) 834 : cluster [DBG] pgmap v1327: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:42:58.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:58 smithi161 bash[27879]: audit 2024-02-13T03:42:56.662837+0000 mgr.y (mgr.24473) 835 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:58.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:58 smithi161 bash[19769]: audit 2024-02-13T03:42:56.662837+0000 mgr.y (mgr.24473) 835 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:58 smithi183 bash[20824]: audit 2024-02-13T03:42:56.662837+0000 mgr.y (mgr.24473) 835 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:42:59.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:42:59 smithi161 bash[27879]: cluster 2024-02-13T03:42:57.705624+0000 mgr.y (mgr.24473) 836 : cluster [DBG] pgmap v1328: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:59.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:42:59 smithi161 bash[19769]: cluster 2024-02-13T03:42:57.705624+0000 mgr.y (mgr.24473) 836 : cluster [DBG] pgmap v1328: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:42:59 smithi183 bash[20824]: cluster 2024-02-13T03:42:57.705624+0000 mgr.y (mgr.24473) 836 : cluster [DBG] pgmap v1328: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:42:59.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:42:59 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:42:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:43:01.425 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:01 smithi161 bash[19769]: cluster 2024-02-13T03:42:59.706451+0000 mgr.y (mgr.24473) 837 : cluster [DBG] pgmap v1329: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:43:01.426 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:43:01 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:43:01] "GET /metrics HTTP/1.1" 200 190846 "" "Prometheus/2.33.4" 2024-02-13T03:43:01.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:01 smithi161 bash[27879]: cluster 2024-02-13T03:42:59.706451+0000 mgr.y (mgr.24473) 837 : cluster [DBG] pgmap v1329: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:43:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:01 smithi183 bash[20824]: cluster 2024-02-13T03:42:59.706451+0000 mgr.y (mgr.24473) 837 : cluster [DBG] pgmap v1329: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:43:03.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:03 smithi161 bash[27879]: cluster 2024-02-13T03:43:01.708485+0000 mgr.y (mgr.24473) 838 : cluster [DBG] pgmap v1330: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:43:03.676 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:03 smithi161 bash[19769]: cluster 2024-02-13T03:43:01.708485+0000 mgr.y (mgr.24473) 838 : cluster [DBG] pgmap v1330: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:43:03.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:03 smithi183 bash[20824]: cluster 2024-02-13T03:43:01.708485+0000 mgr.y (mgr.24473) 838 : cluster [DBG] pgmap v1330: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T03:43:05.245 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool delete 140c6d73-9bda-4f05-af7a-8f6a188328fa 140c6d73-9bda-4f05-af7a-8f6a188328fa --yes-i-really-really-mean-it 2024-02-13T03:43:05.376 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.373+0000 7fce7e6c3700 1 -- 172.21.15.161:0/1170034105 >> v1:172.21.15.161:6790/0 conn(0x7fce78100890 legacy=0x7fce78102c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:43:05.376 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.373+0000 7fce7e6c3700 1 -- 172.21.15.161:0/1170034105 shutdown_connections 2024-02-13T03:43:05.376 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.373+0000 7fce7e6c3700 1 -- 172.21.15.161:0/1170034105 >> 172.21.15.161:0/1170034105 conn(0x7fce7806c1a0 msgr2=0x7fce7806c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:43:05.376 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.373+0000 7fce7e6c3700 1 -- 172.21.15.161:0/1170034105 shutdown_connections 2024-02-13T03:43:05.377 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.373+0000 7fce7e6c3700 1 -- 172.21.15.161:0/1170034105 wait complete. 2024-02-13T03:43:05.377 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.373+0000 7fce7e6c3700 1 Processor -- start 2024-02-13T03:43:05.377 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.373+0000 7fce7e6c3700 1 -- start start 2024-02-13T03:43:05.377 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce7e6c3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fce78117310 con 0x7fce78100890 2024-02-13T03:43:05.377 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce7e6c3700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fce781184d0 con 0x7fce781043a0 2024-02-13T03:43:05.377 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce7e6c3700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fce781196d0 con 0x7fce780f6380 2024-02-13T03:43:05.379 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce777fe700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fce78100890 0x7fce780fa1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59084/0 (socket says 172.21.15.161:59084) 2024-02-13T03:43:05.379 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce777fe700 1 -- 172.21.15.161:0/3336378937 learned_addr learned my addr 172.21.15.161:0/3336378937 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:43:05.379 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce77fff700 1 --1- 172.21.15.161:0/3336378937 >> v1:172.21.15.161:6790/0 conn(0x7fce780f6380 0x7fce780f9ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:59652/0 (socket says 172.21.15.161:59652) 2024-02-13T03:43:05.379 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247373148 0 0) 0x7fce78117310 con 0x7fce78100890 2024-02-13T03:43:05.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce640033b0 con 0x7fce78100890 2024-02-13T03:43:05.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250182763 0 0) 0x7fce781196d0 con 0x7fce780f6380 2024-02-13T03:43:05.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce78117310 con 0x7fce780f6380 2024-02-13T03:43:05.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1397822600 0 0) 0x7fce640033b0 con 0x7fce78100890 2024-02-13T03:43:05.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce781196d0 con 0x7fce78100890 2024-02-13T03:43:05.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fce68001330 con 0x7fce78100890 2024-02-13T03:43:05.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026769179 0 0) 0x7fce781184d0 con 0x7fce781043a0 2024-02-13T03:43:05.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce640033b0 con 0x7fce781043a0 2024-02-13T03:43:05.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.0 v1:172.21.15.161:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2121340053 0 0) 0x7fce781196d0 con 0x7fce78100890 2024-02-13T03:43:05.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 >> v1:172.21.15.161:6790/0 conn(0x7fce780f6380 legacy=0x7fce780f9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:43:05.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 >> v1:172.21.15.183:6789/0 conn(0x7fce781043a0 legacy=0x7fce78115a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:43:05.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7811a8d0 con 0x7fce78100890 2024-02-13T03:43:05.380 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce7e6c3700 1 -- 172.21.15.161:0/3336378937 --> v1:172.21.15.161:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce781198c0 con 0x7fce78100890 2024-02-13T03:43:05.381 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce7e6c3700 1 -- 172.21.15.161:0/3336378937 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fce78119d10 con 0x7fce78100890 2024-02-13T03:43:05.381 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.0 v1:172.21.15.161:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fce680015a0 con 0x7fce78100890 2024-02-13T03:43:05.381 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.377+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.0 v1:172.21.15.161:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fce68003950 con 0x7fce78100890 2024-02-13T03:43:05.381 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.381+0000 7fce7e6c3700 1 -- 172.21.15.161:0/3336378937 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce58004f10 con 0x7fce78100890 2024-02-13T03:43:05.381 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.381+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.0 v1:172.21.15.161:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fce68003b90 con 0x7fce78100890 2024-02-13T03:43:05.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.381+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.0 v1:172.21.15.161:6789/0 8 ==== osd_map(744..744 src has 1..744) v4 ==== 7974+0+0 (unknown 3065769808 0 0) 0x7fce68079040 con 0x7fce78100890 2024-02-13T03:43:05.382 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.381+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=745}) v3 -- 0x7fce781196d0 con 0x7fce78100890 2024-02-13T03:43:05.387 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.385+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.0 v1:172.21.15.161:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fce6804c5b0 con 0x7fce78100890 2024-02-13T03:43:05.567 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:05.565+0000 7fce7e6c3700 1 -- 172.21.15.161:0/3336378937 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pool2": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce580051c0 con 0x7fce78100890 2024-02-13T03:43:05.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:05 smithi161 bash[27879]: cluster 2024-02-13T03:43:03.709261+0000 mgr.y (mgr.24473) 839 : cluster [DBG] pgmap v1331: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:43:05.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:05 smithi161 bash[19769]: cluster 2024-02-13T03:43:03.709261+0000 mgr.y (mgr.24473) 839 : cluster [DBG] pgmap v1331: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:43:05.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:05 smithi183 bash[20824]: cluster 2024-02-13T03:43:03.709261+0000 mgr.y (mgr.24473) 839 : cluster [DBG] pgmap v1331: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:43:06.355 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.353+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.0 v1:172.21.15.161:6789/0 10 ==== osd_map(745..745 src has 1..745) v4 ==== 288+0+0 (unknown 1248633341 0 0) 0x7fce6804c7e0 con 0x7fce78100890 2024-02-13T03:43:06.355 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.353+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 --> v1:172.21.15.161:6789/0 -- mon_subscribe({osdmap=746}) v3 -- 0x7fce78117310 con 0x7fce78100890 2024-02-13T03:43:06.359 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.357+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.0 v1:172.21.15.161:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pool2": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "yes_i_really_really_mean_it": true}]=0 pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' removed v745) v1 ==== 248+0+0 (unknown 724214580 0 0) 0x7fce6804aa20 con 0x7fce78100890 2024-02-13T03:43:06.479 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.477+0000 7fce7e6c3700 1 -- 172.21.15.161:0/3336378937 --> v1:172.21.15.161:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pool2": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce58005970 con 0x7fce78100890 2024-02-13T03:43:06.480 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.477+0000 7fce757fa700 1 -- 172.21.15.161:0/3336378937 <== mon.0 v1:172.21.15.161:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pool2": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "yes_i_really_really_mean_it": true}]=0 pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' does not exist v745) v1 ==== 255+0+0 (unknown 686862435 0 0) 0x7fce6804c1a0 con 0x7fce78100890 2024-02-13T03:43:06.483 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.481+0000 7fce7e6c3700 1 -- 172.21.15.161:0/3336378937 >> v1:172.21.15.161:6800/4114337139 conn(0x7fce64061f40 legacy=0x7fce64064400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:43:06.483 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.481+0000 7fce7e6c3700 1 -- 172.21.15.161:0/3336378937 >> v1:172.21.15.161:6789/0 conn(0x7fce78100890 legacy=0x7fce780fa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:43:06.483 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.481+0000 7fce7e6c3700 1 -- 172.21.15.161:0/3336378937 shutdown_connections 2024-02-13T03:43:06.483 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.481+0000 7fce7e6c3700 1 -- 172.21.15.161:0/3336378937 >> 172.21.15.161:0/3336378937 conn(0x7fce7806c1a0 msgr2=0x7fce7806cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:43:06.483 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.481+0000 7fce7e6c3700 1 -- 172.21.15.161:0/3336378937 shutdown_connections 2024-02-13T03:43:06.483 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.481+0000 7fce7e6c3700 1 -- 172.21.15.161:0/3336378937 wait complete. 2024-02-13T03:43:06.484 INFO:tasks.workunit.client.0.smithi161.stderr:pool '140c6d73-9bda-4f05-af7a-8f6a188328fa' does not exist 2024-02-13T03:43:06.497 INFO:tasks.workunit.client.0.smithi161.stderr:+ ceph osd pool delete 3842dba2-777f-4b83-b6dc-f0e3b09c5464 3842dba2-777f-4b83-b6dc-f0e3b09c5464 --yes-i-really-really-mean-it 2024-02-13T03:43:06.622 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd4365ca700 1 -- 172.21.15.161:0/3872658938 >> v1:172.21.15.161:6789/0 conn(0x7fd430102a40 legacy=0x7fd430104e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:43:06.623 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd4365ca700 1 -- 172.21.15.161:0/3872658938 shutdown_connections 2024-02-13T03:43:06.623 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd4365ca700 1 -- 172.21.15.161:0/3872658938 >> 172.21.15.161:0/3872658938 conn(0x7fd4300f5860 msgr2=0x7fd4300f7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:43:06.623 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd4365ca700 1 -- 172.21.15.161:0/3872658938 shutdown_connections 2024-02-13T03:43:06.623 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd4365ca700 1 -- 172.21.15.161:0/3872658938 wait complete. 2024-02-13T03:43:06.624 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd4365ca700 1 Processor -- start 2024-02-13T03:43:06.624 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd4365ca700 1 -- start start 2024-02-13T03:43:06.624 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd4365ca700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd430119520 con 0x7fd4300ffbd0 2024-02-13T03:43:06.624 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd4365ca700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd43011a720 con 0x7fd430102a40 2024-02-13T03:43:06.625 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd4365ca700 1 -- --> v1:172.21.15.161:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd43011b920 con 0x7fd430106550 2024-02-13T03:43:06.625 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd42ffff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd4300ffbd0 0x7fd4300fbc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:59104/0 (socket says 172.21.15.161:59104) 2024-02-13T03:43:06.625 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd434b67700 1 --1- >> v1:172.21.15.161:6790/0 conn(0x7fd430106550 0x7fd430117be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.161:6790/0 says I am v1:172.21.15.161:59680/0 (socket says 172.21.15.161:59680) 2024-02-13T03:43:06.625 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd42ffff700 1 -- 172.21.15.161:0/2191487095 learned_addr learned my addr 172.21.15.161:0/2191487095 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-02-13T03:43:06.627 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.2 v1:172.21.15.161:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414467851 0 0) 0x7fd43011b920 con 0x7fd430106550 2024-02-13T03:43:06.627 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 --> v1:172.21.15.161:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd41c003110 con 0x7fd430106550 2024-02-13T03:43:06.627 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.0 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086772168 0 0) 0x7fd430119520 con 0x7fd4300ffbd0 2024-02-13T03:43:06.627 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd43011b920 con 0x7fd4300ffbd0 2024-02-13T03:43:06.627 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759522821 0 0) 0x7fd43011a720 con 0x7fd430102a40 2024-02-13T03:43:06.627 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd430119520 con 0x7fd430102a40 2024-02-13T03:43:06.627 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.621+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.2 v1:172.21.15.161:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3403723455 0 0) 0x7fd41c003110 con 0x7fd430106550 2024-02-13T03:43:06.627 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 --> v1:172.21.15.161:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd43011a720 con 0x7fd430106550 2024-02-13T03:43:06.627 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.2 v1:172.21.15.161:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fd428001c20 con 0x7fd430106550 2024-02-13T03:43:06.628 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.0 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 547281915 0 0) 0x7fd43011b920 con 0x7fd4300ffbd0 2024-02-13T03:43:06.628 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 --> v1:172.21.15.161:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd41c003110 con 0x7fd4300ffbd0 2024-02-13T03:43:06.628 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.0 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fd420001900 con 0x7fd4300ffbd0 2024-02-13T03:43:06.628 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.2 v1:172.21.15.161:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3442687229 0 0) 0x7fd43011a720 con 0x7fd430106550 2024-02-13T03:43:06.628 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 >> v1:172.21.15.183:6789/0 conn(0x7fd430102a40 legacy=0x7fd4300fc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:43:06.628 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 >> v1:172.21.15.161:6789/0 conn(0x7fd4300ffbd0 legacy=0x7fd4300fbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:43:06.628 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 --> v1:172.21.15.161:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43011cb20 con 0x7fd430106550 2024-02-13T03:43:06.628 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd4365ca700 1 -- 172.21.15.161:0/2191487095 --> v1:172.21.15.161:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd430119710 con 0x7fd430106550 2024-02-13T03:43:06.628 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd4365ca700 1 -- 172.21.15.161:0/2191487095 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd430119b60 con 0x7fd430106550 2024-02-13T03:43:06.628 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.2 v1:172.21.15.161:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 34136010 0 0) 0x7fd428002650 con 0x7fd430106550 2024-02-13T03:43:06.628 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.2 v1:172.21.15.161:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3458469638 0 0) 0x7fd428005cb0 con 0x7fd430106550 2024-02-13T03:43:06.629 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd426ffd700 1 -- 172.21.15.161:0/2191487095 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd414004ee0 con 0x7fd430106550 2024-02-13T03:43:06.629 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.625+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.2 v1:172.21.15.161:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 2575163184 0 0) 0x7fd428018bc0 con 0x7fd430106550 2024-02-13T03:43:06.629 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.629+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.2 v1:172.21.15.161:6790/0 8 ==== osd_map(745..745 src has 1..745) v4 ==== 7586+0+0 (unknown 901354812 0 0) 0x7fd428078e10 con 0x7fd430106550 2024-02-13T03:43:06.629 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.629+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=746}) v3 -- 0x7fd43011a720 con 0x7fd430106550 2024-02-13T03:43:06.638 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.637+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.2 v1:172.21.15.161:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fd42804c3e0 con 0x7fd430106550 2024-02-13T03:43:06.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:06 smithi161 bash[27879]: audit 2024-02-13T03:43:05.567119+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? v1:172.21.15.161:0/3336378937' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pool2": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:06.675 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:06 smithi183 bash[20824]: audit 2024-02-13T03:43:05.567119+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? v1:172.21.15.161:0/3336378937' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pool2": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:06.675 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:06 smithi161 bash[19769]: audit 2024-02-13T03:43:05.567119+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? v1:172.21.15.161:0/3336378937' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pool2": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:06.816 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:06.813+0000 7fd426ffd700 1 -- 172.21.15.161:0/2191487095 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd414005710 con 0x7fd430106550 2024-02-13T03:43:06.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:43:06 smithi183 bash[43494]: debug there is no tcmu-runner data available 2024-02-13T03:43:07.367 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:07.365+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.2 v1:172.21.15.161:6790/0 10 ==== osd_map(746..746 src has 1..746) v4 ==== 288+0+0 (unknown 587869505 0 0) 0x7fd42804c610 con 0x7fd430106550 2024-02-13T03:43:07.368 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:07.365+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 --> v1:172.21.15.161:6790/0 -- mon_subscribe({osdmap=747}) v3 -- 0x7fd43011b920 con 0x7fd430106550 2024-02-13T03:43:07.372 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:07.369+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.2 v1:172.21.15.161:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]=0 pool '3842dba2-777f-4b83-b6dc-f0e3b09c5464' removed v746) v1 ==== 248+0+0 (unknown 2206362152 0 0) 0x7fd42804a7f0 con 0x7fd430106550 2024-02-13T03:43:07.491 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:07.489+0000 7fd426ffd700 1 -- 172.21.15.161:0/2191487095 --> v1:172.21.15.161:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd414001190 con 0x7fd430106550 2024-02-13T03:43:07.492 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:07.489+0000 7fd42d7fa700 1 -- 172.21.15.161:0/2191487095 <== mon.2 v1:172.21.15.161:6790/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]=0 pool '3842dba2-777f-4b83-b6dc-f0e3b09c5464' does not exist v746) v1 ==== 255+0+0 (unknown 828455817 0 0) 0x7fd42804bf70 con 0x7fd430106550 2024-02-13T03:43:07.495 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:07.493+0000 7fd426ffd700 1 -- 172.21.15.161:0/2191487095 >> v1:172.21.15.161:6800/4114337139 conn(0x7fd41c0623d0 legacy=0x7fd41c064890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:43:07.495 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:07.493+0000 7fd426ffd700 1 -- 172.21.15.161:0/2191487095 >> v1:172.21.15.161:6790/0 conn(0x7fd430106550 legacy=0x7fd430117be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-13T03:43:07.495 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:07.493+0000 7fd426ffd700 1 -- 172.21.15.161:0/2191487095 shutdown_connections 2024-02-13T03:43:07.495 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:07.493+0000 7fd426ffd700 1 -- 172.21.15.161:0/2191487095 >> 172.21.15.161:0/2191487095 conn(0x7fd4300f5860 msgr2=0x7fd430109a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-13T03:43:07.495 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:07.493+0000 7fd426ffd700 1 -- 172.21.15.161:0/2191487095 shutdown_connections 2024-02-13T03:43:07.496 INFO:tasks.workunit.client.0.smithi161.stderr:2024-02-13T03:43:07.493+0000 7fd426ffd700 1 -- 172.21.15.161:0/2191487095 wait complete. 2024-02-13T03:43:07.496 INFO:tasks.workunit.client.0.smithi161.stderr:pool '3842dba2-777f-4b83-b6dc-f0e3b09c5464' does not exist 2024-02-13T03:43:07.511 INFO:tasks.workunit.client.0.smithi161.stdout:OK 2024-02-13T03:43:07.512 INFO:tasks.workunit.client.0.smithi161.stderr:+ echo OK 2024-02-13T03:43:07.512 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-13T03:43:07.513 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-13T03:43:07.568 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-02-13T03:43:07.568 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-13T03:43:07.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:07 smithi161 bash[27879]: cluster 2024-02-13T03:43:05.710268+0000 mgr.y (mgr.24473) 840 : cluster [DBG] pgmap v1332: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:43:07.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:07 smithi161 bash[27879]: audit 2024-02-13T03:43:06.359217+0000 mon.a (mon.0) 3230 : audit [INF] from='client.? v1:172.21.15.161:0/3336378937' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pool2": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "yes_i_really_really_mean_it": true}]': finished 2024-02-13T03:43:07.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:07 smithi161 bash[27879]: cluster 2024-02-13T03:43:06.359308+0000 mon.a (mon.0) 3231 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-02-13T03:43:07.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:07 smithi161 bash[27879]: audit 2024-02-13T03:43:06.480209+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? v1:172.21.15.161:0/3336378937' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pool2": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:07.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:07 smithi161 bash[27879]: audit 2024-02-13T03:43:06.816951+0000 mon.c (mon.2) 597 : audit [INF] from='client.? v1:172.21.15.161:0/2191487095' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:07.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:07 smithi161 bash[27879]: audit 2024-02-13T03:43:06.817790+0000 mon.a (mon.0) 3233 : audit [INF] from='client.48701 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:07.621 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:07 smithi161 bash[19769]: cluster 2024-02-13T03:43:05.710268+0000 mgr.y (mgr.24473) 840 : cluster [DBG] pgmap v1332: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:43:07.621 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:07 smithi161 bash[19769]: audit 2024-02-13T03:43:06.359217+0000 mon.a (mon.0) 3230 : audit [INF] from='client.? v1:172.21.15.161:0/3336378937' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pool2": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "yes_i_really_really_mean_it": true}]': finished 2024-02-13T03:43:07.621 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:07 smithi161 bash[19769]: cluster 2024-02-13T03:43:06.359308+0000 mon.a (mon.0) 3231 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-02-13T03:43:07.622 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:07 smithi161 bash[19769]: audit 2024-02-13T03:43:06.480209+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? v1:172.21.15.161:0/3336378937' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pool2": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:07.622 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:07 smithi161 bash[19769]: audit 2024-02-13T03:43:06.816951+0000 mon.c (mon.2) 597 : audit [INF] from='client.? v1:172.21.15.161:0/2191487095' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:07.622 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:07 smithi161 bash[19769]: audit 2024-02-13T03:43:06.817790+0000 mon.a (mon.0) 3233 : audit [INF] from='client.48701 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:07 smithi183 bash[20824]: cluster 2024-02-13T03:43:05.710268+0000 mgr.y (mgr.24473) 840 : cluster [DBG] pgmap v1332: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-13T03:43:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:07 smithi183 bash[20824]: audit 2024-02-13T03:43:06.359217+0000 mon.a (mon.0) 3230 : audit [INF] from='client.? v1:172.21.15.161:0/3336378937' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pool2": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "yes_i_really_really_mean_it": true}]': finished 2024-02-13T03:43:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:07 smithi183 bash[20824]: cluster 2024-02-13T03:43:06.359308+0000 mon.a (mon.0) 3231 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-02-13T03:43:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:07 smithi183 bash[20824]: audit 2024-02-13T03:43:06.480209+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? v1:172.21.15.161:0/3336378937' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "pool2": "140c6d73-9bda-4f05-af7a-8f6a188328fa", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:07 smithi183 bash[20824]: audit 2024-02-13T03:43:06.816951+0000 mon.c (mon.2) 597 : audit [INF] from='client.? v1:172.21.15.161:0/2191487095' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:07.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:07 smithi183 bash[20824]: audit 2024-02-13T03:43:06.817790+0000 mon.a (mon.0) 3233 : audit [INF] from='client.48701 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:07.886 DEBUG:teuthology.parallel:result is None 2024-02-13T03:43:07.886 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-13T03:43:07.946 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-13T03:43:07.946 DEBUG:teuthology.orchestra.run.smithi161:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-02-13T03:43:08.011 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-02-13T03:43:08.011 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-13T03:43:08.021 INFO:tasks.cephadm:Teardown begin 2024-02-13T03:43:08.022 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T03:43:08.067 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T03:43:08.083 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-13T03:43:08.083 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-13T03:43:08.115 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-13T03:43:08.129 INFO:tasks.cephadm:Stopping all daemons... 2024-02-13T03:43:08.129 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-13T03:43:08.130 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mon.a 2024-02-13T03:43:08.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[27879]: audit 2024-02-13T03:43:06.674083+0000 mgr.y (mgr.24473) 841 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:43:08.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[27879]: cluster 2024-02-13T03:43:07.355439+0000 mon.a (mon.0) 3234 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-13T03:43:08.425 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[27879]: audit 2024-02-13T03:43:07.370588+0000 mon.a (mon.0) 3235 : audit [INF] from='client.48701 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]': finished 2024-02-13T03:43:08.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[27879]: cluster 2024-02-13T03:43:07.370716+0000 mon.a (mon.0) 3236 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-02-13T03:43:08.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[27879]: audit 2024-02-13T03:43:07.491412+0000 mon.c (mon.2) 598 : audit [INF] from='client.? v1:172.21.15.161:0/2191487095' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:08.426 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[27879]: audit 2024-02-13T03:43:07.492124+0000 mon.a (mon.0) 3237 : audit [INF] from='client.48701 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:08.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:08 smithi161 systemd[1]: Stopping Ceph mon.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:43:08.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[129053]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-mon.a 2024-02-13T03:43:08.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[19769]: audit 2024-02-13T03:43:06.674083+0000 mgr.y (mgr.24473) 841 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:43:08.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[19769]: cluster 2024-02-13T03:43:07.355439+0000 mon.a (mon.0) 3234 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-13T03:43:08.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[19769]: audit 2024-02-13T03:43:07.370588+0000 mon.a (mon.0) 3235 : audit [INF] from='client.48701 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]': finished 2024-02-13T03:43:08.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[19769]: cluster 2024-02-13T03:43:07.370716+0000 mon.a (mon.0) 3236 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-02-13T03:43:08.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[19769]: audit 2024-02-13T03:43:07.491412+0000 mon.c (mon.2) 598 : audit [INF] from='client.? v1:172.21.15.161:0/2191487095' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:08.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[19769]: audit 2024-02-13T03:43:07.492124+0000 mon.a (mon.0) 3237 : audit [INF] from='client.48701 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:08.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[19769]: debug 2024-02-13T03:43:08.397+0000 7f588c765700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-13T03:43:08.426 INFO:journalctl@ceph.mon.a.smithi161.stdout:Feb 13 03:43:08 smithi161 bash[19769]: debug 2024-02-13T03:43:08.397+0000 7f588c765700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-13T03:43:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:08 smithi183 bash[20824]: audit 2024-02-13T03:43:06.674083+0000 mgr.y (mgr.24473) 841 : audit [DBG] from='client.24442 v1:172.21.15.183:0/549609834' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T03:43:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:08 smithi183 bash[20824]: cluster 2024-02-13T03:43:07.355439+0000 mon.a (mon.0) 3234 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-13T03:43:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:08 smithi183 bash[20824]: audit 2024-02-13T03:43:07.370588+0000 mon.a (mon.0) 3235 : audit [INF] from='client.48701 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]': finished 2024-02-13T03:43:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:08 smithi183 bash[20824]: cluster 2024-02-13T03:43:07.370716+0000 mon.a (mon.0) 3236 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-02-13T03:43:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:08 smithi183 bash[20824]: audit 2024-02-13T03:43:07.491412+0000 mon.c (mon.2) 598 : audit [INF] from='client.? v1:172.21.15.161:0/2191487095' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:08.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:08 smithi183 bash[20824]: audit 2024-02-13T03:43:07.492124+0000 mon.a (mon.0) 3237 : audit [INF] from='client.48701 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "pool2": "3842dba2-777f-4b83-b6dc-f0e3b09c5464", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-13T03:43:09.395 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mon.a.service' 2024-02-13T03:43:09.417 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:43:09.417 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-13T03:43:09.417 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-02-13T03:43:09.417 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mon.c 2024-02-13T03:43:09.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:09 smithi161 systemd[1]: Stopping Ceph mon.c for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:43:09.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:09 smithi161 bash[129184]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-mon.c 2024-02-13T03:43:09.737 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:43:09 smithi183 bash[22114]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:43:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-13T03:43:09.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:09 smithi161 bash[27879]: debug 2024-02-13T03:43:09.661+0000 7fdf98613700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-13T03:43:09.925 INFO:journalctl@ceph.mon.c.smithi161.stdout:Feb 13 03:43:09 smithi161 bash[27879]: debug 2024-02-13T03:43:09.661+0000 7fdf98613700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-02-13T03:43:09.926 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:43:09 smithi161 bash[20075]: [13/Feb/2024:03:43:09] ENGINE Bus STOPPING 2024-02-13T03:43:09.926 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:43:09 smithi161 bash[20075]: [13/Feb/2024:03:43:09] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T03:43:09.926 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:43:09 smithi161 bash[20075]: [13/Feb/2024:03:43:09] ENGINE Bus STOPPED 2024-02-13T03:43:09.926 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:43:09 smithi161 bash[20075]: [13/Feb/2024:03:43:09] ENGINE Bus STARTING 2024-02-13T03:43:09.926 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:43:09 smithi161 bash[20075]: [13/Feb/2024:03:43:09] ENGINE Serving on http://:::9283 2024-02-13T03:43:09.926 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:43:09 smithi161 bash[20075]: [13/Feb/2024:03:43:09] ENGINE Bus STARTED 2024-02-13T03:43:10.527 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mon.c.service' 2024-02-13T03:43:10.550 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:43:10.550 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-02-13T03:43:10.550 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-02-13T03:43:10.550 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mon.b 2024-02-13T03:43:10.972 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:10 smithi183 systemd[1]: Stopping Ceph mon.b for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:43:10.972 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:10 smithi183 bash[49468]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-mon.b 2024-02-13T03:43:10.972 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:10 smithi183 bash[20824]: debug 2024-02-13T03:43:10.762+0000 7f25cfdc9700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-13T03:43:10.972 INFO:journalctl@ceph.mon.b.smithi183.stdout:Feb 13 03:43:10 smithi183 bash[20824]: debug 2024-02-13T03:43:10.762+0000 7f25cfdc9700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-02-13T03:43:11.425 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:43:11 smithi161 bash[20075]: ::ffff:172.21.15.183 - - [13/Feb/2024:03:43:11] "GET /metrics HTTP/1.1" 200 190835 "" "Prometheus/2.33.4" 2024-02-13T03:43:11.699 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mon.b.service' 2024-02-13T03:43:11.726 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:43:11.726 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-02-13T03:43:11.726 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-13T03:43:11.726 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mgr.y 2024-02-13T03:43:11.999 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:43:11 smithi161 systemd[1]: Stopping Ceph mgr.y for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:43:12.000 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:43:11 smithi161 bash[129321]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-mgr.y 2024-02-13T03:43:12.000 INFO:journalctl@ceph.mgr.y.smithi161.stdout:Feb 13 03:43:11 smithi161 bash[20075]: debug 2024-02-13T03:43:11.929+0000 7f0178b2a700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-13T03:43:12.730 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mgr.y.service' 2024-02-13T03:43:12.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:43:12.749 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-13T03:43:12.749 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-13T03:43:12.749 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mgr.x 2024-02-13T03:43:13.039 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:43:12 smithi183 systemd[1]: Stopping Ceph mgr.x for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:43:13.039 INFO:journalctl@ceph.mgr.x.smithi183.stdout:Feb 13 03:43:12 smithi183 bash[49599]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-mgr.x 2024-02-13T03:43:13.742 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@mgr.x.service' 2024-02-13T03:43:13.765 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:43:13.765 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-13T03:43:13.765 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-13T03:43:13.765 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.0 2024-02-13T03:43:14.175 INFO:journalctl@ceph.osd.0.smithi161.stdout:Feb 13 03:43:13 smithi161 systemd[1]: Stopping Ceph osd.0 for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:43:14.175 INFO:journalctl@ceph.osd.0.smithi161.stdout:Feb 13 03:43:13 smithi161 bash[129451]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd.0 2024-02-13T03:43:14.175 INFO:journalctl@ceph.osd.0.smithi161.stdout:Feb 13 03:43:14 smithi161 bash[31189]: debug 2024-02-13T03:43:14.001+0000 7fcf9e71b700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-13T03:43:14.175 INFO:journalctl@ceph.osd.0.smithi161.stdout:Feb 13 03:43:14 smithi161 bash[31189]: debug 2024-02-13T03:43:14.001+0000 7fcf9e71b700 -1 osd.0 746 *** Got signal Terminated *** 2024-02-13T03:43:14.176 INFO:journalctl@ceph.osd.0.smithi161.stdout:Feb 13 03:43:14 smithi161 bash[31189]: debug 2024-02-13T03:43:14.001+0000 7fcf9e71b700 -1 osd.0 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T03:43:20.175 INFO:journalctl@ceph.osd.0.smithi161.stdout:Feb 13 03:43:19 smithi161 bash[129451]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd-0 2024-02-13T03:43:21.733 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.0.service' 2024-02-13T03:43:21.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:43:21.760 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-13T03:43:21.760 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-13T03:43:21.760 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.1 2024-02-13T03:43:21.995 INFO:journalctl@ceph.osd.1.smithi161.stdout:Feb 13 03:43:21 smithi161 systemd[1]: Stopping Ceph osd.1 for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:43:21.995 INFO:journalctl@ceph.osd.1.smithi161.stdout:Feb 13 03:43:21 smithi161 bash[129724]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd.1 2024-02-13T03:43:22.425 INFO:journalctl@ceph.osd.1.smithi161.stdout:Feb 13 03:43:21 smithi161 bash[36121]: debug 2024-02-13T03:43:21.992+0000 7fa2aa9f5700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-13T03:43:22.425 INFO:journalctl@ceph.osd.1.smithi161.stdout:Feb 13 03:43:21 smithi161 bash[36121]: debug 2024-02-13T03:43:21.992+0000 7fa2aa9f5700 -1 osd.1 746 *** Got signal Terminated *** 2024-02-13T03:43:22.425 INFO:journalctl@ceph.osd.1.smithi161.stdout:Feb 13 03:43:21 smithi161 bash[36121]: debug 2024-02-13T03:43:21.992+0000 7fa2aa9f5700 -1 osd.1 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T03:43:23.926 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:23 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:23.595Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:43:23.926 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:23 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:23.597Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:43:28.175 INFO:journalctl@ceph.osd.1.smithi161.stdout:Feb 13 03:43:27 smithi161 bash[129724]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd-1 2024-02-13T03:43:29.643 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.1.service' 2024-02-13T03:43:29.669 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:43:29.670 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-13T03:43:29.670 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-13T03:43:29.670 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.2 2024-02-13T03:43:29.911 INFO:journalctl@ceph.osd.2.smithi161.stdout:Feb 13 03:43:29 smithi161 systemd[1]: Stopping Ceph osd.2 for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:43:29.911 INFO:journalctl@ceph.osd.2.smithi161.stdout:Feb 13 03:43:29 smithi161 bash[129993]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd.2 2024-02-13T03:43:30.175 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:30 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:30.144Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:43:30.175 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:30 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:30.145Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:43:30.175 INFO:journalctl@ceph.osd.2.smithi161.stdout:Feb 13 03:43:29 smithi161 bash[41241]: debug 2024-02-13T03:43:29.908+0000 7f6d2fa67700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-13T03:43:30.176 INFO:journalctl@ceph.osd.2.smithi161.stdout:Feb 13 03:43:29 smithi161 bash[41241]: debug 2024-02-13T03:43:29.908+0000 7f6d2fa67700 -1 osd.2 746 *** Got signal Terminated *** 2024-02-13T03:43:30.176 INFO:journalctl@ceph.osd.2.smithi161.stdout:Feb 13 03:43:29 smithi161 bash[41241]: debug 2024-02-13T03:43:29.908+0000 7f6d2fa67700 -1 osd.2 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T03:43:33.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:33 smithi161 bash[53972]: level=error ts=2024-02-13T03:43:33.595Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:43:33.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:33 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:33.596Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:43:33.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:33 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:33.597Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:43:35.925 INFO:journalctl@ceph.osd.2.smithi161.stdout:Feb 13 03:43:35 smithi161 bash[129993]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd-2 2024-02-13T03:43:37.593 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.2.service' 2024-02-13T03:43:37.616 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:43:37.617 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-13T03:43:37.617 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-13T03:43:37.617 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.3 2024-02-13T03:43:37.925 INFO:journalctl@ceph.osd.3.smithi161.stdout:Feb 13 03:43:37 smithi161 systemd[1]: Stopping Ceph osd.3 for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:43:37.925 INFO:journalctl@ceph.osd.3.smithi161.stdout:Feb 13 03:43:37 smithi161 bash[130262]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd.3 2024-02-13T03:43:37.926 INFO:journalctl@ceph.osd.3.smithi161.stdout:Feb 13 03:43:37 smithi161 bash[46243]: debug 2024-02-13T03:43:37.832+0000 7f9d332b7700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-13T03:43:37.926 INFO:journalctl@ceph.osd.3.smithi161.stdout:Feb 13 03:43:37 smithi161 bash[46243]: debug 2024-02-13T03:43:37.832+0000 7f9d332b7700 -1 osd.3 746 *** Got signal Terminated *** 2024-02-13T03:43:37.926 INFO:journalctl@ceph.osd.3.smithi161.stdout:Feb 13 03:43:37 smithi161 bash[46243]: debug 2024-02-13T03:43:37.832+0000 7f9d332b7700 -1 osd.3 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T03:43:40.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:40 smithi161 bash[53972]: level=error ts=2024-02-13T03:43:40.144Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:43:40.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:40 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:40.145Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:43:40.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:40 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:40.146Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:43:40.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:40 smithi183 bash[34690]: debug 2024-02-13T03:43:40.693+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:41.175 INFO:journalctl@ceph.osd.3.smithi161.stdout:Feb 13 03:43:40 smithi161 bash[46243]: debug 2024-02-13T03:43:40.736+0000 7f9d30a35700 -1 osd.3 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:16.653811+0000 front 2024-02-13T03:43:16.653855+0000 (oldest deadline 2024-02-13T03:43:40.153833+0000) 2024-02-13T03:43:41.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:41 smithi183 bash[34690]: debug 2024-02-13T03:43:41.741+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:41.987 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:41 smithi183 bash[24826]: debug 2024-02-13T03:43:41.909+0000 7f27e873f700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:16.210478+0000 front 2024-02-13T03:43:16.210705+0000 (oldest deadline 2024-02-13T03:43:41.510515+0000) 2024-02-13T03:43:41.987 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:41 smithi183 bash[29749]: debug 2024-02-13T03:43:41.717+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:41.988 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:41 smithi183 bash[39652]: debug 2024-02-13T03:43:41.905+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:42.175 INFO:journalctl@ceph.osd.3.smithi161.stdout:Feb 13 03:43:41 smithi161 bash[46243]: debug 2024-02-13T03:43:41.704+0000 7f9d30a35700 -1 osd.3 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:16.653811+0000 front 2024-02-13T03:43:16.653855+0000 (oldest deadline 2024-02-13T03:43:40.153833+0000) 2024-02-13T03:43:42.987 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:42 smithi183 bash[24826]: debug 2024-02-13T03:43:42.933+0000 7f27e873f700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:16.210478+0000 front 2024-02-13T03:43:16.210705+0000 (oldest deadline 2024-02-13T03:43:41.510515+0000) 2024-02-13T03:43:42.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:42 smithi183 bash[34690]: debug 2024-02-13T03:43:42.693+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:42.988 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:42 smithi183 bash[29749]: debug 2024-02-13T03:43:42.701+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:42.988 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:42 smithi183 bash[39652]: debug 2024-02-13T03:43:42.901+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:43.175 INFO:journalctl@ceph.osd.3.smithi161.stdout:Feb 13 03:43:42 smithi161 bash[46243]: debug 2024-02-13T03:43:42.752+0000 7f9d30a35700 -1 osd.3 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:16.653811+0000 front 2024-02-13T03:43:16.653855+0000 (oldest deadline 2024-02-13T03:43:40.153833+0000) 2024-02-13T03:43:43.925 INFO:journalctl@ceph.osd.3.smithi161.stdout:Feb 13 03:43:43 smithi161 bash[130262]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd-3 2024-02-13T03:43:43.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:43 smithi161 bash[53972]: level=error ts=2024-02-13T03:43:43.596Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:43:43.926 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:43 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:43.597Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:43:43.926 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:43 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:43.599Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:43:43.949 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:43 smithi183 bash[29749]: debug 2024-02-13T03:43:43.657+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:43.949 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:43 smithi183 bash[34690]: debug 2024-02-13T03:43:43.733+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:44.237 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:43 smithi183 bash[24826]: debug 2024-02-13T03:43:43.961+0000 7f27e873f700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:16.210478+0000 front 2024-02-13T03:43:16.210705+0000 (oldest deadline 2024-02-13T03:43:41.510515+0000) 2024-02-13T03:43:44.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:43 smithi183 bash[39652]: debug 2024-02-13T03:43:43.945+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:44.909 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:44 smithi183 bash[29749]: debug 2024-02-13T03:43:44.625+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:44.910 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:44 smithi183 bash[34690]: debug 2024-02-13T03:43:44.765+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:45.237 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:44 smithi183 bash[24826]: debug 2024-02-13T03:43:44.985+0000 7f27e873f700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:16.210478+0000 front 2024-02-13T03:43:16.210705+0000 (oldest deadline 2024-02-13T03:43:41.510515+0000) 2024-02-13T03:43:45.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:44 smithi183 bash[39652]: debug 2024-02-13T03:43:44.909+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:45.570 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.3.service' 2024-02-13T03:43:45.596 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:43:45.596 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-13T03:43:45.597 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-13T03:43:45.597 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.4 2024-02-13T03:43:45.910 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:45 smithi183 bash[29749]: debug 2024-02-13T03:43:45.581+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:45.910 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:45 smithi183 bash[34690]: debug 2024-02-13T03:43:45.749+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:45.910 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:45 smithi183 systemd[1]: Stopping Ceph osd.4 for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:43:45.911 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:45 smithi183 bash[49736]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd.4 2024-02-13T03:43:45.911 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:45 smithi183 bash[24826]: debug 2024-02-13T03:43:45.821+0000 7f27eb7c2700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-13T03:43:45.911 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:45 smithi183 bash[24826]: debug 2024-02-13T03:43:45.821+0000 7f27eb7c2700 -1 osd.4 746 *** Got signal Terminated *** 2024-02-13T03:43:45.911 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:45 smithi183 bash[24826]: debug 2024-02-13T03:43:45.821+0000 7f27eb7c2700 -1 osd.4 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T03:43:46.237 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:46 smithi183 bash[24826]: debug 2024-02-13T03:43:46.033+0000 7f27e873f700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:16.210478+0000 front 2024-02-13T03:43:16.210705+0000 (oldest deadline 2024-02-13T03:43:41.510515+0000) 2024-02-13T03:43:46.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:45 smithi183 bash[39652]: debug 2024-02-13T03:43:45.909+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:46.919 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:46 smithi183 bash[29749]: debug 2024-02-13T03:43:46.581+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:46.919 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:46 smithi183 bash[34690]: debug 2024-02-13T03:43:46.769+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:47.237 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:47 smithi183 bash[24826]: debug 2024-02-13T03:43:47.085+0000 7f27e873f700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:16.210478+0000 front 2024-02-13T03:43:16.210705+0000 (oldest deadline 2024-02-13T03:43:41.510515+0000) 2024-02-13T03:43:47.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:46 smithi183 bash[39652]: debug 2024-02-13T03:43:46.917+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:47.801 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:47 smithi183 bash[29749]: debug 2024-02-13T03:43:47.541+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:47.801 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:47 smithi183 bash[29749]: debug 2024-02-13T03:43:47.541+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.845289+0000 front 2024-02-13T03:43:24.845711+0000 (oldest deadline 2024-02-13T03:43:47.145476+0000) 2024-02-13T03:43:48.126 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:47 smithi183 bash[34690]: debug 2024-02-13T03:43:47.801+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:48.126 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:47 smithi183 bash[34690]: debug 2024-02-13T03:43:47.801+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:43:48.126 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:47 smithi183 bash[39652]: debug 2024-02-13T03:43:47.921+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:48.127 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:47 smithi183 bash[39652]: debug 2024-02-13T03:43:47.921+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:43:48.487 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:48 smithi183 bash[24826]: debug 2024-02-13T03:43:48.125+0000 7f27e873f700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:16.210478+0000 front 2024-02-13T03:43:16.210705+0000 (oldest deadline 2024-02-13T03:43:41.510515+0000) 2024-02-13T03:43:48.487 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:48 smithi183 bash[24826]: debug 2024-02-13T03:43:48.125+0000 7f27e873f700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.511488+0000 front 2024-02-13T03:43:21.511594+0000 (oldest deadline 2024-02-13T03:43:47.411358+0000) 2024-02-13T03:43:48.971 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:48 smithi183 bash[29749]: debug 2024-02-13T03:43:48.565+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:48.972 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:48 smithi183 bash[29749]: debug 2024-02-13T03:43:48.565+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.845289+0000 front 2024-02-13T03:43:24.845711+0000 (oldest deadline 2024-02-13T03:43:47.145476+0000) 2024-02-13T03:43:48.972 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:48 smithi183 bash[34690]: debug 2024-02-13T03:43:48.805+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:48.972 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:48 smithi183 bash[34690]: debug 2024-02-13T03:43:48.805+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:43:49.237 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:49 smithi183 bash[24826]: debug 2024-02-13T03:43:49.153+0000 7f27e873f700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:16.210478+0000 front 2024-02-13T03:43:16.210705+0000 (oldest deadline 2024-02-13T03:43:41.510515+0000) 2024-02-13T03:43:49.237 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:49 smithi183 bash[24826]: debug 2024-02-13T03:43:49.153+0000 7f27e873f700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.511488+0000 front 2024-02-13T03:43:21.511594+0000 (oldest deadline 2024-02-13T03:43:47.411358+0000) 2024-02-13T03:43:49.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:48 smithi183 bash[39652]: debug 2024-02-13T03:43:48.969+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:49.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:48 smithi183 bash[39652]: debug 2024-02-13T03:43:48.969+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:43:49.976 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:49 smithi183 bash[29749]: debug 2024-02-13T03:43:49.521+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:49.976 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:49 smithi183 bash[29749]: debug 2024-02-13T03:43:49.521+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.845289+0000 front 2024-02-13T03:43:24.845711+0000 (oldest deadline 2024-02-13T03:43:47.145476+0000) 2024-02-13T03:43:49.977 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:49 smithi183 bash[34690]: debug 2024-02-13T03:43:49.765+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:49.977 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:49 smithi183 bash[34690]: debug 2024-02-13T03:43:49.765+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:43:50.237 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:50 smithi183 bash[24826]: debug 2024-02-13T03:43:50.189+0000 7f27e873f700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:16.210478+0000 front 2024-02-13T03:43:16.210705+0000 (oldest deadline 2024-02-13T03:43:41.510515+0000) 2024-02-13T03:43:50.237 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:50 smithi183 bash[24826]: debug 2024-02-13T03:43:50.189+0000 7f27e873f700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.511488+0000 front 2024-02-13T03:43:21.511594+0000 (oldest deadline 2024-02-13T03:43:47.411358+0000) 2024-02-13T03:43:50.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:49 smithi183 bash[39652]: debug 2024-02-13T03:43:49.973+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:50.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:49 smithi183 bash[39652]: debug 2024-02-13T03:43:49.973+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:43:50.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:50 smithi161 bash[53972]: level=error ts=2024-02-13T03:43:50.144Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:43:50.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:50 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:50.145Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:43:50.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:50 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:50.146Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:43:50.750 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:50 smithi183 bash[29749]: debug 2024-02-13T03:43:50.497+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:50.750 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:50 smithi183 bash[29749]: debug 2024-02-13T03:43:50.497+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.845289+0000 front 2024-02-13T03:43:24.845711+0000 (oldest deadline 2024-02-13T03:43:47.145476+0000) 2024-02-13T03:43:51.237 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:50 smithi183 bash[34690]: debug 2024-02-13T03:43:50.749+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:51.237 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:50 smithi183 bash[34690]: debug 2024-02-13T03:43:50.749+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:43:51.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:50 smithi183 bash[39652]: debug 2024-02-13T03:43:50.945+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:51.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:50 smithi183 bash[39652]: debug 2024-02-13T03:43:50.945+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:43:51.949 INFO:journalctl@ceph.osd.4.smithi183.stdout:Feb 13 03:43:51 smithi183 bash[49736]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd-4 2024-02-13T03:43:51.949 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:51 smithi183 bash[29749]: debug 2024-02-13T03:43:51.505+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:51.949 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:51 smithi183 bash[29749]: debug 2024-02-13T03:43:51.505+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.845289+0000 front 2024-02-13T03:43:24.845711+0000 (oldest deadline 2024-02-13T03:43:47.145476+0000) 2024-02-13T03:43:51.949 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:51 smithi183 bash[34690]: debug 2024-02-13T03:43:51.741+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:51.949 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:51 smithi183 bash[34690]: debug 2024-02-13T03:43:51.741+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:43:52.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:51 smithi183 bash[39652]: debug 2024-02-13T03:43:51.949+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:52.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:51 smithi183 bash[39652]: debug 2024-02-13T03:43:51.949+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:43:52.797 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:52 smithi183 bash[29749]: debug 2024-02-13T03:43:52.497+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:52.797 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:52 smithi183 bash[29749]: debug 2024-02-13T03:43:52.497+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.845289+0000 front 2024-02-13T03:43:24.845711+0000 (oldest deadline 2024-02-13T03:43:47.145476+0000) 2024-02-13T03:43:52.797 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:52 smithi183 bash[34690]: debug 2024-02-13T03:43:52.737+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:52.798 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:52 smithi183 bash[34690]: debug 2024-02-13T03:43:52.737+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:43:53.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:52 smithi183 bash[39652]: debug 2024-02-13T03:43:52.973+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:53.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:52 smithi183 bash[39652]: debug 2024-02-13T03:43:52.973+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:43:53.716 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.4.service' 2024-02-13T03:43:53.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:43:53.743 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-13T03:43:53.743 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-13T03:43:53.743 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.5 2024-02-13T03:43:53.768 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:53 smithi183 bash[29749]: debug 2024-02-13T03:43:53.509+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:53.768 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:53 smithi183 bash[29749]: debug 2024-02-13T03:43:53.509+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.845289+0000 front 2024-02-13T03:43:24.845711+0000 (oldest deadline 2024-02-13T03:43:47.145476+0000) 2024-02-13T03:43:53.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:53 smithi161 bash[53972]: level=error ts=2024-02-13T03:43:53.597Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:43:53.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:53 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:53.598Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:43:53.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:43:53 smithi161 bash[53972]: level=warn ts=2024-02-13T03:43:53.599Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:43:54.237 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:53 smithi183 bash[34690]: debug 2024-02-13T03:43:53.765+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:54.237 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:53 smithi183 bash[34690]: debug 2024-02-13T03:43:53.765+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:43:54.237 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:53 smithi183 systemd[1]: Stopping Ceph osd.5 for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:43:54.237 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:53 smithi183 bash[50001]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd.5 2024-02-13T03:43:54.238 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:53 smithi183 bash[29749]: debug 2024-02-13T03:43:53.957+0000 7f47e3429700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-13T03:43:54.238 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:53 smithi183 bash[29749]: debug 2024-02-13T03:43:53.957+0000 7f47e3429700 -1 osd.5 746 *** Got signal Terminated *** 2024-02-13T03:43:54.238 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:53 smithi183 bash[29749]: debug 2024-02-13T03:43:53.957+0000 7f47e3429700 -1 osd.5 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T03:43:54.238 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:54 smithi183 bash[39652]: debug 2024-02-13T03:43:54.001+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:54.238 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:54 smithi183 bash[39652]: debug 2024-02-13T03:43:54.001+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:43:54.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:54 smithi183 bash[34690]: debug 2024-02-13T03:43:54.777+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:54.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:54 smithi183 bash[34690]: debug 2024-02-13T03:43:54.777+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:43:54.987 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:54 smithi183 bash[29749]: debug 2024-02-13T03:43:54.545+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:54.987 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:54 smithi183 bash[29749]: debug 2024-02-13T03:43:54.545+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.845289+0000 front 2024-02-13T03:43:24.845711+0000 (oldest deadline 2024-02-13T03:43:47.145476+0000) 2024-02-13T03:43:55.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:55 smithi183 bash[39652]: debug 2024-02-13T03:43:55.009+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:55.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:55 smithi183 bash[39652]: debug 2024-02-13T03:43:55.009+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:43:55.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:55 smithi183 bash[34690]: debug 2024-02-13T03:43:55.772+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:55.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:55 smithi183 bash[34690]: debug 2024-02-13T03:43:55.772+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:43:55.987 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:55 smithi183 bash[29749]: debug 2024-02-13T03:43:55.592+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:55.987 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:55 smithi183 bash[29749]: debug 2024-02-13T03:43:55.592+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.845289+0000 front 2024-02-13T03:43:24.845711+0000 (oldest deadline 2024-02-13T03:43:47.145476+0000) 2024-02-13T03:43:56.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:56 smithi183 bash[39652]: debug 2024-02-13T03:43:56.004+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:56.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:56 smithi183 bash[39652]: debug 2024-02-13T03:43:56.004+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:43:56.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:56 smithi183 bash[34690]: debug 2024-02-13T03:43:56.768+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:56.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:56 smithi183 bash[34690]: debug 2024-02-13T03:43:56.768+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:43:56.987 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:56 smithi183 bash[29749]: debug 2024-02-13T03:43:56.548+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:56.987 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:56 smithi183 bash[29749]: debug 2024-02-13T03:43:56.548+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.845289+0000 front 2024-02-13T03:43:24.845711+0000 (oldest deadline 2024-02-13T03:43:47.145476+0000) 2024-02-13T03:43:57.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:57 smithi183 bash[39652]: debug 2024-02-13T03:43:57.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:57.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:57 smithi183 bash[39652]: debug 2024-02-13T03:43:57.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:43:57.987 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:57 smithi183 bash[29749]: debug 2024-02-13T03:43:57.528+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:57.987 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:57 smithi183 bash[29749]: debug 2024-02-13T03:43:57.528+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.845289+0000 front 2024-02-13T03:43:24.845711+0000 (oldest deadline 2024-02-13T03:43:47.145476+0000) 2024-02-13T03:43:57.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:57 smithi183 bash[34690]: debug 2024-02-13T03:43:57.744+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:57.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:57 smithi183 bash[34690]: debug 2024-02-13T03:43:57.744+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:43:57.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:57 smithi183 bash[34690]: debug 2024-02-13T03:43:57.744+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:32.686729+0000 front 2024-02-13T03:43:32.687047+0000 (oldest deadline 2024-02-13T03:43:56.786962+0000) 2024-02-13T03:43:58.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:58 smithi183 bash[39652]: debug 2024-02-13T03:43:58.076+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:58.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:58 smithi183 bash[39652]: debug 2024-02-13T03:43:58.076+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:43:58.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:58 smithi183 bash[39652]: debug 2024-02-13T03:43:58.076+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:43:58.987 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:58 smithi183 bash[29749]: debug 2024-02-13T03:43:58.572+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.543138+0000 front 2024-02-13T03:43:18.543396+0000 (oldest deadline 2024-02-13T03:43:40.843057+0000) 2024-02-13T03:43:58.987 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:58 smithi183 bash[29749]: debug 2024-02-13T03:43:58.572+0000 7f47e03a6700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.845289+0000 front 2024-02-13T03:43:24.845711+0000 (oldest deadline 2024-02-13T03:43:47.145476+0000) 2024-02-13T03:43:58.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:58 smithi183 bash[34690]: debug 2024-02-13T03:43:58.712+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:58.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:58 smithi183 bash[34690]: debug 2024-02-13T03:43:58.712+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:43:58.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:58 smithi183 bash[34690]: debug 2024-02-13T03:43:58.712+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:32.686729+0000 front 2024-02-13T03:43:32.687047+0000 (oldest deadline 2024-02-13T03:43:56.786962+0000) 2024-02-13T03:43:59.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:59 smithi183 bash[39652]: debug 2024-02-13T03:43:59.044+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:43:59.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:59 smithi183 bash[39652]: debug 2024-02-13T03:43:59.044+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:43:59.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:43:59 smithi183 bash[39652]: debug 2024-02-13T03:43:59.044+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:43:59.987 INFO:journalctl@ceph.osd.5.smithi183.stdout:Feb 13 03:43:59 smithi183 bash[50001]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd-5 2024-02-13T03:43:59.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:59 smithi183 bash[34690]: debug 2024-02-13T03:43:59.680+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:43:59.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:59 smithi183 bash[34690]: debug 2024-02-13T03:43:59.680+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:43:59.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:43:59 smithi183 bash[34690]: debug 2024-02-13T03:43:59.680+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:32.686729+0000 front 2024-02-13T03:43:32.687047+0000 (oldest deadline 2024-02-13T03:43:56.786962+0000) 2024-02-13T03:44:00.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:00 smithi161 bash[53972]: level=error ts=2024-02-13T03:44:00.145Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:44:00.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:00 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:00.146Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:44:00.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:00 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:00.147Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:44:00.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:00 smithi183 bash[39652]: debug 2024-02-13T03:44:00.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:00.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:00 smithi183 bash[39652]: debug 2024-02-13T03:44:00.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:00.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:00 smithi183 bash[39652]: debug 2024-02-13T03:44:00.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:00.965 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:00 smithi183 bash[34690]: debug 2024-02-13T03:44:00.712+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:44:00.965 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:00 smithi183 bash[34690]: debug 2024-02-13T03:44:00.712+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:44:00.965 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:00 smithi183 bash[34690]: debug 2024-02-13T03:44:00.712+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:32.686729+0000 front 2024-02-13T03:43:32.687047+0000 (oldest deadline 2024-02-13T03:43:56.786962+0000) 2024-02-13T03:44:01.317 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:01 smithi183 bash[39652]: debug 2024-02-13T03:44:00.996+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:01.317 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:01 smithi183 bash[39652]: debug 2024-02-13T03:44:00.996+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:01.317 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:01 smithi183 bash[39652]: debug 2024-02-13T03:44:00.996+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:01.858 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.5.service' 2024-02-13T03:44:01.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:44:01.884 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-13T03:44:01.884 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-13T03:44:01.885 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.6 2024-02-13T03:44:01.936 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:01 smithi183 bash[34690]: debug 2024-02-13T03:44:01.676+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:44:01.936 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:01 smithi183 bash[34690]: debug 2024-02-13T03:44:01.676+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:44:01.936 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:01 smithi183 bash[34690]: debug 2024-02-13T03:44:01.676+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:32.686729+0000 front 2024-02-13T03:43:32.687047+0000 (oldest deadline 2024-02-13T03:43:56.786962+0000) 2024-02-13T03:44:02.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:02 smithi183 bash[39652]: debug 2024-02-13T03:44:02.044+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:02.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:02 smithi183 bash[39652]: debug 2024-02-13T03:44:02.044+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:02.238 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:02 smithi183 bash[39652]: debug 2024-02-13T03:44:02.044+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:02.238 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:01 smithi183 systemd[1]: Stopping Ceph osd.6 for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:44:02.238 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:01 smithi183 bash[50253]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd.6 2024-02-13T03:44:02.238 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:02 smithi183 bash[34690]: debug 2024-02-13T03:44:02.124+0000 7ffa3c951700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-13T03:44:02.238 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:02 smithi183 bash[34690]: debug 2024-02-13T03:44:02.124+0000 7ffa3c951700 -1 osd.6 746 *** Got signal Terminated *** 2024-02-13T03:44:02.239 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:02 smithi183 bash[34690]: debug 2024-02-13T03:44:02.124+0000 7ffa3c951700 -1 osd.6 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T03:44:02.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:02 smithi183 bash[34690]: debug 2024-02-13T03:44:02.684+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:44:02.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:02 smithi183 bash[34690]: debug 2024-02-13T03:44:02.684+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:44:02.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:02 smithi183 bash[34690]: debug 2024-02-13T03:44:02.684+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:32.686729+0000 front 2024-02-13T03:43:32.687047+0000 (oldest deadline 2024-02-13T03:43:56.786962+0000) 2024-02-13T03:44:03.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:03 smithi183 bash[39652]: debug 2024-02-13T03:44:03.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:03.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:03 smithi183 bash[39652]: debug 2024-02-13T03:44:03.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:03.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:03 smithi183 bash[39652]: debug 2024-02-13T03:44:03.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:03.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:03 smithi161 bash[53972]: level=error ts=2024-02-13T03:44:03.598Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:44:03.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:03 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:03.599Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:44:03.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:03 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:03.600Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:44:03.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:03 smithi183 bash[34690]: debug 2024-02-13T03:44:03.700+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:44:03.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:03 smithi183 bash[34690]: debug 2024-02-13T03:44:03.700+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:44:03.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:03 smithi183 bash[34690]: debug 2024-02-13T03:44:03.700+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:32.686729+0000 front 2024-02-13T03:43:32.687047+0000 (oldest deadline 2024-02-13T03:43:56.786962+0000) 2024-02-13T03:44:04.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:04 smithi183 bash[39652]: debug 2024-02-13T03:44:04.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:04.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:04 smithi183 bash[39652]: debug 2024-02-13T03:44:04.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:04.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:04 smithi183 bash[39652]: debug 2024-02-13T03:44:04.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:04.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:04 smithi183 bash[39652]: debug 2024-02-13T03:44:04.048+0000 7f3049e2d700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1329 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-13T03:44:04.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:04 smithi183 bash[34690]: debug 2024-02-13T03:44:04.704+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:44:04.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:04 smithi183 bash[34690]: debug 2024-02-13T03:44:04.704+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:44:04.988 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:04 smithi183 bash[34690]: debug 2024-02-13T03:44:04.704+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:32.686729+0000 front 2024-02-13T03:43:32.687047+0000 (oldest deadline 2024-02-13T03:43:56.786962+0000) 2024-02-13T03:44:05.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:05 smithi183 bash[39652]: debug 2024-02-13T03:44:05.060+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:05.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:05 smithi183 bash[39652]: debug 2024-02-13T03:44:05.060+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:05.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:05 smithi183 bash[39652]: debug 2024-02-13T03:44:05.060+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:05.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:05 smithi183 bash[39652]: debug 2024-02-13T03:44:05.060+0000 7f3049e2d700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1329 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-13T03:44:05.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:05 smithi183 bash[34690]: debug 2024-02-13T03:44:05.700+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:44:05.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:05 smithi183 bash[34690]: debug 2024-02-13T03:44:05.700+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:44:05.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:05 smithi183 bash[34690]: debug 2024-02-13T03:44:05.700+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:32.686729+0000 front 2024-02-13T03:43:32.687047+0000 (oldest deadline 2024-02-13T03:43:56.786962+0000) 2024-02-13T03:44:06.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:06 smithi183 bash[39652]: debug 2024-02-13T03:44:06.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:06.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:06 smithi183 bash[39652]: debug 2024-02-13T03:44:06.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:06.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:06 smithi183 bash[39652]: debug 2024-02-13T03:44:06.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:06.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:06 smithi183 bash[39652]: debug 2024-02-13T03:44:06.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6815 osd.3 since back 2024-02-13T03:43:42.557908+0000 front 2024-02-13T03:43:42.557939+0000 (oldest deadline 2024-02-13T03:44:05.457720+0000) 2024-02-13T03:44:06.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:06 smithi183 bash[39652]: debug 2024-02-13T03:44:06.036+0000 7f3049e2d700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1329 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-13T03:44:06.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:06 smithi183 bash[34690]: debug 2024-02-13T03:44:06.668+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:15.083431+0000 front 2024-02-13T03:43:15.083855+0000 (oldest deadline 2024-02-13T03:43:39.783567+0000) 2024-02-13T03:44:06.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:06 smithi183 bash[34690]: debug 2024-02-13T03:44:06.668+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:24.485387+0000 front 2024-02-13T03:43:24.485170+0000 (oldest deadline 2024-02-13T03:43:47.385254+0000) 2024-02-13T03:44:06.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:06 smithi183 bash[34690]: debug 2024-02-13T03:44:06.668+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:32.686729+0000 front 2024-02-13T03:43:32.687047+0000 (oldest deadline 2024-02-13T03:43:56.786962+0000) 2024-02-13T03:44:06.987 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:06 smithi183 bash[34690]: debug 2024-02-13T03:44:06.668+0000 7ffa380cb700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.161:6815 osd.3 since back 2024-02-13T03:43:42.390586+0000 front 2024-02-13T03:43:42.390692+0000 (oldest deadline 2024-02-13T03:44:05.890566+0000) 2024-02-13T03:44:07.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:07 smithi183 bash[39652]: debug 2024-02-13T03:44:07.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:07.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:07 smithi183 bash[39652]: debug 2024-02-13T03:44:07.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:07.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:07 smithi183 bash[39652]: debug 2024-02-13T03:44:07.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:07.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:07 smithi183 bash[39652]: debug 2024-02-13T03:44:07.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6815 osd.3 since back 2024-02-13T03:43:42.557908+0000 front 2024-02-13T03:43:42.557939+0000 (oldest deadline 2024-02-13T03:44:05.457720+0000) 2024-02-13T03:44:07.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:07 smithi183 bash[39652]: debug 2024-02-13T03:44:07.048+0000 7f3049e2d700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1329 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-13T03:44:08.237 INFO:journalctl@ceph.osd.6.smithi183.stdout:Feb 13 03:44:07 smithi183 bash[50253]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd-6 2024-02-13T03:44:08.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:08 smithi183 bash[39652]: debug 2024-02-13T03:44:08.024+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:08.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:08 smithi183 bash[39652]: debug 2024-02-13T03:44:08.024+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:08.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:08 smithi183 bash[39652]: debug 2024-02-13T03:44:08.024+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:08.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:08 smithi183 bash[39652]: debug 2024-02-13T03:44:08.024+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6815 osd.3 since back 2024-02-13T03:43:42.557908+0000 front 2024-02-13T03:43:42.557939+0000 (oldest deadline 2024-02-13T03:44:05.457720+0000) 2024-02-13T03:44:08.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:08 smithi183 bash[39652]: debug 2024-02-13T03:44:08.024+0000 7f3049e2d700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1329 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-13T03:44:09.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:09 smithi183 bash[39652]: debug 2024-02-13T03:44:09.004+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:09.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:09 smithi183 bash[39652]: debug 2024-02-13T03:44:09.004+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:09.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:09 smithi183 bash[39652]: debug 2024-02-13T03:44:09.004+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:09.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:09 smithi183 bash[39652]: debug 2024-02-13T03:44:09.004+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6815 osd.3 since back 2024-02-13T03:43:42.557908+0000 front 2024-02-13T03:43:42.557939+0000 (oldest deadline 2024-02-13T03:44:05.457720+0000) 2024-02-13T03:44:09.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:09 smithi183 bash[39652]: debug 2024-02-13T03:44:09.004+0000 7f3049e2d700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1329 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-13T03:44:09.906 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.6.service' 2024-02-13T03:44:09.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:44:09.975 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-13T03:44:09.975 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-13T03:44:09.975 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.7 2024-02-13T03:44:10.216 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:10 smithi183 systemd[1]: Stopping Ceph osd.7 for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:44:10.216 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:10 smithi183 bash[39652]: debug 2024-02-13T03:44:10.052+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:10.216 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:10 smithi183 bash[39652]: debug 2024-02-13T03:44:10.052+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:10.217 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:10 smithi183 bash[39652]: debug 2024-02-13T03:44:10.052+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:10.217 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:10 smithi183 bash[39652]: debug 2024-02-13T03:44:10.052+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6815 osd.3 since back 2024-02-13T03:43:42.557908+0000 front 2024-02-13T03:43:42.557939+0000 (oldest deadline 2024-02-13T03:44:05.457720+0000) 2024-02-13T03:44:10.217 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:10 smithi183 bash[39652]: debug 2024-02-13T03:44:10.052+0000 7f3049e2d700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1329 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-13T03:44:10.217 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:10 smithi183 bash[50512]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd.7 2024-02-13T03:44:10.217 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:10 smithi183 bash[39652]: debug 2024-02-13T03:44:10.212+0000 7f304e6b3700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-13T03:44:10.217 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:10 smithi183 bash[39652]: debug 2024-02-13T03:44:10.212+0000 7f304e6b3700 -1 osd.7 746 *** Got signal Terminated *** 2024-02-13T03:44:10.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:10 smithi161 bash[53972]: level=error ts=2024-02-13T03:44:10.145Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:44:10.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:10 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:10.146Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:44:10.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:10 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:10.147Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:44:11.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:10 smithi183 bash[39652]: debug 2024-02-13T03:44:10.212+0000 7f304e6b3700 -1 osd.7 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T03:44:11.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:11 smithi183 bash[39652]: debug 2024-02-13T03:44:11.052+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:11.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:11 smithi183 bash[39652]: debug 2024-02-13T03:44:11.052+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:11.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:11 smithi183 bash[39652]: debug 2024-02-13T03:44:11.052+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:11.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:11 smithi183 bash[39652]: debug 2024-02-13T03:44:11.052+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6815 osd.3 since back 2024-02-13T03:43:42.557908+0000 front 2024-02-13T03:43:42.557939+0000 (oldest deadline 2024-02-13T03:44:05.457720+0000) 2024-02-13T03:44:11.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:11 smithi183 bash[39652]: debug 2024-02-13T03:44:11.052+0000 7f3049e2d700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1329 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-13T03:44:12.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:12 smithi183 bash[39652]: debug 2024-02-13T03:44:12.056+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:12.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:12 smithi183 bash[39652]: debug 2024-02-13T03:44:12.056+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:12.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:12 smithi183 bash[39652]: debug 2024-02-13T03:44:12.056+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:12.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:12 smithi183 bash[39652]: debug 2024-02-13T03:44:12.056+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6815 osd.3 since back 2024-02-13T03:43:42.557908+0000 front 2024-02-13T03:43:42.557939+0000 (oldest deadline 2024-02-13T03:44:05.457720+0000) 2024-02-13T03:44:12.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:12 smithi183 bash[39652]: debug 2024-02-13T03:44:12.056+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.183:6802 osd.4 since back 2024-02-13T03:43:45.458546+0000 front 2024-02-13T03:43:45.458689+0000 (oldest deadline 2024-02-13T03:44:11.358387+0000) 2024-02-13T03:44:12.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:12 smithi183 bash[39652]: debug 2024-02-13T03:44:12.056+0000 7f3049e2d700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1329 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-13T03:44:13.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:13 smithi183 bash[39652]: debug 2024-02-13T03:44:13.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:13.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:13 smithi183 bash[39652]: debug 2024-02-13T03:44:13.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:13.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:13 smithi183 bash[39652]: debug 2024-02-13T03:44:13.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:13.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:13 smithi183 bash[39652]: debug 2024-02-13T03:44:13.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6815 osd.3 since back 2024-02-13T03:43:42.557908+0000 front 2024-02-13T03:43:42.557939+0000 (oldest deadline 2024-02-13T03:44:05.457720+0000) 2024-02-13T03:44:13.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:13 smithi183 bash[39652]: debug 2024-02-13T03:44:13.048+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.183:6802 osd.4 since back 2024-02-13T03:43:45.458546+0000 front 2024-02-13T03:43:45.458689+0000 (oldest deadline 2024-02-13T03:44:11.358387+0000) 2024-02-13T03:44:13.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:13 smithi183 bash[39652]: debug 2024-02-13T03:44:13.048+0000 7f3049e2d700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1329 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-13T03:44:13.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:13 smithi161 bash[53972]: level=error ts=2024-02-13T03:44:13.599Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:44:13.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:13 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:13.600Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:44:13.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:13 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:13.601Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:44:14.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:14 smithi183 bash[39652]: debug 2024-02-13T03:44:14.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:14.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:14 smithi183 bash[39652]: debug 2024-02-13T03:44:14.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:14.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:14 smithi183 bash[39652]: debug 2024-02-13T03:44:14.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:14.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:14 smithi183 bash[39652]: debug 2024-02-13T03:44:14.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6815 osd.3 since back 2024-02-13T03:43:42.557908+0000 front 2024-02-13T03:43:42.557939+0000 (oldest deadline 2024-02-13T03:44:05.457720+0000) 2024-02-13T03:44:14.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:14 smithi183 bash[39652]: debug 2024-02-13T03:44:14.036+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.183:6802 osd.4 since back 2024-02-13T03:43:45.458546+0000 front 2024-02-13T03:43:45.458689+0000 (oldest deadline 2024-02-13T03:44:11.358387+0000) 2024-02-13T03:44:14.488 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:14 smithi183 bash[39652]: debug 2024-02-13T03:44:14.036+0000 7f3049e2d700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1329 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-13T03:44:15.343 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:14 smithi183 bash[39652]: debug 2024-02-13T03:44:14.988+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6803 osd.0 since back 2024-02-13T03:43:18.250984+0000 front 2024-02-13T03:43:18.251254+0000 (oldest deadline 2024-02-13T03:43:41.751160+0000) 2024-02-13T03:44:15.343 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:14 smithi183 bash[39652]: debug 2024-02-13T03:44:14.988+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6807 osd.1 since back 2024-02-13T03:43:21.751990+0000 front 2024-02-13T03:43:21.752254+0000 (oldest deadline 2024-02-13T03:43:47.652090+0000) 2024-02-13T03:44:15.343 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:14 smithi183 bash[39652]: debug 2024-02-13T03:44:14.988+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6811 osd.2 since back 2024-02-13T03:43:33.954845+0000 front 2024-02-13T03:43:33.955203+0000 (oldest deadline 2024-02-13T03:43:58.055071+0000) 2024-02-13T03:44:15.344 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:14 smithi183 bash[39652]: debug 2024-02-13T03:44:14.988+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.161:6815 osd.3 since back 2024-02-13T03:43:42.557908+0000 front 2024-02-13T03:43:42.557939+0000 (oldest deadline 2024-02-13T03:44:05.457720+0000) 2024-02-13T03:44:15.344 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:14 smithi183 bash[39652]: debug 2024-02-13T03:44:14.988+0000 7f3049e2d700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.183:6802 osd.4 since back 2024-02-13T03:43:45.458546+0000 front 2024-02-13T03:43:45.458689+0000 (oldest deadline 2024-02-13T03:44:11.358387+0000) 2024-02-13T03:44:15.344 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:14 smithi183 bash[39652]: debug 2024-02-13T03:44:14.988+0000 7f3049e2d700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24445.0:1329 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-13T03:44:16.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Feb 13 03:44:15 smithi183 bash[50512]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-osd-7 2024-02-13T03:44:17.933 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@osd.7.service' 2024-02-13T03:44:17.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:44:17.957 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-13T03:44:17.957 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-13T03:44:17.957 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@rgw.foo.a 2024-02-13T03:44:18.425 INFO:journalctl@ceph.rgw.foo.a.smithi161.stdout:Feb 13 03:44:17 smithi161 systemd[1]: Stopping Ceph rgw.foo.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:44:18.425 INFO:journalctl@ceph.rgw.foo.a.smithi161.stdout:Feb 13 03:44:18 smithi161 bash[130526]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-rgw.foo.a 2024-02-13T03:44:18.426 INFO:journalctl@ceph.rgw.foo.a.smithi161.stdout:Feb 13 03:44:18 smithi161 bash[49453]: debug 2024-02-13T03:44:18.167+0000 7f5af5f7c700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-13T03:44:18.426 INFO:journalctl@ceph.rgw.foo.a.smithi161.stdout:Feb 13 03:44:18 smithi161 bash[49453]: debug 2024-02-13T03:44:18.167+0000 7f5b0d046880 -1 shutting down 2024-02-13T03:44:20.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:20 smithi161 bash[53972]: level=error ts=2024-02-13T03:44:20.146Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:44:20.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:20 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:20.147Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:44:20.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:20 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:20.148Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:44:23.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:23 smithi161 bash[53972]: level=error ts=2024-02-13T03:44:23.599Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:44:23.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:23 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:23.600Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:44:23.926 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:23 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:23.601Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:44:28.835 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@rgw.foo.a.service' 2024-02-13T03:44:28.857 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:44:28.857 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-13T03:44:28.857 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-13T03:44:28.857 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@prometheus.a 2024-02-13T03:44:29.131 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:28 smithi183 systemd[1]: Stopping Ceph prometheus.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:44:29.132 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:28 smithi183 bash[50775]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-prometheus.a 2024-02-13T03:44:29.132 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:29 smithi183 bash[47417]: ts=2024-02-13T03:44:29.077Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-13T03:44:29.132 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:29 smithi183 bash[47417]: ts=2024-02-13T03:44:29.077Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-02-13T03:44:29.132 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:29 smithi183 bash[47417]: ts=2024-02-13T03:44:29.077Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-02-13T03:44:29.132 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:29 smithi183 bash[47417]: ts=2024-02-13T03:44:29.077Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-02-13T03:44:29.132 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:29 smithi183 bash[47417]: ts=2024-02-13T03:44:29.077Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-02-13T03:44:29.132 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:29 smithi183 bash[47417]: ts=2024-02-13T03:44:29.077Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-02-13T03:44:29.132 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:29 smithi183 bash[47417]: ts=2024-02-13T03:44:29.077Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-13T03:44:29.132 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:29 smithi183 bash[47417]: ts=2024-02-13T03:44:29.077Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-02-13T03:44:29.132 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:29 smithi183 bash[47417]: ts=2024-02-13T03:44:29.077Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-02-13T03:44:29.132 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:29 smithi183 bash[47417]: ts=2024-02-13T03:44:29.078Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-02-13T03:44:29.132 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:29 smithi183 bash[47417]: ts=2024-02-13T03:44:29.078Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-02-13T03:44:29.132 INFO:journalctl@ceph.prometheus.a.smithi183.stdout:Feb 13 03:44:29 smithi183 bash[47417]: ts=2024-02-13T03:44:29.078Z caller=main.go:1066 level=info msg="See you next time!" 2024-02-13T03:44:29.737 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@prometheus.a.service' 2024-02-13T03:44:29.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T03:44:29.759 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-13T03:44:29.759 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm rm-cluster --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-13T03:44:30.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:30 smithi161 bash[53972]: level=error ts=2024-02-13T03:44:30.147Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:44:30.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:30 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:30.148Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:44:30.425 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:30 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:30.149Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:44:31.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:31 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:31.627Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:44:31.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:31 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:31.628Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:44:33.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:33 smithi161 bash[53972]: level=error ts=2024-02-13T03:44:33.601Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:44:33.925 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:33 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:33.602Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi161.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.161:8443: connect: connection refused" 2024-02-13T03:44:33.926 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:33 smithi161 bash[53972]: level=warn ts=2024-02-13T03:44:33.603Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi183.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.183:8443: connect: connection refused" 2024-02-13T03:44:35.925 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:44:35 smithi161 systemd[1]: Stopping Ceph node-exporter.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:44:35.925 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:44:35 smithi161 bash[130942]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-node-exporter.a 2024-02-13T03:44:36.675 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:36 smithi161 systemd[1]: Stopping Ceph alertmanager.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:44:36.675 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:36 smithi161 bash[131099]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-alertmanager.a 2024-02-13T03:44:36.676 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:44:36 smithi161 bash[130942]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-node-exporter-a 2024-02-13T03:44:36.676 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:44:36 smithi161 systemd[1]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-13T03:44:36.676 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:44:36 smithi161 systemd[1]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-13T03:44:36.676 INFO:journalctl@ceph.node-exporter.a.smithi161.stdout:Feb 13 03:44:36 smithi161 systemd[1]: Stopped Ceph node-exporter.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:44:37.175 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:36 smithi161 bash[53972]: level=info ts=2024-02-13T03:44:36.788Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-02-13T03:44:37.675 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:37 smithi161 bash[131099]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-alertmanager-a 2024-02-13T03:44:37.675 INFO:journalctl@ceph.alertmanager.a.smithi161.stdout:Feb 13 03:44:37 smithi161 systemd[1]: Stopped Ceph alertmanager.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:44:37.963 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm rm-cluster --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-13T03:44:43.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:44:43 smithi183 systemd[1]: Stopping Ceph iscsi.iscsi.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:44:43.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:44:43 smithi183 bash[51129]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-iscsi.iscsi.a 2024-02-13T03:44:43.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:44:43 smithi183 bash[43494]: debug Shutdown received 2024-02-13T03:44:54.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:44:54 smithi183 bash[51129]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-iscsi-iscsi-a 2024-02-13T03:44:54.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:44:54 smithi183 systemd[1]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-02-13T03:44:55.699 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:44:55 smithi183 systemd[1]: Stopping Ceph node-exporter.b for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:44:55.700 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:44:55 smithi183 bash[51334]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-node-exporter.b 2024-02-13T03:44:55.700 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:44:55 smithi183 bash[51226]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-iscsi-iscsi-a-tcmu 2024-02-13T03:44:55.700 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:44:55 smithi183 systemd[1]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-02-13T03:44:55.700 INFO:journalctl@ceph.iscsi.iscsi.a.smithi183.stdout:Feb 13 03:44:55 smithi183 systemd[1]: Stopped Ceph iscsi.iscsi.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:44:56.487 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:44:56 smithi183 bash[51334]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-node-exporter-b 2024-02-13T03:44:56.487 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:44:56 smithi183 systemd[1]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-13T03:44:56.487 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:44:56 smithi183 systemd[1]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-13T03:44:56.487 INFO:journalctl@ceph.node-exporter.b.smithi183.stdout:Feb 13 03:44:56 smithi183 systemd[1]: Stopped Ceph node-exporter.b for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:44:56.828 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:44:56 smithi183 systemd[1]: Stopping Ceph grafana.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715... 2024-02-13T03:44:56.828 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:44:56 smithi183 bash[51511]: Error response from daemon: No such container: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-grafana.a 2024-02-13T03:44:56.828 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:44:56 smithi183 bash[46554]: t=2024-02-13T03:44:56+0000 lvl=info msg="Shutdown started" logger=server reason="System signal: terminated" 2024-02-13T03:44:57.708 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:44:57 smithi183 bash[51511]: ceph-b9b3ef98-ca1d-11ee-95b9-87774f69a715-grafana-a 2024-02-13T03:44:57.709 INFO:journalctl@ceph.grafana.a.smithi183.stdout:Feb 13 03:44:57 smithi183 systemd[1]: Stopped Ceph grafana.a for b9b3ef98-ca1d-11ee-95b9-87774f69a715. 2024-02-13T03:44:57.913 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T03:44:57.928 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T03:44:57.942 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-13T03:44:57.944 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7557169/remote/ubuntu@smithi161.front.sepia.ceph.com/crash 2024-02-13T03:44:57.946 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/crash -- . 2024-02-13T03:44:57.983 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-13T03:44:57.984 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-02-13T03:44:57.985 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7557169/remote/ubuntu@smithi183.front.sepia.ceph.com/crash 2024-02-13T03:44:57.986 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/crash -- . 2024-02-13T03:44:57.999 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-13T03:44:57.999 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-02-13T03:44:58.000 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-13T03:44:58.001 DEBUG:teuthology.orchestra.run.smithi161:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(MON_DOWN\)' | egrep -v 'missing hit_sets' | egrep -v 'do not have an application enabled' | egrep -v 'application not enabled on pool' | egrep -v 'mons down' | head -n 1 2024-02-13T03:44:58.042 INFO:teuthology.orchestra.run.smithi161.stdout:2024-02-13T03:27:06.994492+0000 mon.a (mon.0) 808 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-13T03:44:58.043 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-13T03:44:58.044 DEBUG:teuthology.orchestra.run.smithi161:> sudo egrep '\[SEC\]' /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(MON_DOWN\)' | egrep -v 'missing hit_sets' | egrep -v 'do not have an application enabled' | egrep -v 'application not enabled on pool' | egrep -v 'mons down' | head -n 1 2024-02-13T03:44:58.104 DEBUG:teuthology.orchestra.run.smithi161:> sudo egrep '\[ERR\]' /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(MON_DOWN\)' | egrep -v 'missing hit_sets' | egrep -v 'do not have an application enabled' | egrep -v 'application not enabled on pool' | egrep -v 'mons down' | head -n 1 2024-02-13T03:44:58.164 DEBUG:teuthology.orchestra.run.smithi161:> sudo egrep '\[WRN\]' /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(MON_DOWN\)' | egrep -v 'missing hit_sets' | egrep -v 'do not have an application enabled' | egrep -v 'application not enabled on pool' | egrep -v 'mons down' | head -n 1 2024-02-13T03:44:58.228 INFO:teuthology.orchestra.run.smithi161.stdout:2024-02-13T03:27:06.994492+0000 mon.a (mon.0) 808 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-13T03:44:58.228 INFO:tasks.cephadm:Compressing logs... 2024-02-13T03:44:58.228 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-13T03:44:58.271 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-13T03:44:58.284 INFO:teuthology.orchestra.run.smithi161.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-02-13T03:44:58.284 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-13T03:44:58.284 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-13T03:44:58.285 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.0.log 2024-02-13T03:44:58.287 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mon.a.log 2024-02-13T03:44:58.287 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-13T03:44:58.287 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.0.log: /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mgr.y.log 2024-02-13T03:44:58.293 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.1.log 2024-02-13T03:44:58.293 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.cephadm.log: gzip /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mgr.y.log: -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.1.log: /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-13T03:44:58.293 INFO:teuthology.orchestra.run.smithi161.stderr: 89.0% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-13T03:44:58.293 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-13T03:44:58.293 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.2.log 2024-02-13T03:44:58.294 INFO:teuthology.orchestra.run.smithi183.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-02-13T03:44:58.294 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-13T03:44:58.295 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.6.log 2024-02-13T03:44:58.295 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-13T03:44:58.295 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-13T03:44:58.295 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mon.b.log 2024-02-13T03:44:58.295 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-volume.log: /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/tcmu-runner.log 2024-02-13T03:44:58.295 INFO:teuthology.orchestra.run.smithi183.stderr: 81.5% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-13T03:44:58.295 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.7.log 2024-02-13T03:44:58.295 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/tcmu-runner.log: 67.5% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/tcmu-runner.log.gz 2024-02-13T03:44:58.295 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.5.log 2024-02-13T03:44:58.295 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-13T03:44:58.295 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mgr.x.log 2024-02-13T03:44:58.301 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.audit.log: 91.2% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-13T03:44:58.301 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.3.log 2024-02-13T03:44:58.303 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.audit.log: 88.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-13T03:44:58.303 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.4.log 2024-02-13T03:44:58.303 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.log 2024-02-13T03:44:58.303 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.4.log: 91.1% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-13T03:44:58.305 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.2.log: 90.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-13T03:44:58.306 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.log 2024-02-13T03:44:58.308 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.log: 87.5% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-13T03:44:58.308 INFO:teuthology.orchestra.run.smithi183.stderr: 92.3% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-13T03:44:58.310 INFO:teuthology.orchestra.run.smithi183.stderr: 92.0% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mgr.x.log.gz 2024-02-13T03:44:58.311 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.3.log: 95.2% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-13T03:44:58.312 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mon.c.log 2024-02-13T03:44:58.320 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.log: 92.9% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-13T03:44:58.345 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mon.c.log: 92.7% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-13T03:44:58.705 INFO:teuthology.orchestra.run.smithi161.stderr: 90.3% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mgr.y.log.gz 2024-02-13T03:44:59.405 INFO:teuthology.orchestra.run.smithi183.stderr: 92.7% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mon.b.log.gz 2024-02-13T03:44:59.422 INFO:teuthology.orchestra.run.smithi161.stderr: 92.4% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mon.c.log.gz 2024-02-13T03:45:00.872 INFO:teuthology.orchestra.run.smithi161.stderr: 91.8% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-mon.a.log.gz 2024-02-13T03:45:02.753 INFO:teuthology.orchestra.run.smithi183.stderr: 94.0% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.5.log.gz 2024-02-13T03:45:03.104 INFO:teuthology.orchestra.run.smithi161.stderr: 93.9% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.2.log.gz 2024-02-13T03:45:03.107 INFO:teuthology.orchestra.run.smithi183.stderr: 93.9% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.6.log.gz 2024-02-13T03:45:03.350 INFO:teuthology.orchestra.run.smithi161.stderr: 94.1% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.0.log.gz 2024-02-13T03:45:03.366 INFO:teuthology.orchestra.run.smithi161.stderr: 94.1% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.3.log.gz 2024-02-13T03:45:03.431 INFO:teuthology.orchestra.run.smithi183.stderr: 94.1% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.4.log.gz 2024-02-13T03:45:03.723 INFO:teuthology.orchestra.run.smithi183.stderr: 94.0% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.7.log.gz 2024-02-13T03:45:03.724 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-02-13T03:45:03.724 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m5.449s 2024-02-13T03:45:03.725 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m20.315s 2024-02-13T03:45:03.725 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m0.731s 2024-02-13T03:45:03.768 INFO:teuthology.orchestra.run.smithi161.stderr: 94.0% -- replaced with /var/log/ceph/b9b3ef98-ca1d-11ee-95b9-87774f69a715/ceph-osd.1.log.gz 2024-02-13T03:45:03.770 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-02-13T03:45:03.770 INFO:teuthology.orchestra.run.smithi161.stderr:real 0m5.496s 2024-02-13T03:45:03.770 INFO:teuthology.orchestra.run.smithi161.stderr:user 0m23.712s 2024-02-13T03:45:03.770 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m0.926s 2024-02-13T03:45:03.771 INFO:tasks.cephadm:Archiving logs... 2024-02-13T03:45:03.771 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7557169/remote/ubuntu@smithi161.front.sepia.ceph.com/log 2024-02-13T03:45:03.772 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-13T03:45:06.934 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7557169/remote/ubuntu@smithi183.front.sepia.ceph.com/log 2024-02-13T03:45:06.935 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-13T03:45:09.835 INFO:tasks.cephadm:Removing cluster... 2024-02-13T03:45:09.835 DEBUG:teuthology.orchestra.run.smithi161:> sudo cephadm rm-cluster --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 --force 2024-02-13T03:45:11.374 DEBUG:teuthology.orchestra.run.smithi183:> sudo cephadm rm-cluster --fsid b9b3ef98-ca1d-11ee-95b9-87774f69a715 --force 2024-02-13T03:45:12.914 INFO:tasks.cephadm:Teardown complete 2024-02-13T03:45:12.915 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-13T03:45:12.925 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-13T03:45:12.925 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-13T03:45:12.927 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-13T03:45:12.987 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-13T03:45:12.996 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-13T03:45:12.996 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T03:45:12.998 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T03:45:13.019 INFO:teuthology.orchestra.run.smithi161.stdout: remote refid st t when poll reach delay offset jitter 2024-02-13T03:45:13.019 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================== 2024-02-13T03:45:13.019 INFO:teuthology.orchestra.run.smithi161.stdout:*hv01.front.sepi 44.4.53.2 2 u 117 128 377 0.096 1.659 0.295 2024-02-13T03:45:13.019 INFO:teuthology.orchestra.run.smithi161.stdout:+hv02.front.sepi 63.231.80.2 3 u 113 128 377 0.090 2.152 0.275 2024-02-13T03:45:13.019 INFO:teuthology.orchestra.run.smithi161.stdout:+hv03.front.sepi 74.6.168.72 3 u 114 128 377 0.101 0.370 0.295 2024-02-13T03:45:13.019 INFO:teuthology.orchestra.run.smithi161.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-02-13T03:45:13.021 INFO:teuthology.orchestra.run.smithi183.stdout: remote refid st t when poll reach delay offset jitter 2024-02-13T03:45:13.021 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================== 2024-02-13T03:45:13.021 INFO:teuthology.orchestra.run.smithi183.stdout:*hv01.front.sepi 44.4.53.2 2 u 44 128 377 0.083 -0.458 0.233 2024-02-13T03:45:13.021 INFO:teuthology.orchestra.run.smithi183.stdout:+hv02.front.sepi 63.231.80.2 3 u 56 64 377 0.084 -0.198 0.239 2024-02-13T03:45:13.021 INFO:teuthology.orchestra.run.smithi183.stdout:+hv03.front.sepi 74.6.168.72 3 u 44 64 377 0.097 -1.711 0.253 2024-02-13T03:45:13.021 INFO:teuthology.orchestra.run.smithi183.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-02-13T03:45:13.021 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-13T03:45:13.030 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-13T03:45:13.031 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-13T03:45:13.039 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-13T03:45:13.048 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-13T03:45:13.056 INFO:teuthology.task.internal:Duration was 2414.889808 seconds 2024-02-13T03:45:13.057 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-13T03:45:13.065 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-13T03:45:13.066 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-13T03:45:13.068 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-13T03:45:13.119 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-13T03:45:13.119 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-02-13T03:45:13.119 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-13T03:45:13.132 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-02-13T03:45:13.132 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-13T03:45:13.144 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-13T03:45:13.144 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-13T03:45:13.175 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-13T03:45:13.248 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-13T03:45:13.248 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-13T03:45:13.250 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-13T03:45:13.407 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-13T03:45:13.417 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-13T03:45:13.417 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-13T03:45:13.427 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-13T03:45:13.443 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-13T03:45:13.452 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-13T03:45:13.470 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-13T03:45:13.481 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-02-13T03:45:13.497 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-02-13T03:45:13.526 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-13T03:45:13.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T03:45:13.546 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-13T03:45:13.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T03:45:13.573 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-13T03:45:13.582 INFO:teuthology.task.internal:Transferring archived files... 2024-02-13T03:45:13.583 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7557169/remote/smithi161 2024-02-13T03:45:13.584 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-13T03:45:13.620 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7557169/remote/smithi183 2024-02-13T03:45:13.621 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-13T03:45:13.651 INFO:teuthology.task.internal:Removing archive directory... 2024-02-13T03:45:13.651 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-13T03:45:13.658 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-13T03:45:13.694 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-13T03:45:13.705 INFO:teuthology.task.internal:Not uploading archives. 2024-02-13T03:45:13.705 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-13T03:45:13.714 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-13T03:45:13.714 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-13T03:45:13.716 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-13T03:45:13.722 INFO:teuthology.orchestra.run.smithi161.stdout: 659551 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 03:45 /home/ubuntu/cephtest 2024-02-13T03:45:13.743 INFO:teuthology.orchestra.run.smithi183.stdout: 659550 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 03:45 /home/ubuntu/cephtest 2024-02-13T03:45:13.746 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-13T03:45:13.755 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-13T03:45:13.772 INFO:teuthology.nuke:Checking targets against current locks 2024-02-13T03:45:13.799 DEBUG:teuthology.nuke:shortname: smithi161 2024-02-13T03:45:13.799 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-13T03:45:13.824 DEBUG:teuthology.nuke:shortname: smithi183 2024-02-13T03:45:13.825 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-13T03:45:13.829 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7557169', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '18.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 02:56:43.344395', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSAlWjESMc0Gra5f/6dvvbjuYJc2qVK13Ztw1gt4S0hdg9pLl/eAcLsaHQz3OAGEzcbKS0DF4bknjuwe5HHhPE='} 2024-02-13T03:45:13.853 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-02-12_21:56:27-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7557169', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '18.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 02:56:43.342707', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSAlWjESMc0Gra5f/6dvvbjuYJc2qVK13Ztw1gt4S0hdg9pLl/eAcLsaHQz3OAGEzcbKS0DF4bknjuwe5HHhPE='} 2024-02-13T03:45:13.879 INFO:teuthology.orchestra.console.smithi161:Power off 2024-02-13T03:45:13.880 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-13T03:45:13.901 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off 2024-02-13T03:45:13.901 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T03:45:13.917 INFO:teuthology.orchestra.console.smithi183:Power off 2024-02-13T03:45:13.917 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-13T03:45:13.927 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-02-13T03:45:13.941 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2024-02-13T03:45:13.941 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T03:45:13.963 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-02-13T03:45:17.928 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T03:45:17.964 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T03:45:18.038 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-02-13T03:45:18.072 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-02-13T03:45:22.043 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T03:45:22.073 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T03:45:22.153 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off 2024-02-13T03:45:22.183 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-02-13T03:45:22.254 INFO:teuthology.orchestra.console.smithi161:Power off completed 2024-02-13T03:45:26.186 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T03:45:26.295 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2024-02-13T03:45:26.395 INFO:teuthology.orchestra.console.smithi183:Power off completed 2024-02-13T03:45:26.499 INFO:teuthology.run:Summary data: description: rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} duration: 2414.8898084163666 failure_reason: '"2024-02-13T03:27:06.994492+0000 mon.a (mon.0) 808 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log' flavor: default owner: scheduled_lflores@teuthology success: false 2024-02-13T03:45:26.499 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-13T03:45:26.578 INFO:teuthology.run:FAIL